AI Pipeline Backend & Interactive Segmentation Tool

The Brief

A platform client had existing ComfyUI workflows for background replacement and product or person swapping in images. The bottleneck was accurate masking. Text-based prompting alone could not reliably isolate the exact product or region that needed editing, especially for the non-technical end users on their platform.

They needed a better input mechanism, and wanted to explore what was technically possible before committing to a full build.

My Approach

The goal was a prototype that let users click or draw on an image to select an object. That selection is automatically converted to an accurate segmentation mask, which then feeds into the existing ComfyUI backend.

I scoped this as an exploration: test the technical approach, validate the UX, and produce something hand-off-ready for their engineering team if it proved viable.

What Was Built

• A lightweight web interface for object selection directly on an image
• A server-side segmentation pipeline that converts the selection into a clean, accurate mask
• API interfaces structured so the prototype can be integrated into their existing platform by their own engineer

The result is that their end users can now select a product in an image by clicking on it, rather than describing it in a text prompt. Mask accuracy improved considerably and the prompt-writing barrier is gone for less technical users.

Outcome

The prototype validated the technical approach and is ready for integration. It significantly reduces the failure rate of their masking step and opens the workflow to a broader, less technical user base.