AI Pipeline Backend & Interactive Segmentation Tool

The brief

A platform client had existing ComfyUI workflows for background replacement and product or person swapping in images. The bottleneck was accurate masking. Text prompting alone couldn't reliably isolate the exact product or region that needed editing, especially for the non-technical users on their platform.

They needed a better input mechanism, and wanted to see what was technically possible before committing to a full build.

My approach

The goal was a prototype where users click or draw on an image to select an object. That selection gets converted into an accurate segmentation mask, which then feeds into the existing ComfyUI backend.

I scoped it as an exploration: test the technical approach, validate the UX, and hand over something their engineering team could build on if it proved viable.

What I built

• A lightweight web interface for selecting objects directly on an image
• A server-side segmentation pipeline that turns the selection into a clean, accurate mask
• API interfaces structured so their own engineer can integrate the prototype into the platform

Their end users now select a product by clicking on it instead of describing it in a text prompt. Mask accuracy went up a lot, and the prompt-writing barrier is gone.

Outcome

The prototype proved the approach and is ready for integration. The masking step fails far less often, and the workflow now works for people who would never write a prompt.