AI Pipeline Backend & Interactive Segmentation Tool

The Brief

A platform client had existing ComfyUI workflows for background replacement and product/person swapping in images. The bottleneck: accurate masking. Text-based prompting alone couldn't reliably isolate the exact product or region that needed editing — especially for non-technical end users of their platform.

They needed a better input mechanism, and wanted to explore what was technically possible before committing to a full build.

My Approach

The goal was a prototype that let users click or draw on an image to select an object, with that selection automatically converted to an accurate segmentation mask — which then feeds into the existing ComfyUI backend.

I scoped this as an exploration: test the technical approach, validate the UX, and produce something hand-off-ready for their engineering team if it proved viable.

What Was Built

• A lightweight web interface for object selection directly on an image
• A server-side segmentation pipeline that converts the selection into a clean, accurate mask
• API interfaces structured so the prototype can be integrated into their existing platform by their own engineer

The result: their end users can now select a product in an image by clicking on it, rather than describing it in a text prompt — dramatically improving mask accuracy and removing the prompt-writing barrier for less technical users.

Outcome

The prototype validated the technical approach and is ready for integration. It significantly reduces the failure rate of their masking step and opens the workflow to a broader, less technical user base.