ClairVision - Scalable AI Image Curation & Vector Search Engine
A distributed, backend-first AI image curation system that streams remote images, builds vector embeddings, enables similarity graph clustering, and powers instant face + visual retrieval across events.
Timeline
Oct 2025 – Present
Role
Backend & ML Systems Architecture
Team
3
Status
In developmentTechnology Stack
Key Challenges
- Streaming Large Remote Image Datasets
- Scalable Vector Indexing & Similarity Graph Construction
- Async Pipeline Orchestration
- Low-Latency On-Demand High-Res Streaming
- Cross-Event Identity Matching
Key Learnings
- Vector Database Architecture (pgvector)
- Similarity Graph Modeling
- Async Worker Queue Design
- High-Throughput Ingestion Pipelines
- Backend-Driven ML System Design
Overview
Event photographers often return with thousands of images from a single event. Sorting through them manually is slow, repetitive, and error-prone. Blurry photos need to be removed, duplicates need to be grouped, and finding all photos of a particular person can take hours.
ClairVision was built to automate that workflow.
The system analyzes large image collections, generates visual and facial embeddings, and transforms raw photo dumps into a searchable archive. Instead of scrolling through thousands of files, users can instantly find similar images, identify duplicate shots, and retrieve photos featuring the same person across multiple events.
The Problem
Most event photography workflows still rely heavily on manual review.
Photographers often spend more time organizing images than capturing them. Large albums contain duplicate shots, near-identical frames, out-of-focus photos, and images spread across multiple events. Traditional folder structures make it difficult to discover relationships between images or quickly retrieve specific content.
ClairVision approaches the problem as a search and retrieval challenge rather than a storage challenge.
Key Features
Automated Photo Organization
- Detects visually similar images
- Groups near-duplicate photographs
- Enables large-scale photo clustering
- Reduces manual sorting effort
Instant Visual Search
- Search using image similarity instead of filenames
- Retrieve visually related images in milliseconds
- Explore image collections through semantic similarity
Face-Based Retrieval
- Find photos containing the same person
- Match identities across independent event collections
- Enable cross-event discovery workflows
Built for Scale
- Streams remote images without permanently storing originals
- Processes thousands of images asynchronously
- Supports large-scale vector indexing and retrieval
How It Works
- Images are streamed from remote sources.
- Background workers process and analyze each image.
- CLIP generates visual embeddings for semantic similarity.
- FaceNet extracts identity embeddings for facial matching.
- Embeddings are stored and indexed using pgvector and FAISS.
- Similarity graphs are generated to identify related images.
- Search APIs expose fast visual and identity-based retrieval.
Architecture
The system follows a backend-first design focused on scalability and retrieval performance.
- Python-based computer vision pipeline
- Redis-backed asynchronous worker architecture
- PostgreSQL + pgvector for vector storage
- FAISS for nearest-neighbour search
- REST APIs powering retrieval workflows
- WebGL visualization layer for cluster exploration
Rather than optimizing for image storage, the architecture is optimized for search, discovery, and large-scale image organization.
Current Progress
Completed
- Image ingestion pipeline
- CLIP and FaceNet embedding generation
- Vector indexing with pgvector
- Similarity graph construction
- Visual search APIs
- Cross-event identity matching
- Streaming-based image retrieval
In Progress
- Scaling beyond 10,000+ images
- Distributed worker orchestration
- Hybrid visual + identity ranking
- Dockerized deployment pipeline
- Public demonstration interface
Key Takeaways
ClairVision has been my deepest exploration into computer vision infrastructure, vector search systems, and backend-focused machine learning engineering.
Beyond the machine learning models themselves, the project has been an exercise in designing systems that can efficiently process, organize, and retrieve information at scale.