An AI vision pipeline that refuses to guess
Nutrition tracking fails because logging a meal costs more effort than the insight returns. FitRas replaces manual entry with a photo — and gates the answer when the model isn't sure.
The challenge
Food recognition is easy to demo and hard to trust. A model will confidently return macros for a portion it cannot actually see, and a confidently wrong calorie count is worse for the user than no answer at all. Regional cuisine makes it harder still, since the food databases most apps depend on barely cover it.
The approach
- Split recognition and nutrition calculation into two distinct stages so each can be evaluated separately
- Introduced a portion gate: when the quantity is ambiguous, the API returns a needs-clarification response instead of a number
- Checked a local food database on the fast path before falling through to model calculation
- Wired the pipeline into CI with fixed accuracy thresholds that fail the build when breached
- Layered a pgvector memory system underneath the coaching agents so advice reflects the user's actual history
What exists now
- A two-step vision pipeline running in production code across web, iOS and Android
- Recognition quality enforced in CI: success rate at or above 90%, wrong-accept rate at or below 2%
- Three specialised AI agents with separate prompts, memory namespaces and rate limits
- Dashboard load consolidated from several API calls into one cached, deduplicated request
- Not publicly launched — no adoption figures are claimed