Case study
A food-scanning prototype that spans a mobile app, a web client, and a Python inference service. A user photographs a food item; a multi-task ResNet-50 classifies it, and the backend attaches basic nutritional and Ayurvedic (Dosha) reference information where the reference data supports it. It is deliberately framed as a prototype, not a clinical or nutritional authority.
The goal was to build one inference service and put it behind real clients - a camera-first mobile experience and a browser experience - rather than a single script. That forced the interesting engineering work: bounding memory on a CPU host, validating uploads, and returning a structured response that both clients can render.
Each scan flows through a single documented endpoint, POST /api/v1/scan/food, which returns the recognised food, a confidence value, optional reference information, and model metadata.
Model accuracy for the 101-class problem is not yet verified, and the application is not a clinical or nutritional authority. The nutrition and Ayurveda fields are advisory reference data, not instructions.
Three surfaces, one inference service. The clients stay thin; all recognition and mapping happens in the backend.
The mobile app captures a photo with Expo Camera; the web client accepts an uploaded image file.
ImplementedThe image is posted as multipart form data to /api/v1/scan/food. JPEG, PNG, and WebP are accepted up to 5 MiB; other content types and oversized files are rejected.
The multi-task ResNet-50 runs on CPU. The model loads lazily on the first request under a lock, so startup stays quick and the memory spike is bounded.
ImplementedThe response builder attaches nutritional and Ayurvedic (Dosha) reference data from a static mapping file. When an item has no mapping, the field reports that information is unavailable instead of inventing values.
ImplementedThe API returns JSON containing the food label, a confidence value, the optional reference sections, and the model name and version.
ImplementedBoth clients display the result. The mobile API client adds retries with backoff, request timeouts, and secure storage for its configuration.
Implementedmtl_model_latest.pth, about 95 MB) with a Food-101 class listexpo-cameraThe PyTorch model consumes substantial RAM. The backend is deliberately pinned to one Uvicorn worker to avoid out-of-memory failures on small hosts, trading concurrency for reliability.
The ~95 MB model is loaded on first use rather than at import. That keeps cold start fast and avoids duplicate loads under concurrent requests.
Nutrition and Dosha data come from a small static mapping file. Unsupported foods return a "not available" note rather than fabricated advice.
Running on CPU avoids a GPU requirement and keeps the image portable; the trade-off is inference time of roughly 0.1 to 0.3 seconds per image after the initial load.
The repository also contains separate experiment code for a quantum-circuit classifier. It is not part of the scan API or the primary runtime and is noted here only for completeness.
The fastest path is Docker Compose, which runs the full stack locally:
docker compose build
docker compose up -d
docker compose psThe web client is then on http://localhost:8080 and the API on http://localhost:8000. To run natively instead:
cd 01_Backend
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
cp .env.example .env
uvicorn app.main:app --host 0.0.0.0 --port 8000The web client starts with npm install and npm run dev in 02_Frontend; the mobile app starts with npm install and npx expo start in 03_Mobile, pointed at a reachable LAN address for a physical device.
Like the other builds here, SecretEye pairs an applied model with real clients; it shares its Python service style with the QBioForge and microplastic projects.