RecipeWalk

Challenge

Cooking from a phone is a hands-busy problem that recipe blogs make worse: ads, life stories, and a wall of text between you and the next step. The goal was to strip a recipe to clean structured data and then let a real-time voice assistant walk the cook through it, hands-free, without the “ChatGPT forgetfulness” failure mode where the model loses track of which step you are on.

Process

A pasted or photographed recipe is parsed by Gemini 2.5 Flash into structured ingredients, steps, and timers. The voice layer is the interesting architectural choice: it runs client-direct to the Gemini Live API over WebSocket, not proxied through the backend, which keeps latency and cost down. The backend owns a server-side state machine that hands the model step-scoped system instructions and issues single-use, scoped, ephemeral session tokens, so the assistant always knows exactly where the cook is and cannot wander.

RecipeWalk is also the workspace’s reference implementation for patterns other projects copy: the secrets-loading pattern (secrets pulled from AWS at service start into a locked-down env file), the IAM deployer scoping, and the redeemable-gift security model (high-entropy codes encrypted at rest, atomic single-use redemption, IDOR-hardened, auto-revoked on refund or dispute). A bespoke voice-eval harness feeds synthesized audio through the exact production voice config to catch tool-calling regressions, and it caught a real one before it shipped.

Result

Built and deployed to production at recipewalk.com, with the full test suite green (277 backend, 120 frontend). The gift-purchase flow has been proven end-to-end with a real-money buy, redeem, refund, and revoke cycle. It is not yet opened to customers; the remaining step to a paid beta is a deliberate go-live decision, not a missing feature.