10 Voice Agents in 10 Days: How I Survived the Murf AI Challenge

They say thousands start, but only a few finish. Specifically, in the Murf AI Voice Agents Challenge, only 321 builders crossed the finish line. I just became one of them.



For ten straight days, I committed to "building in public." My goal wasn't just to make Hello World demos; I wanted to push the boundaries of what real-time Voice AI can do building agents that remember context, handle complex business logic, and recover gracefully when APIs fail.

Here is a breakdown of my journey, the technical hurdles I overcame, and why consistency is the ultimate engineering skill.


🚀 The Mission

The challenge was simple but grueling: Build 10 functional AI Voice Agents in 10 days. My toolkit:

  • Voice: Murf Falcon (for ultra-low latency TTS).

  • Brain: Google Gemini (for reasoning and intent extraction).

  • Frontend: LiveKit / Custom React UIs.

  • Backend: Python (Flask/FastAPI).

Here is how my architecture evolved from simple scripts to complex, fault-tolerant systems.


Phase 1: Foundations & State Management (Days 1-3)

The early days were about mastering the "Loop": Speech → Text → Reasoning → Voice.

Day 1: The Hello World

I started by setting up the starter repo and connecting the backend to the frontend. The focus was purely on latency getting that "real-time" feel using Murf Falcon.

  • Milestone: Achieved a lag-free conversation loop in the browser.

Day 2: The Starbucks Barista (Structured State)

A voice agent is useless if it forgets what you ordered 10 seconds ago. I built a State-Aware Barista that maintains a structured JSON object (drink, size, milk, extras).

  • Tech Highlight: I implemented a JSON Reasoning Loop where Gemini returns both the natural language reply and the updated state object simultaneously.

Day 3: The Wellness Companion (Long-Term Memory)

I shifted from transactional (ordering coffee) to relational interactions.

  • Tech Highlight: Introduced Long-Term Memory via a wellness_log.json. The agent now reads past moods before generating a greeting, allowing for context-aware conversations ("You were feeling low yesterday, how are you today?").


Phase 2: Business Logic & Real-World Integration (Days 4-7)

Here, I moved from "chatbots" to "agents that do work."

Day 4: Active Recall Coach (Multi-Modal Logic)

I built an AI Tutor with three distinct modes: Learn, Quiz, and Teach-Back.

  • Tech Highlight: Dynamic Voice Switching. The agent swaps voice profiles (personas) based on the active mode (e.g., a strict voice for quizzing, a supportive voice for teaching) to psychologically cue the user.

Day 5: Zomato SDR (CRM Integration)

I simulated a Sales Development Rep that stealthily qualifies leads.

  • Tech Highlight: Stealth Data Extraction. Instead of asking boring form questions, the agent weaves qualification queries (Team Size, Budget) into natural conversation. The frontend updates a live CRM dashboard in real-time as the AI extracts entities.

Day 6: HDFC Fraud Alert (Security)

A high-stakes banking agent that handles sensitive data.

  • Tech Highlight: Read/Write Database Logic. The agent interacts with a suspicious_transactions.json file. It enforces strict 2FA (verifying the last 4 digits of a card) before revealing any data, mimicking real banking security protocols.

Day 7: Blinkit Grocery Agent (Intent vs. Keyword)

  • Tech Highlight: Fuzzy Intent Extraction. I moved beyond keyword matching. If a user says "I need ingredients for pasta," the agent parses the intent and auto-bundles the correct items (Pasta + Sauce + Cheese) into the cart using a tagged grocery_catalog.json.


Phase 3: Advanced Architecture & Resilience (Days 8-10)

The final stretch was about immersion and engineering robustness.

Day 8: Cyberpunk Game Master

I built a D&D-style narrator with a dark neon UI.

  • Tech Highlight: Narrative Consistency. The agent tracks inventory across turns. If you pick up an ID card in Turn 1, the agent remembers you have it in Turn 5, allowing for non-linear storytelling.

Day 9: E-Commerce & ACP Lite

I built a shopping assistant inspired by the Agentic Commerce Protocol (ACP).

  • Tech Highlight: TTS Sanitizer. Text-to-Speech engines often crash on symbols like ₹ or parentheses. I wrote a backend "sanitizer" layer that converts symbols to text (e.g., ₹1500 → "1500 Rupees") before sending the payload to the TTS API, preventing audio failures.

Day 10: The Improv Battle (Engineering for Failure)

For the finale, I built a Voice Improv Game. But the real challenge was handling API 500 Errors.

  • Tech Highlight: Hybrid Voice Fallback System.

    • Primary: Try Murf API for high-quality voice.

    • Fallback: If Murf fails/times out, the backend returns a null audio flag.

    • Recovery: The client detects the null flag and instantly switches to the browser’s native SpeechSynthesis.

    • Result: Zero dead air. The show goes on even if the server crashes.

🏆 The Takeaway: Momentum Beats Motivation

Participating in this challenge wasn't just about learning a specific tool; it was about the discipline of Shipping Daily.

When you build 10 distinct architectures in 10 days, you stop treating errors as roadblocks and start treating them as data. You learn that great AI isn't just about the LLM it's about the State Management, the Latency Optimization, and the Fallback Architectures that wrap around it.

I’m excited to take these lessons in resilience and architecture and apply them as I continue my journey in cloud-native development.

🔗 Check Out the Code & Demos

This entire journey all 10 days, 10 agents, and every line of code is documented publicly. If you want to see these agents in action or dig into the architecture, here is where to find them:

📺 Watch the Demos on LinkedIn I posted daily video updates showcasing the UI, the voice interactions, and the real-time responses. 👉 Click Here for My LinkedIn 10th Day Post (Tip: Once you are on my profile, scroll down through my recent posts. You will find them in reverse order starting from Day 10, then Day 9, etc.)

💻 Get the Source Code on GitHub Want to see how I handled the JSON state management or the fallback logic? The complete code for all 10 agents is available on my GitHub. 👉 Visit My GitHub Profile (Tip: Just search for "day" in my repositories tab to see the full list of all 10 agent projects.)

Next Stop: Applying these "Build in Public" principles to the AWS Community Builders Program. 🚀

Comments

Post a Comment

Popular posts from this blog

🧠 How I Built a Sudoku Game Using Amazon Q CLI — With Just Prompts!

Image
What happens when you mix a strong idea with a smart AI CLI? You get a fully working game — without writing a single line of code yourself. Here's how I, Kavya Trivedi , built a complete Sudoku game using Amazon Q CLI , step by step. 🚀 Quick Links 🎥 Watch the Game in Action 📦 View the Source Code on GitHub 📲 See the LinkedIn Carousel Post 💡 Why I Built This As a developer passionate about trying emerging tools, I, Kavya Trivedi , recently explored Amazon Q CLI , a GenAI-powered CLI by AWS that helps developers build applications with natural language. I was curious: Could I actually build something real — like a game — just by describing it? Could GenAI handle logic, UI, and interactivity? So I put it to the test. The result? A complete Sudoku game — no manual coding required. ⚙️ Setup Phase — The Toughest Part Getting started with Amazon Q CLI wasn’t easy. I spent nearly 2 hours setting things up — here's what I did: Signed up for an AWS Builder ID ...
Read more

I Built a Sci-Fi "Universal Translator" for the 2026 Hackathon

Image
The Problem: Why Do Translators Sound So... Dead? 💀 We’ve all used translation apps. You pour your heart out, say something meaningful, and the app spits back a robotic, monotone voice that sounds like a GPS from 2005. The vibe? Instantly killed. For the 2026 Hackathon , I wanted to build something different. I didn't want a tool that just swaps words; I wanted an agent that understands context and speaks with feeling . Meet LingoVoice AI .  The Tech Stack: The Best of Both Worlds   To build a "Next Level" agent, I needed the best tools for two specific jobs: The Brain (Context): Lingo.dev . Most APIs just translate text literally. Lingo.dev understands context , technical jargon, and cultural nuance. The Voice (Emotion): Murf AI Gen2 . These aren't standard text-to-speech voices. They breathe, pause, and have specific accents (like Enrique for Spanish or Baolin for Chinese). But there was a catch.   My backend logic was in Python (Flask) . The Lingo.dev SDK is...
Read more