Real-Estate Lead Company

A real-time voice sales agent on a phone number: a rented model behind an eval-gated brain, deployed with a kill switch and a pinned rollback.

An inbound phone agent for a real-estate lead company: ~517 ms to first audio on a real call, a 16-scenario eval gating every prompt change, and every call persisted as a redacted record.

Problem

The problem.

A real-estate lead company sells a subscription product over the phone: every call is a live sales conversation with real objections, price pushback, and qualification rules that matter. Business owners have learned to distrust voice AI, and they are right to: most demos are canned recordings or a laggy chatbot with a voice. The questions that decide whether an agent can take real calls are engineering questions. How fast does it speak? Does it hold the rules it was given when a caller pushes? And what happens when it gets something wrong?

Approach

How I built it.

Bridge Twilio Media Streams to the Gemini Live API on Cloud Run and measure the thing callers actually feel: ~517 ms to first audio on a real call, against ~1.7 to 2.3 s on the alternative model. An audio playback backlog that made latency snowball over long calls was found and fixed; per-turn latency now stays flat.
The agent's brain is a hand-authored knowledge wiki (product facts, a 16-objection library, discovery and qualify/disqualify branches) compiled into a lean system prompt. Facts the client has not yet confirmed are quarantined behind a verify-with-owner gate: the agent defers them to a human instead of speaking them.
A repeatable eval harness simulates 16 caller scenarios (price-only shoppers, aggressive negotiators, out-of-state callers, opt-outs) and scores every prompt change with an LLM judge on compliance, qualification discipline, and outcome logging before it ships. Knowledge enrichment lifted eval compliance from 71% to 93%; the shipping brain passes compliance in all 16 scenarios.
Two release gates run before traffic moves: an automated call-quality gate on the compiled prompt, then an adversarial deploy review. Every rollout pins a rollback revision, and a documented kill switch can pull the agent off the number.
Every call writes a redacted record to Firestore and a stats row to BigQuery, off the latency path, so the client gets call analytics and prompt fixes can be checked against real calls.
Load-tested the model at 50+ concurrent live sessions, which de-risks the designed scale phase: thousands of calls a day needs roughly 22 concurrent. Outbound calling stays off until a consent, calling-hours, and do-not-call compliance engine is built. That is the order production telephony should ship in.

Architecture

What's running, and why.

Telephony bridge

Twilio Programmable Voice + Media Streams: Inbound calls stream audio over a websocket to a Python bridge service.
Cloud Run bridge service: Kept warm with min-instances and one call per instance, so first audio lands in ~517 ms instead of a cold-start pause. Secrets live in Secret Manager.

Brain

Knowledge wiki, compiled to a prompt: Hand-authored product facts, a 16-objection library, and qualify/disqualify branches, distilled into a lean system prompt for the Gemini Live API.
Verify-with-owner quarantine: Claims the client has not confirmed are fenced off. The agent defers them to a human rather than improvising terms.

Eval + release gates

16-scenario eval harness: Simulated callers run each scenario against the real model; an LLM judge scores compliance, qualification discipline, call flow, and outcome logging per run.
Two gates, pinned rollback, kill switch: A call-quality gate and an adversarial deploy review both pass before rollout. Each deploy pins a rollback revision; a documented kill switch pulls the agent off the number.

Persistence + analytics

Redacted call records: Every call writes a redacted record to Firestore and a stats row to BigQuery, off the latency path, with analytics queries over outcomes.

Outcome

What shipped.

~517 ms

To first audio on a real call, measured. The alternative model measured ~1.7 to 2.3 s.

71% to 93%

Eval compliance lift from knowledge enrichment. The shipping brain passes compliance in all 16 scenarios.

16 scenarios

Simulated-caller eval harness, LLM-judged. Every prompt change is scored before it ships.

~4.3 / 5

Qualification-discipline average across the eval suite. The current weak spot, still being hardened.

50+

Concurrent live sessions handled in load testing. The designed scale phase needs roughly 22.

Every call

Persisted as a redacted record plus an analytics row, off the latency path.

Pending

Production call volume and booked-outcome rate. Outbound stays off until the consent, calling-hours, and do-not-call engine ships.

Stack

Python
Gemini Live API
Twilio Programmable Voice
Twilio Media Streams
Cloud Run
Secret Manager
Firestore
BigQuery

What this proves

Where this ports.

Voice AI is buyable when it is engineered like software, and the model is the rented part. What the client owns is the harness: the setup around the AI that connects it to their systems, enforces their rules, catches mistakes, and remembers what works. The same pattern (eval-gated prompt changes, two release gates, redacted call records, a kill switch) ports to any business that answers or makes phone calls: intake, scheduling, qualification, follow-up.

Want the same pattern in your business?

Thirty minutes. Describe what you're working on and I'll tell you if I can ship it.

Tell me what you're working on ← All case studies