I spent last week on two floors of the same building. Upstairs, clinical teams talked about care gaps, no-shows, and the quiet panic of a nurse with forty callbacks and no time. Downstairs, engineers demoed models that could hold a conversation indistinguishable from a person. The gap between those two floors is the whole problem.
Everyone in voice AI right now is selling the downstairs. The model. The latency. The voice that sounds human. And it's genuinely impressive, the tide really has risen for everyone. But healthcare doesn't buy a model. It buys an outcome it can defend to a compliance officer, an outcome that survives an audit, an outcome that a nurse trusts enough to hand a patient to.
The demo that lands and the system that ships are different things
The most useful thing I heard all week came from a clinical operations lead who'd sat through a dozen vendor demos. "They all sound amazing for ninety seconds," she said. "Then I ask what happens when the patient says something the script didn't expect, and the room goes quiet."
That's the line between a demo and a system. A demo optimizes for the happy path. A system has to handle the patient who's confused, the wrong person picking up, the consent that was never captured, the state whose rules differ from the one next door.
"Generative AI is great at conversation. It's terrible at compliance. The work is sequencing the two so neither one breaks." From the engineering floor
Three things the clinical floor kept repeating
Stripped of jargon, the asks were remarkably consistent across every team we met:
- Don't make me trust a black box. Every call needs a trail, what was asked, what was disclosed, what was consented, exportable for an auditor.
- Don't let the AI freelance on the things that matter. Disclosures and screening questions must be asked the same way, every time.
- Hand off cleanly. When a human is needed, the human should arrive with full context, not a cold transfer.
Why we build compliance-first, not model-first
The instinct in this market is to start with the most capable model and bolt guardrails on afterward. We do the opposite. The compliance engine runs first; generative AI only fires inside the states it's explicitly allowed to, and never with direct access to sensitive data. It's a less flashy demo. It's a far more defensible system.
That's the conversation healthcare needs to have, not "how human does it sound," but "what happens on call number four hundred thousand, in the state with the strictest rules, when the patient says something nobody scripted." Answer that, and you've earned the right to be on the phone.
We left Orlando with a clearer picture of what platforms actually need next, and a few new partners who want to build it with us. More on that soon.