Ask HN: Why does every AI demo sound perfect but real world deployment always
AI demos often sound perfect due to controlled environments and hand-crafted prompts. However, real-world deployment of AI systems is more challenging due to various factors such as background noise and poor connections. Real-world AI deployment requires additional efforts to establish evaluations, identify loss patterns, and process user feedback.
- ▪AI demos are often hand-crafted for specific scenarios.
- ▪Real-world AI deployment is more difficult due to various environmental factors.
- ▪Establishing human and offline evaluations is crucial for real-world AI deployment.
Opening excerpt (first ~120 words) tap to expand
You don't need quality or flexibility for an AI demo.Hand crafting a voice agent to schedule simple appointments for a barber in San Francisco where the caller is in a quiet environment, is a one day exercise in prompt engineering.Building a voice agent to schedule real appointments (on a real calendar of a working business) for real customers, for any business type, in any city: significantly more difficult. Real customers can be on a bad cell connection, have background noise, or worse, there's somebody in the background having another conversation.Building a working agent isn't the hard part of building a real world agent.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Ycombinator.