Fan-out: how to open voice, whether it sees images, the API name, latency, and how it splits from Astra.
Realtime voice in ChatGPT#
Open voice mode on a supported client and allow the mic. Interrupt naturally; drop images into the same thread and say “this button.” Plus and higher hold quota more steadily. If an enterprise workspace disabled voice or files, ask an admin—do not fight the browser.
When the Realtime API is the right pipe#
If you own the microphone stream, use Realtime instead of polling chat text. The session is bidirectional audio; tools still go through functions. Keep secrets on a server or a short-lived client token, never in the frontend repo.
Three easy UX failures#
- Barge-in: the old reply must stop when the user talks over it.
- Deixis: camera frames and uploaded images do not share a coordinate system—say “current view.”
- Weak networks: downsample then reconnect; do not execute a half sentence.
Split from deep-reasoning models#
Live calls want low latency. Do not run o3 or Astra max on every turn. Let 4o / realtime hear intent, hand hard jobs to Sol or Astra, then summarize in speech. One pipeline, not a voice-only microsite.
Takeaways#
- The 4o family is low-latency multimodal, not the strongest reasoner.
- Product uses voice mode; apps use Realtime; keys stay off the frontend.
- Ship barge-in, deixis, and weak-net tests before arguing about timbre.