SurfacesRoom 05 of 13
Say it out loud. The same gates apply.
Voice is another way into the runtime you already run, not a second assistant. A spoken request is queued and approval-gated the same way as a typed message and appears live in the chat thread. High- or Critical-risk actions need you to say “yes, approve”. On every tier.
Recreation of the Sophon Voice page, request “Delete the weekly metrics digest schedule.”. Now: A bare “yes” is asked again. Voice status Awaiting approval. Said: You: Yes. Sophon · browser voice: Sorry, I didn’t catch that. This needs you to say ‘yes, approve’, or no to reject. Note: High risk: a bare “yes” is asked again.
11 / 16 · A bare “yes” is asked again
Recreated from the Dashboard’s Voice page with conversation mode on, Deepgram for speech-to-text and a text-to-speech provider. Lines under the frame caption what is said aloud, plus a few notes; none of them is on the page. Approval prompts are read in the browser’s own voice, and a reply spoken by a provider is heard, not printed: its text lands in the chat thread.
AWatch risk change what counts as yes
Risk decides what counts as yes.
Which actions ask at all is set by your approval policy, the same one typed requests answer to. Voice changes how you answer: Sophon reads out the kind of action and its risk, and sets a higher bar for the riskier ones.
Voice page approval, High risk: Delete a scheduled job permanently. Sophon · browser voice: I need your approval. Delete a scheduled job permanently. This is high risk. Say ‘yes, approve’ to approve, or no to reject. Result: Awaiting your answer.
Step 1 of 6: The prompt is read out
01 / 06 · The prompt is read out
On the Voice page and the Command Bridge, Sophon closes the mic and reads out the kind of action and its risk, in your browser’s built-in voice, not your text-to-speech provider’s. Turn that voice off and you get the card and the mic instead.
It names the kind of action, not its exact details. When the details matter, read the card before you answer.
What you can say
| You say | Low or Medium | High or Critical |
|---|---|---|
| “Yes.” | Approved | Asked again |
| “Yes, approve.” | Approved | Approved |
| “Yes… wait.” | Asked again | Asked again |
| “No.” | Rejected | Rejected |
Asked again means Sophon repeats what it needs, at most twice, then waits for a tap or a mic press.
01A voice turn
A spoken request is an ordinary task.
Sophon runs what you say as a regular agent task on your chat, stores it in the thread, gates it with the same approvals as a typed message, and then speaks the answer.
- 01
Transcribed
A pause or a press ends it; your speech-to-text provider or your browser turns it into text.
- 02
Queued
It joins that chat’s queue like a typed message and, by default, waits behind anything already running.
- 03
Gated
A tool that needs approval stops and asks before it runs; on the Voice page, out loud.
- 04
Spoken and kept
The answer is spoken, and your words and the reply are saved in the chat thread.
- Same queue, same thread
A voice request runs as a regular agent task on your chat session. It is queued and approval-gated the same way as a typed message and appears live in the chat thread.
Voice turns use a smaller, voice-focused tool set, so some jobs are better typed.
- Failures are said, not swallowed
If a voice request fails, Sophon tells you by voice or on screen instead of going quiet.
A turn you interrupted is the exception: it stays silent by design, and its text still reaches the chat.
- Interrupting silences the reply, not the work
On the Voice page, press the mic while Sophon is thinking or speaking to silence it and talk. The interrupted reply isn’t spoken later, but it still appears in the chat.
Interrupting stops the speech, not the task: work already under way keeps running, and by default your next request waits behind it. There’s no talk-over detection; interrupting is always a press.
02When the agent asks
Numbered choices. Nothing guessed.
When an agent asks a question with options, the Voice page and the Command Bridge read it out in your browser’s voice with numbered choices. Answer by saying the number (one to four) or the option’s name, or tap it.
- One question at a time
Multi-part questions are asked one at a time, and your answers are sent together at the end. If you can pick more than one, Sophon says so.
- Say it the way you’d say it
Option names with symbols are read out and matched as spoken words, so you can say “C plus plus” or “C sharp”.
If your speech recognizer writes the symbol instead of the words, Sophon asks again; tap the option.
- “Don’t” means don’t
If you negate an option (“don’t send”), Sophon won’t pick it. It asks again, or passes your own words to the agent when the question accepts a free-text answer.
- Short answers for numbers
A spoken number counts in a short answer. Options past the fourth are picked by name or tap.
Question cards come from plans and other tasks on the chat: a voice turn is set up to ask its own follow-ups out loud, in conversation, rather than as a card. Sophon Mobile shows questions on screen and doesn’t read them out.
A Sophon Voice question card: question one of two, “Which calendar should the review go on?”, with the options Work, Personal and Board. It is answered out loud or by tapping an option.
A still of the Voice page’s question card.
03One decision, recorded once
Answer anywhere. The rest close.
An approval is one record on your Gateway, not a dialog on one screen. The first answer decides it.
Where a prompt is asked out loud
- Voice page
Approvals and questions are read out in your browser’s voice, with the card and its buttons on screen. Answer by voice or tap.
- Command Bridge
Prompts are read out in your browser’s voice. Pressing the mic while one is waiting opens the mic so you can answer it out loud. A separate Stop button turns voice off without answering, and the prompt stays on its chat card.
- Chat voice ribbon
Never reads prompts aloud. Answer on the chat’s own cards.
- Sophon Mobile
Reads approvals and applies the same “yes, approve” rule, for its own voice conversation only. Questions are answered on screen.
Voice only speaks prompts that belong to its own conversation, so an approval from an unrelated session never interrupts you.
How it ends
- Closed everywhere
Answer an approval in one place, whether by voice, on a chat card, in another tab or in the mobile app, and any other open chat card or voice prompt for it closes on its own.
The Dashboard’s Approvals page catches up on its next refresh.
- Recorded exactly once
Each approval is recorded exactly once, so when two devices answer the same card only one answer counts, and an approval that timed out or was cancelled can never later become an approval.
- Says how it ended
Approval cards on the Dashboard and Mobile say how an approval ended: Approved, Rejected, Cancelled or Timed out.
If a plan pauses for your approval and finishes later, its result is spoken to a voice session still open on that chat. If you’re mid-conversation at that moment, it appears in the chat thread instead.
04What changed in 2.0
Same orb. New engine.
If you used voice in 1.x, the Voice page will look familiar: the look barely changed. The work went underneath: one voice engine behind the Voice page, the chat ribbon and the Command Bridge, and a voice session that belongs to the conversation, not to a browser connection.
- 01
One engine
Listening, playback and interrupting follow the same rules on the Voice page, the chat ribbon and the Command Bridge.
- 02
A session that holds
A dropped connection rejoins the reply in progress; audio sent while you were offline isn’t replayed.
- 03
Every tab hears it
Every tab attached to the voice conversation hears the reply.
- 04
A clean exit
Leave voice and the microphone is released, every time.
- Drop the connection, keep the conversation
If your connection drops while Sophon is answering, voice picks back up when you reconnect: you hear the rest of the reply from that point, and the full reply is in the chat.
Audio sent while you were offline isn’t replayed.
- Every tab on the conversation
Every open tab attached to the same voice conversation hears the reply.
Ending voice in one tab ends it in all of them, and each visit to the Voice page opens a new conversation.
- Nothing looks fine when it isn’t
Every voice error shows in a strip across the top of the Voice page, and the microphone is released every time you leave voice, so there’s no lingering recording indicator.
- Transcripts matched to the moment
With server speech-to-text, each transcript is matched to what you said when it was said, so a slow transcription of an earlier sentence is much less likely to be taken as your answer or as a new request.
Where you can talk to it
- Voice page
Full screen: the orb, a preview of what you said, and approval and question cards you answer out loud or with a tap.
Each visit starts a new conversation with the default agent.
- Chat voice ribbon
Talks on the chat you have open and uses your speech-to-text provider when one is active. Switching chats ends voice on the old chat and restarts it, listening, on the new one, so replies aren’t spoken into the wrong conversation.
It never reads prompts aloud.
- Command Bridge
The Dashboard’s full-screen fleet view has a hands-free voice bar that listens once you arm it. When it stops (after silence, inactivity or an error), it switches itself off and shows why.
Hands-free only, with no push-to-talk; each arm starts a new conversation with the default agent.
- Sophon Mobile
The Voice tab uses your phone’s own speech recognition by default, speaks replies in your provider’s voice or the device voice, and Conversation mode listens again after each reply.
Questions are answered on screen, not read aloud.
- CLI
Manages voice (providers, settings and sophon voice status) but has no voice conversation of its own.
05Listening and speaking
Hands-free with any provider. Or none.
Conversation mode works end to end with any provider. Your browser’s own speech is enough to start, and with a text-to-speech provider Sophon starts talking before the reply is finished.
Listening
- Conversation mode
Turn on Conversation mode in Voice settings and your first mic press starts a hands-free conversation: speak, pause, hear the answer, and the mic reopens only after the reply has finished playing. Silence and inactivity limits close the mic when you walk away.
It isn’t a wake word or always-on listening: it starts with your press.
- Any speech-to-text provider
Hands-free works with any supported speech-to-text provider, not just ones with built-in end-of-speech detection: Sophon detects the pause itself.
With no provider, your browser’s recognition ends each utterance at a pause, in Chrome or Edge.
- Words as you say them
In the Dashboard, with Deepgram as the speech-to-text provider, your words appear on screen while you’re still speaking. Your browser’s recognition shows them live too.
Other providers transcribe once you stop, one round trip per utterance.
SpeakingNeeds a text-to-speech provider
- A sentence at a time
With a text-to-speech provider configured, Sophon starts speaking a reply sentence by sentence while it is still writing the rest of it.
A one-sentence reply is spoken once it’s complete. Without a provider, your browser’s voice reads the whole reply at the end.
- Progress, out loud
During long, multi-step tasks, Sophon can speak a short progress line after a quiet stretch, for example when it moves on to a new step. Long silences are less likely to feel like a dropped connection.
Needs a text-to-speech provider. A single long step can stay quiet for its whole run.
- Formatting stays on screen
When a text-to-speech provider streams the reply, Sophon strips Markdown formatting before speaking and reads links as their text or site name.
Code and tables are best read in the chat.
While a text-to-speech provider speaks, the Voice page shows your words and the orb, not the reply. The full reply is written to the chat thread.
06For whoever runs the Gateway
When voice breaks, it says where.
Nothing is preconfigured. With no speech provider, voice uses your browser’s speech recognition and voice (Chrome or Edge). Add speech providers for server-side transcription and provider voices.
- Not set up, or failing
When voice can’t use a speech provider, the Voice page explains why in plain words: either speech isn’t set up and your browser’s speech is used, or the configured provider is failing, and it names that provider.
The notice appears only when no working provider of that kind is left; while another one works, voice simply uses it.
- Checked and time-limited
Configured speech providers are health-checked automatically in the background, and a single transient failure doesn’t flag a provider as broken. Speech provider requests (transcription, synthesis, health checks and connecting) have time limits, so a provider that stops responding shows up as a named error instead of a stuck conversation.
A provider is flagged only after repeated failed checks, so its status can trail an outage, and it resets when the Gateway restarts.
- One command to check
sophon voice status shows the health of every configured speech provider and whether hands-free is available. Operators can track time to first audio, per-provider text-to-speech latency and text-to-speech provider failures on Sophon’s authenticated Prometheus metrics endpoint.
Those metrics are recorded when a text-to-speech provider speaks the reply.
Speech-to-text
- Deepgram
- OpenAI
- Google Cloud Speech
- Azure Speech
- ElevenLabs
- Sophon Managed Speech
Text-to-speech
- ElevenLabs
- OpenAI
- Google Cloud
- Azure
- Deepgram
- Sophon Managed Speech
In the Dashboard, Deepgram shows your words as you speak; the others transcribe each utterance after you stop. Admins add speech providers, set their priority and test them, and provider credentials are kept in Sophon’s credential vault. Priority is set when a provider is added, and providers are configured per Gateway, not per tenant.
Where your audio goesNot on-device
- To the providers you chose
In the Dashboard, with a speech-to-text provider active, your mic audio travels over your signed-in connection to your Gateway, which passes it to that provider. Reply sentences go to your text-to-speech provider. Those vendors’ terms apply.
- Or to your browser’s vendor
With no provider, your browser’s recognizer does the listening and needs an internet connection; the browser vendor decides where that audio goes (Chrome, for example, uses Google’s speech service). On Sophon Mobile, your phone’s own recognizer listens first.
- What’s stored is the text
Your words and the reply are saved in the chat thread like typed messages. The Gateway doesn’t write your recorded audio to disk. Reply sentences already spoken by a text-to-speech provider are kept in an in-memory cache and replayed instead of being synthesized and billed again, until the Gateway restarts.
07Start with nothing
Three steps and a status check.
- 01
Open Voice
In the Dashboard, open Voice. In Chrome or Edge it works straight away with your browser’s speech.
- 02
Go hands-free
Settings → Voice (Voice Center) → Conversation mode. Your first mic press then starts a hands-free conversation.
- 03
Add providers when you want them
An admin adds speech-to-text and text-to-speech providers in Voice settings (where their priority is set) or from the CLI, and tests them. Speech that starts before the reply is finished needs a text-to-speech provider.
sophon voice statussophon voice settings set --conversation-mode truesophon voice stt-providers add --name deepgram --type deepgram --api-key …sophon voice providers add --name elevenlabs --type elevenlabs --api-key …
sophon voice status lists every configured speech provider with its health, and shows server and browser hands-free separately. The CLI can add, list, test and remove the same providers and edit the same personal and host listening settings; provider priority and a provider’s voice are set in Voice settings. Adding providers needs an admin.
Limits
Where voice stops.
There’s no talk-over detection. Speaking over Sophon doesn’t stop it; pressing the mic does, and that silences the reply, not the task behind it.
Voice limitsVoice turns use a smaller, voice-focused tool set than typed chat. Some jobs are better typed.
What a voice turn isA spoken approval names the kind of action and its risk, not its exact details. When the details matter, read the card before you say “yes, approve”.
What Sophon saysApproval and question prompts are read in your browser’s built-in voice (approvals on Sophon Mobile in your phone’s), not the provider voice you picked.
Which voice reads themWhile a text-to-speech provider speaks, the Voice page doesn’t show the reply; the full text is in the chat thread. Speech is streamed a sentence at a time, never word by word, and only with a text-to-speech provider.
Spoken repliesIn the Dashboard, only Deepgram among speech-to-text providers shows your words while you speak; the others transcribe after you stop.
Speech providersSophon Mobile doesn’t read questions aloud and has no live transcription through the Gateway; it relies on your phone’s recognizer.
Voice limitsThe CLI manages voice but can’t hold a voice conversation, and the desktop app has no push-to-talk shortcut.
Where you can talkOut of the box, Sophon Node takes no part in voice conversations: it can’t listen, has no wake word and doesn’t speak replies, and its voice routes stay off until an operator turns them on.
The node voice routesThe Voice page and the Command Bridge start a new conversation with the default agent each time, with no agent picker. Ending voice in one tab ends it in every tab.
Where you can talkWithout a provider, voice needs a browser with speech recognition, such as Chrome or Edge. Firefox has none.
When no provider is configuredSpeech providers are set per Gateway, not per tenant, and can’t be reordered after they’re added. Semantic end-of-turn detection and plugin speech providers aren’t built yet.
Voice limits and roadmap
Talk to it. Keep the gates.
Voice ships with Sophon on every tier and needs no speech provider to start: open the Voice page in Chrome or Edge, and add providers when you want server transcription and provider voices. The Personal tier is free for individual, non-commercial use.