The architecture of a VoiceOS integration, in one page.
An integration is a folder. At its center is voiceos.integration.json — the
manifest — plus a server implementing the tools it declares. VoiceOS reads
the manifest, runs the server, and handles the rest: voice, the agent,
confirmations, rendering, secrets.
coffee-tracker/├── voiceos.integration.json # identity, runtime, permissions, tools├── server.ts # a standard MCP server (your logic)├── verify.ts # smoke test — run after every change├── voiceos.integration.preview.json # sample inputs for tests & previews└── AGENTS.md # teaches AI coding agents the contract
VoiceOS launches your server over MCP and
lists its tools. Each tool’s description is what the agent reads when deciding
whether to call it — write it as a routing rule.
2
Confirmation — acting tools only
If the tool declares a confirmation view, the notch renders it before any of
your code runs. The user edits bound arguments inline; the edited values are
what execute. Read-only tools skip this.
Rendered from the manifest. {{text}} and {{voice}} are bound to the tool call's arguments.
3
Execution
Your handler runs with the validated arguments. Secrets the user entered during
setup arrive as environment variables.
4
Result
You return JSON. The model reads the data and narrates it. Attach a glance
card with glanceResult([...]) and VoiceOS lifts it out of the payload — the
model never sees UI JSON — validates every block, and renders it natively.
A read tool's glance card: header block + list block, returned as plain JSON.
Integrations describe UI as data, never as components with host access.
There’s one vocabulary per moment:
Before a tool runs
After a tool runs
Purpose
Confirm and edit
Show the result
Blocks
card, text, textField, select, toggle, actions, …
header, list, stats, keyValue, charts, …
Authored
Once, in the manifest
At runtime, via glanceResult([...])
Bindings
{{arg}} binds fields to tool arguments
None — you have the real data
Apple analogy
App Intents confirmation snippets
WidgetKit
Both are rendered by VoiceOS’s own themed components, so they inherit every
restyle for free, and anything off-schema degrades safely instead of breaking
the notch.
When the built-in blocks aren’t enough, either moment also accepts a single
sandboxed custom widget — your own HTML in an
isolated iframe with a strict message bridge.
Declare what you need — API keys via auth: { kind: "apiKey" }, everything else
via preferences. VoiceOS collects the values from the user
right when they’re first needed, stores them
encrypted, and injects them into your server as environment variables named
after each field (process.env.MY_FIELD).
Declared an API key? The first time a tool needs it, the notch asks — you build no settings screen.
Secret values never appear in the agent transcript and are never persisted in
plaintext.
The contract is versioned by MANIFEST_SCHEMA_VERSION and UI_SCHEMA_VERSION,
both currently 1. Additive changes never bump them, and the host renders
unknown future blocks as safe placeholders — so an integration authored today
keeps working across years of VoiceOS updates.
Quickstart
Build your first integration in the app — no code, three minutes.
Build with code
The same integration by hand: manifest, server, cards.
