:
```
For example, a revoked or expired API key returns:
```
error from server 1008: API key is revoked or expired
```
The numeric `` matches the codes used by the WebSocket `error`
message:
| Code | Meaning |
| - | - |
| `1008` | Policy violation: invalid/missing API key, missing setup message, invalid audio format, etc. |
| `1011` | Internal server error |
**Proxy-level rejections.** Errors raised before the request reaches
the upstream worker, for example an unsupported `Content-Type` header, come back as raw error strings without the `error from server` prefix:
```
unsupported content type for SST 'audio/mpeg'
```
In both cases the response is `HTTP 500` and the body is plain text
(not JSON). Treat any non-2xx response as fatal for that request. To
recover a Gradium error code, check whether the body starts with
`error from server`; if it does, parse the prefix.
### In-stream errors
Errors that surface after the response has started streaming are
reported as JSON messages within the streamed body. The HTTP status will
already be `200`.
**TTS** (`POST /api/post/speech/tts`, when `only_audio` is `false`):
```json theme={null}
{"type": "error", "message": "Error description"}
```
**STT** (`POST /api/post/speech/asr`, always NDJSON):
```json theme={null}
{"type": "error", "message": "Error description"}
```
In both cases the connection terminates after the error message. Treat
any `error` line you see in the stream as the end of useful output.
## WebSocket endpoints
When something goes wrong on a WebSocket session (`/api/speech/tts`,
`/api/speech/asr`, `/api/speech/s2s`), the server sends an `error`
message and then closes the socket:
```json theme={null}
{"type": "error", "message": "Error description", "code": 1008}
```
**Common close codes:**
| Code | Meaning |
| - | - |
| `1008` | Policy violation. Causes include invalid API key, missing or malformed setup message, sending data before setup, invalid audio format. |
| `1011` | Internal server error. The session ended due to an unexpected server-side failure. |
After receiving an `error` message, the server closes the WebSocket.
Open a new connection to retry.
For the full setup/input/flush/end lifecycle, see
[WebSocket Lifecycle](/guides/websocket-lifecycle).
## Authentication errors
Authentication failures (missing, invalid, or revoked API key) always
produce code `1008`:
* **REST**: `HTTP 500` with body `error from server 1008: API key is revoked or expired` (or similar).
* **WebSocket**: `error` message with `code: 1008`, then close.
Generate a new API key in the Gradium dashboard and replace the value of
the `x-api-key` header.
# FAQ
Source: https://docs.gradium.ai/guides/faq
Frequently asked questions about the Gradium API
## What language do you support?
We currently support English, French, Spanish, Portuguese, and German, with more
languages currently in development. Sign up to be updated when more languages
become available!
## What is the maximum session duration?
A session can last up to 3000 seconds. If you want to generate longer chunks of
text or transcribe longer audio, it's better to split it into different
sessions.
When using the free tier, there is an additional limitation of 1500 characters
per session.
## How many credits do I need?
For Text-to-Speech, one minute of audio typically requires 750 characters,
which corresponds to 750 credits. This may vary depending on the language and
the speaker's pace. For Speech-to-Text, each second of audio costs 3 credits.
## Can I use my own voice?
Yes! We offer two levels of voice cloning. Our standard feature allows you to
create a high-quality clone with just a 10-second audio sample. For those
seeking even higher fidelity, we now offer Pro Voice Clones; by providing 30
minutes to a few hours of audio data, you can create a custom voice with
unmatched quality and nuance.
## Can I create a voice without an audio sample?
Yes. [Voice Design](/guides/voices/voice-design) creates a voice from a
written description of up to 500 characters. Generate up to five
candidates, audition them, and convert the one you like into a
permanent `voice_id`.
## What is an Instant Voice Clone?
Instant voice cloning enables you to create realistic digital replicas using just
a few seconds of reference audio (typically 10 seconds). Depending on your
subscription plan, you can clone up to 1,000 voices. Please note that explicit
consent from the voice owner is required.
## What is a Pro Voice Clone?
A Pro Voice Clone is a high-fidelity, hyper-realistic voice model created by
fine-tuning our dedicated AI on a large dataset of your audio. Unlike standard
cloning, this process captures the speaker's deepest emotional nuances, unique
accents, and natural pacing, resulting in a digital voice indistinguishable from
the original.
## How do I generate a Pro Voice Clone?
To get started, navigate to the Pro Voice Clone tab in Gradium Studio and upload
your audio dataset. You will receive a notification once the upload is processed. After
training is complete, the voice will appear in your library and be ready for
Text-to-Speech (TTS) generation.
## How much audio do I need for a voice clone?
For an Instant Voice Clone, we get optimal results with only 10 seconds of data. For a
Pro Voice Clone, we require a minimum of 30 minutes of clean audio data. For optimal results
where the voice captures full emotional range and stability, we recommend
providing 2 hours of audio.
## How do I enable Zero Data Retention?
Zero Data Retention is available on our paid plans. To enable it, sign in to
[Gradium Studio](https://studio.gradium.ai/), open the Profile dropdown in the
top right, and select **My organization**. Under **Privacy Settings**, disable
data retention. Once turned off, Gradium will no longer retain your request or
response payloads.
## What does Zero Data Retention cover?
Standard Zero Data Retention covers the request and response payloads of your
speech sessions: the text and audio you send to Text-to-Speech, Speech-to-Text
and Speech-to-Speech, and the audio and transcripts we send back. Once data
retention is disabled in **Privacy Settings**, none of that is retained.
It does **not** cover the inputs you give to our voice creation tools:
* The written prompt you send to [Voice Design](/guides/voices/voice-design).
* The audio sample you upload to create a
[custom voice](/guides/voices/custom-voices), whether it is an Instant Voice
Clone or a Pro Voice Clone.
Disabling data retention does not apply to either of them.
## Can voice design prompts and voice samples be covered too?
Not under the standard policy, but yes on Enterprise plans. If your
organization needs Zero Data Retention extended to voice design prompts or
voice cloning samples, contact
[support@gradium.ai](mailto:support@gradium.ai): on Enterprise plans we can put
a custom privacy policy in place that covers them.
# Installation
Source: https://docs.gradium.ai/guides/installation
Install the Gradium Python SDK and get started
## Prerequisites
* **Python 3.10 or higher** - The Gradium SDK requires Python 3.10+. Check your version with:
```bash theme={null}
python3 --version
```
## Installation
```bash theme={null}
pip install gradium
```
## Quick Start
```python theme={null}
import asyncio
import gradium
async def main():
client = gradium.client.GradiumClient(api_key="your-api-key")
result = await client.tts(
setup={"voice_id": "YTpq7expH9539ERJ", "output_format": "wav"},
text="Welcome to Gradium! Transform your text into natural-sounding speech in seconds."
)
with open("welcome.wav", "wb") as f:
f.write(result.raw_data)
if __name__ == "__main__":
asyncio.run(main())
```
## Creating a Client
### Using API Key Directly
```python theme={null}
import gradium
client = gradium.client.GradiumClient(api_key="gd_your_api_key_here")
```
### Using Environment Variable
Set the `GRADIUM_API_KEY` environment variable:
```bash theme={null}
export GRADIUM_API_KEY=gd_your_api_key_here
```
Then create the client without passing the API key:
```python theme={null}
client = gradium.client.GradiumClient()
```
# Introduction
Source: https://docs.gradium.ai/guides/introduction
Low-latency, high-quality Text-to-Speech and Speech-to-Text API
This documentation covers the Gradium API.
This API exposes our Text-To-Speech and Speech-To-Text models, which offers low-latency, high-quality & natural sounding output and best in class accuracy.
For issues, questions, or feature requests, please contact us at [support@gradium.ai](mailto:support@gradium.ai)
## Features
* **Multilingual**: We currently support five languages: English (en), French (fr), German (de), Spanish (es) and Portuguese (pt) for our Text-To-Speech and Speech-To-Text with more languages to come. Speech-To-Text also accepts `any` to auto-detect the language.
* **Low-latency**: Our servers are based in Europe and in the US, with our expected time-to-first-token is below 300ms when streaming.
* **Voice selection**: We provide a voice library, with multiple voices to choose from in different languages. You can also clone voices instantaneously using a 10'' voice sample.
* **Semantic VAD**: STT streams include `step` messages every 80 ms with inactivity probabilities across future horizons, so voice agents can decide when a speaker has actually finished.
* **Adaptive delay control**: Tune `delay_in_frames` for the latency/quality tradeoff and use `flush` at turn boundaries to process buffered audio without waiting for natural silence.
Get started with the Gradium Python SDK
Convert text to natural-sounding speech
Transcribe audio to text in real-time
Explore the full API reference
# Limits
Source: https://docs.gradium.ai/guides/limits
Session duration, character limits, audio formats, and concurrency
This page summarises the request-level limits that apply to Gradium API
calls. For pricing and credit consumption see
[Credits](/guides/credits).
## Session duration
A single TTS or STT session can last up to **3000 seconds**. This applies
to both the WebSocket and REST POST transports.
If you need to generate or transcribe longer content, split it across
multiple sessions:
* **TTS**: chunk text at sentence boundaries and start a new session
for each chunk (or open a new WebSocket and send a fresh `setup`).
* **STT**: split audio at silences and submit each segment as a
separate request.
## Supported audio formats
### TTS output formats
`output_format` accepts:
* `wav`: WAV at 48 kHz, 16-bit signed mono.
* `pcm`: raw PCM at 48 kHz, 16-bit signed mono, 3840-sample (80 ms) chunks.
* `opus`: Ogg-wrapped Opus.
* Lower / higher sample-rate PCM variants: `pcm_8000`, `pcm_16000`,
`pcm_22050`, `pcm_24000`, `pcm_44100`, `pcm_48000`.
* Telephony: `ulaw_8000` (alias `mulaw_8000`) and `alaw_8000`.
### STT input formats
For STT WebSocket (`input_format` setup field) and STT REST
(`Content-Type` header or `input_format` query parameter), supported
formats are:
* WAV with PCM data, 16/24/32-bit (`audio/wav`, `input_format: "wav"`).
* Raw PCM, 16-bit signed little-endian, mono. The bare value `"pcm"`
defaults to 24 kHz; explicit rates `"pcm_8000"`, `"pcm_16000"`,
`"pcm_22050"`, `"pcm_24000"`, `"pcm_44100"`, `"pcm_48000"` are also
accepted (`audio/pcm` for the REST `Content-Type`).
* Ogg-wrapped Opus (`audio/ogg` or `audio/opus`,
`input_format: "opus"`).
* Telephony: mu-law (`"ulaw_8000"` / `"mulaw_8000"`) and A-law
(`"alaw_8000"`) encoded PCM at 8 kHz.
Convert your input to one of the formats above before uploading.
## Voice Design
| Limit | Value |
| - | - |
| Description (`prompt`) | 1 to 500 characters |
| Candidates per request (`n_samples`) | 1 to 5 |
| Audition text on a candidate | 100 characters, one-shot REST only |
| Candidate retention | 30 days, cleared once converted |
| Converted voices | Count against the custom-voice allowance |
See [Voice Design](/guides/voices/voice-design).
## Concurrency
Concurrent session limits depend on your plan. Contact
[support@gradium.ai](mailto:support@gradium.ai) for the exact limits on
your tier.
# Migrate from Cartesia
Source: https://docs.gradium.ai/guides/migration/cartesia
Move Cartesia TTS and STT integrations to Gradium by changing endpoints, auth, and request fields.
If your application already has a provider adapter, keep that adapter
shape and move the implementation to Gradium endpoints, `x-api-key`
auth, Gradium `voice_id` values, and Gradium WebSocket messages.
For realtime apps, Gradium also gives you STT semantic VAD and adaptive
delay controls in the same API surface.
## Endpoint Swap
| Flow | Existing integration | Gradium |
| - | - | - |
| One-shot TTS | `POST https://api.cartesia.ai/tts/bytes` | `POST https://api.gradium.ai/api/post/speech/tts` |
| Streaming TTS | `wss://api.cartesia.ai/tts/websocket` | `wss://api.gradium.ai/api/speech/tts` |
| Streaming STT | `wss://api.cartesia.ai/stt/websocket` | `wss://api.gradium.ai/api/speech/asr` |
## Auth Mapping
| Existing auth/version field | Gradium |
| - | - |
| `Authorization: Bearer $CARTESIA_API_KEY` | `x-api-key: $GRADIUM_API_KEY` |
| `X-API-Key` | `x-api-key` |
| Provider API version header | Not required by Gradium |
| `access_token` for browser clients | Gradium short-lived `?token=...`; see [Browser WebSockets](/guides/browser-websockets). |
## Gradium POST TTS
Gradium's POST endpoint returns audio bytes when `only_audio` is
`true`, so the rest of your "write this response to a file or player"
code can stay the same.
```bash Gradium theme={null}
curl -L -X POST https://api.gradium.ai/api/post/speech/tts \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"text": "Hello from Gradium.",
"voice_id": "YTpq7expH9539ERJ",
"output_format": "wav",
"only_audio": true
}' \
> output.wav
```
## TTS Field Mapping
| Existing field | Gradium field |
| - | - |
| `transcript` | `text` |
| `voice.id` | `voice_id` |
| `model_id` | `model_name` |
| `output_format.container` / encoding / sample rate | `output_format` string |
| `language` or localized voice choice | Voice language plus optional `json_config.language` / rewrite rules |
| Pronunciation dictionary | `pronunciation_id` |
Gradium uses a compact `output_format` string: `wav`, `pcm`, `opus`,
`ulaw_8000`, `alaw_8000`, or explicit PCM rates such as `pcm_16000`.
## WebSocket TTS Migration
Gradium sends `setup` first, then one or more `text` messages, then
`end_of_stream`. Use `client_req_id` when you need to correlate
concurrent logical requests on one socket.
```json Gradium messages theme={null}
{"type":"setup","voice_id":"YTpq7expH9539ERJ","model_name":"default","output_format":"pcm","client_req_id":"ctx-1","close_ws_on_eos":false}
{"type":"text","text":"Hello from Gradium.","client_req_id":"ctx-1"}
{"type":"end_of_stream","client_req_id":"ctx-1"}
```
For a single streaming utterance, `client_req_id` and
`close_ws_on_eos: false` are optional. Use them when you want multiple
utterances on one socket.
| Existing streaming step | Gradium streaming step |
| - | - |
| Connect to the provider WebSocket | Connect to `wss://api.gradium.ai/api/speech/tts` |
| Start a logical request | Send Gradium `setup`, optionally with `client_req_id` |
| Continue with more text | Send more `text` messages for the same request |
| Finish the request | Send `end_of_stream` for that `client_req_id` |
| Receive `chunk` / audio responses | Receive `audio` messages |
| Receive timestamps | Receive TTS `text` timestamp messages at segment granularity |
| Cancel context | Stop playback and close the socket, or isolate requests with `client_req_id` |
## TTS Feature Mapping
| Existing feature | Gradium equivalent | Migration note |
| - | - | - |
| `context_id` | `client_req_id` | Use for correlation and multiplexing, not persistent prosody across unrelated requests. |
| `continue: true/false` | Multiple `text` messages then `end_of_stream` | Keep chunks as complete words/phrases; use `` at natural boundaries. |
| Concurrent contexts | [Multiplexing](/guides/multiplexing) | Set `close_ws_on_eos: false` and route by `client_req_id`. |
| `transcript` | `text` | Same concept, different field name. |
| Structured `output_format` | `output_format` string | Choose the Gradium format closest to your downstream player. |
| Word/phoneme timestamps | Segment timestamps | Gradium documents segment-level text timestamps. |
| Access tokens for browser | Gradium `GET /api/api-keys/token` and `?token=...` | See [Browser WebSockets](/guides/browser-websockets). |
## Gradium Streaming STT
Gradium STT uses JSON messages for direct WebSocket audio, with base64
payloads. It returns `text`, `end_text`, `step`, `flushed`, and
`end_of_stream` messages.
```json Gradium STT theme={null}
{"type":"setup","model_name":"default","input_format":"pcm","json_config":{"language":"en"}}
{"type":"audio","audio":"base64_encoded_audio"}
{"type":"flush","flush_id":1}
{"type":"end_of_stream"}
```
| Existing STT concept | Gradium STT equivalent |
| - | - |
| Binary audio frames | Base64 `audio` JSON messages |
| `encoding` and `sample_rate` connection params | `input_format` such as `pcm_16000`, `pcm_24000`, `ulaw_8000` |
| `transcript.is_final` | `text` plus `end_text` segment finalization |
| `finalize` | `flush` with a `flush_id` |
| `flush_done` | `flushed` with the same `flush_id` |
| `done` | `end_of_stream` |
| Word timestamps | Segment timestamps in public Gradium docs |
| External VAD / turn events | Semantic VAD `step` messages |
## Adapter Checklist
* Replace provider URLs with matching Gradium endpoints.
* Remove provider API-version headers; Gradium does not require them.
* Change auth to `x-api-key` on servers or temporary `?token=...` for
browser WebSockets.
* Rename `transcript` to `text`.
* Flatten `voice.id` to `voice_id`.
* Convert structured `output_format` into a Gradium format string.
* For context-style routing, map the request identifier to `client_req_id` when you
need response correlation.
* For STT direct WebSocket clients, base64 encode audio inside JSON
messages.
## Next steps
Streaming setup messages, audio messages, flush, and timestamps.
Replace context-style routing with `client_req_id`.
Use short-lived tokens without exposing API keys.
Map low-sample-rate PCM, mu-law, and A-law formats.
# Migrate from Deepgram
Source: https://docs.gradium.ai/guides/migration/deepgram
Replace Deepgram speech integrations with Gradium STT and TTS endpoints.
Keep your existing adapter shape: audio in still produces transcript
events, and text in still produces audio bytes or chunks. Move the
implementation to Gradium endpoints, `x-api-key` auth, Gradium
`voice_id` values, and Gradium message types. For realtime STT,
Gradium adds semantic VAD, adaptive delay, and explicit flush handling
for turn-taking.
## Endpoint Swap
| Flow | Existing integration | Gradium |
| - | - | - |
| Pre-recorded STT | `POST https://api.deepgram.com/v1/listen` | `POST https://api.gradium.ai/api/post/speech/asr` |
| Streaming STT | `wss://api.deepgram.com/v1/listen` | `wss://api.gradium.ai/api/speech/asr` |
| One-shot TTS | `POST https://api.deepgram.com/v1/speak` | `POST https://api.gradium.ai/api/post/speech/tts` |
| Streaming TTS | `wss://api.deepgram.com/v1/speak` | `wss://api.gradium.ai/api/speech/tts` |
## Auth Mapping
| Existing auth | Gradium |
| - | - |
| `Authorization: Token $DEEPGRAM_API_KEY` | `x-api-key: $GRADIUM_API_KEY` |
| `Authorization: Bearer ` | Browser/mobile clients should use Gradium temporary WebSocket tokens. See [Browser WebSockets](/guides/browser-websockets). |
| API key in WebSocket headers | `x-api-key` header, or `?token=...` for short-lived browser tokens. |
## Gradium Speech-to-Text POST
For pre-recorded audio, keep sending the audio bytes in the request
body and switch the URL. Gradium streams newline-delimited JSON
messages back.
```bash Gradium theme={null}
curl -L -X POST https://api.gradium.ai/api/post/speech/asr \
--url-query 'json_config={"language":"en"}' \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: audio/wav" \
--data-binary @input.wav
```
Gradium's response is NDJSON. Build the transcript by collecting
`text` messages and pairing them with `end_text` when you need segment
end timestamps.
## Speech-to-Text WebSocket
Gradium's direct WebSocket protocol sends audio in JSON messages with
base64 payloads:
```json Gradium messages theme={null}
{"type":"setup","model_name":"default","input_format":"pcm","json_config":{"language":"en"}}
{"type":"audio","audio":"base64_encoded_audio"}
{"type":"end_of_stream"}
```
| Existing STT concept | Gradium STT message |
| - | - |
| WebSocket connect | Connect to `wss://api.gradium.ai/api/speech/asr` |
| Model and language settings | `setup.model_name`, `setup.json_config.language` |
| Audio frame | `{"type":"audio","audio":"base64_encoded_audio"}` |
| Transcript result | `{"type":"text","text":"...","start_s":...}` |
| Final segment timing | `{"type":"end_text","stop_s":...}` |
| Endpointing / turn signal | `step.vad[*].inactivity_prob` |
| Close/finalize stream | `{"type":"end_of_stream"}` |
| Force pending output | `{"type":"flush","flush_id":1}` |
Set `input_format` to match the audio you send: `pcm`, `wav`, `opus`,
`ulaw_8000`, `alaw_8000`, or another supported Gradium format.
## STT Feature Mapping
| Existing feature | Gradium equivalent | Migration note |
| - | - | - |
| `model=nova-*` | `model_name: "default"` unless given another Gradium model alias | Model names are provider-specific. |
| `language=en-US` | `json_config: {"language": "en"}` | Gradium uses short language codes: `en`, `fr`, `de`, `es`, `pt`. Always set it; pass `"any"` to auto-detect. |
| Interim transcript events | Streaming `text` plus `end_text` segment finalization | Gradium emits text segments and separate end timestamps. |
| `speech_final` / `endpointing` | Semantic VAD `step` messages | Use inactivity probabilities across horizons; see [Turn-Taking](/guides/recipes/turn-taking). |
| `utterance_end_ms` | VAD thresholding plus `send_flush()` | Implement turn-end policy in your app. |
| `encoding`, `sample_rate` | `input_format` such as `pcm_16000`, `ulaw_8000` | Pick the exact Gradium format for your audio bytes. |
| `punctuate`, `smart_format` | Built into model behavior where available | There is no one-to-one request flag. |
| `diarize`, channels, alternatives | No direct public Gradium equivalent in these docs | Keep provider-specific code behind an adapter if you rely on these. |
| Word-level timestamps | Segment timestamps | Gradium currently documents segment-level timestamps. |
## Gradium Text-to-Speech POST
Gradium uses `voice_id` in the JSON body and returns raw audio bytes
when `only_audio` is `true`, so your existing file-write or playback
code can usually remain unchanged.
```bash Gradium theme={null}
curl -L -X POST https://api.gradium.ai/api/post/speech/tts \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"text": "Hello from Gradium.",
"voice_id": "YTpq7expH9539ERJ",
"output_format": "wav",
"only_audio": true
}' \
> output.wav
```
## Text-to-Speech WebSocket
For streaming TTS, use Gradium `setup`, `text`, and `end_of_stream`
messages:
| Existing TTS behavior | Gradium TTS message |
| - | - |
| Send text | `{"type":"text","text":"..."}` after `setup` |
| Force generation at a boundary | Include `` in a `text` message at a natural boundary |
| Finish generation | `{"type":"end_of_stream"}` for graceful completion, or close the socket to interrupt |
| Binary/audio response frames | `{"type":"audio","audio":"base64..."}` |
| Metadata response | `ready` message plus `request_id` |
| Warning response | `error` message for terminal errors; validate chunking and limits in your app |
```json Gradium messages theme={null}
{"type":"setup","voice_id":"YTpq7expH9539ERJ","model_name":"default","output_format":"pcm"}
{"type":"text","text":"Hello from Gradium. "}
{"type":"end_of_stream"}
```
## TTS Feature Mapping
| Existing feature | Gradium equivalent | Migration note |
| - | - | - |
| Speak REST `/v1/speak` | `POST /api/post/speech/tts` | Use `only_audio: true` for raw audio responses. |
| Speak WebSocket `/v1/speak` | `wss://api.gradium.ai/api/speech/tts` | Send `setup` before text. |
| `model=aura-*` | `voice_id` plus optional `model_name` | Choose a Gradium voice from the voice library or your custom voice. |
| Output `encoding` / `sample_rate` | `output_format` | Use `pcm`, `pcm_16000`, `ulaw_8000`, `wav`, `opus`, etc. |
| `Flush` command | `` text tag | Flush sparingly, usually after LLM sentence or answer boundaries. |
| `Close` command | `end_of_stream` or close socket | Use `end_of_stream` for graceful completion. |
| TTS metadata | `ready.request_id`, `ready.sample_rate`, `ready.frame_size` | Log `request_id` for debugging. |
## Adapter Checklist
* Replace provider URLs with matching Gradium endpoints.
* Change auth to `x-api-key`, or use browser-safe tokens for client apps.
* For STT, map query params into `setup` and `json_config`.
* For STT WebSocket, wrap audio bytes as base64 JSON messages.
* For TTS, replace provider voice/model names with a Gradium `voice_id`.
* For TTS WebSocket, send a Gradium `setup` message before text.
* Replace provider finality fields with Gradium `end_text`, `step`, and
`flushed` handling.
* Keep provider-specific features such as diarization behind adapter
capability checks.
## Next steps
Real-time audio streaming, semantic VAD, and flush.
Replace endpointing and speech-final logic with Gradium VAD.
Streaming text-to-speech over WebSocket.
Setup, ready, input, flush, end, multiplexing, and errors.
# Migrate from ElevenLabs
Source: https://docs.gradium.ai/guides/migration/elevenlabs
Replace ElevenLabs TTS calls with Gradium REST and WebSocket endpoints.
If your app already sends text to a provider and writes the returned
audio to a file or stream, the migration is intentionally small: change
the endpoint to Gradium, send your Gradium API key as `x-api-key`, and
use a Gradium `voice_id`. The playback path can usually stay the same.
## Endpoint swap
| Flow | Existing integration | Gradium |
| - | - | - |
| One-shot TTS | `POST https://api.elevenlabs.io/v1/text-to-speech/{voice_id}` | `POST https://api.gradium.ai/api/post/speech/tts` |
| Streaming TTS | `wss://api.elevenlabs.io/v1/text-to-speech/{voice_id}/stream-input` | `wss://api.gradium.ai/api/speech/tts` |
## Request mapping
For POST requests, the most important change is that Gradium takes the
voice in the JSON body instead of the URL path:
| Existing concept | Gradium field |
| - | - |
| `{voice_id}` path parameter | `voice_id` |
| `text` | `text` |
| `model_id` | `model_name` |
| Output format query/body settings | `output_format` |
| Provider API-key header | `x-api-key` header |
Use the [Gradium POST example](/guides/migration#gradium-post-example)
as the replacement request. It returns raw audio bytes when
`only_audio` is `true`, so existing "save the response body as audio"
code can stay the same.
## WebSocket mapping
For streaming TTS, the Gradium lifecycle is:
1. Connect to `wss://api.gradium.ai/api/speech/tts`.
2. Send a `setup` message with `voice_id`, `model_name`, and `output_format`.
3. Send one or more `text` messages.
4. Send `end_of_stream`.
5. Read `audio` messages until Gradium sends `end_of_stream`.
Use the [Gradium WebSocket example](/guides/migration#gradium-websocket-example)
as the minimal replacement. Your receive loop should continue buffering
audio chunks in the same place it buffered provider audio.
## Checklist
* Replace the provider URL with the Gradium endpoint.
* Change auth to `x-api-key`.
* Move the voice id from the URL path into `voice_id`.
* Use a Gradium voice id from the [voice library](/guides/voices/flagship-voices).
* Keep your existing audio write/playback path.
Full POST schema, response modes, and supported output formats.
# Migrate from Fish Audio
Source: https://docs.gradium.ai/guides/migration/fish-audio
Move Fish Audio TTS, voice cloning, and ASR integrations to Gradium endpoints, x-api-key auth, and JSON WebSocket messages.
To switch from Fish Audio to Gradium, you need a handful of mechanical
edits: point requests at Gradium URLs, send `x-api-key` instead of a
Bearer token, rename a few request fields, and replace MessagePack
WebSocket events with JSON messages, which removes the msgpack
dependency.
## Why choose Gradium over Fish Audio
* **Official voice catalog.** Gradium's
[flagship voices](/guides/voices/flagship-voices) are first-party
voices built with voice experts and researchers for every supported
language. Fish Audio's library is community-uploaded models owned by
individual accounts.
* **Streaming STT.** Gradium transcribes live audio over WebSocket with
semantic VAD and adaptive delay for turn-taking. Fish Audio only
offers batch transcription.
* **Speech-to-speech translation.** Gradium's
[S2S endpoint](/guides/speech-to-speech) translates live speech into
another language over one WebSocket: audio streams in, translated
audio and transcript stream out. Fish Audio has no equivalent.
* **JSON everywhere.** All Gradium WebSocket messages are JSON, so
clients drop the MessagePack dependency and work directly with
`wscat`, browsers, and standard libraries.
* **Browser-safe auth.** Short-lived tokens from
`GET /api/api-keys/token` let browser and mobile clients connect
without embedding API keys; see
[Browser WebSockets](/guides/browser-websockets).
* **Production controls.** [Multiplexing](/guides/multiplexing) with
`client_req_id`, [pronunciation dictionaries](/api-reference/endpoint/list-pronunciations),
[text rewriting](/guides/text-rewriting), and telephony formats
(`ulaw_8000`, `alaw_8000`) are built into the same endpoints.
## Endpoint Swap
| Flow | Fish Audio | Gradium |
| - | - | - |
| One-shot TTS | `POST https://api.fish.audio/v1/tts` | `POST https://api.gradium.ai/api/post/speech/tts` |
| Streaming TTS | `wss://api.fish.audio/v1/tts/live` | `wss://api.gradium.ai/api/speech/tts` |
| Voice cloning | `POST https://api.fish.audio/model` | `POST https://api.gradium.ai/api/voices/` |
| List voices | `GET https://api.fish.audio/model` | `GET https://api.gradium.ai/api/voices/` |
| Batch STT | `POST https://api.fish.audio/v1/asr` | `POST https://api.gradium.ai/api/post/speech/asr` |
| Streaming STT | Not available | `wss://api.gradium.ai/api/speech/asr` |
## Auth Mapping
| Fish Audio | Gradium |
| - | - |
| `Authorization: Bearer $FISH_API_KEY` | `x-api-key: $GRADIUM_API_KEY` |
| `model` header (`s1`, `s2-pro`, `s2.1-pro`) | `model_name` field, default `default` |
| API key in browser clients | Short-lived `?token=...`; see [Browser WebSockets](/guides/browser-websockets). |
`model_name` goes in the request body or `setup` message; `default`
always points at the current recommended model, so most integrations
never set it.
## Voice Catalog
Gradium's [flagship voice catalog](/guides/voices/flagship-voices)
covers all supported languages, including French, German, Spanish, and
Portuguese, plus regional variants such as Brazilian Portuguese and
Mexican Spanish.
| Fish Audio | Gradium |
| - | - |
| Community-uploaded clones, per-user ownership | First-party flagship voices |
| Availability depends on the uploading account | Stable `voice_id` values |
| Quality varies with the uploader's reference audio | Consistent studio-quality recordings |
| Discovery by search, tags, and popularity | Curated list with description, country, and gender |
Pick a replacement `voice_id` per language from the
[voice library](/guides/voices/flagship-voices), or clone a voice you
own as a private custom voice (see
[Voice Cloning Migration](#voice-cloning-migration)).
## Gradium POST TTS
With `only_audio: true`, the response is raw audio bytes, same as Fish
Audio's `/v1/tts`.
```bash Gradium theme={null}
curl -L -X POST https://api.gradium.ai/api/post/speech/tts \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"text": "Hello from Gradium.",
"voice_id": "YTpq7expH9539ERJ",
"output_format": "wav",
"only_audio": true
}' \
> output.wav
```
## TTS Field Mapping
| Fish Audio field | Gradium field |
| - | - |
| `text` | `text` |
| `reference_id` | `voice_id` |
| `model` header | `model_name` |
| `format` + `sample_rate` | `output_format` string |
| `latency` (`low`, `balanced`, `normal`) | Not needed; use the WebSocket endpoint for low latency |
| `chunk_length` / `min_chunk_length` | Send text in chunks over WebSocket; place `` at natural boundaries |
| `normalize` | `json_config.rewrite_rules`; see [Text Rewriting](/guides/text-rewriting) |
| `prosody.speed` | `json_config.padding_bonus` (negative is faster, positive is slower) |
| `temperature` | `json_config.temp` |
| `top_p` / `repetition_penalty` | No direct equivalent; tune `temp` |
| `references` (inline zero-shot audio) | Create a voice once with `POST /voices/`, then reuse its `voice_id` |
`output_format` is one string instead of separate format and
sample-rate fields: `wav`, `pcm`, `opus`, `ulaw_8000`, `alaw_8000`, or
explicit PCM rates such as `pcm_16000` and `pcm_44100`.
Gradium does not return MP3. Replace `format: "mp3"` with `wav` for
file output or `opus` for compressed streaming.
## WebSocket TTS Migration
Fish Audio's `/v1/tts/live` socket uses MessagePack events. Gradium's
TTS socket uses JSON text messages.
```json Gradium messages theme={null}
{"type":"setup","voice_id":"YTpq7expH9539ERJ","model_name":"default","output_format":"pcm"}
{"type":"text","text":"Hello from Gradium."}
{"type":"end_of_stream"}
```
| Fish Audio event | Gradium message |
| - | - |
| `start` with a `request` config object | `setup` with `voice_id`, `model_name`, `output_format` |
| `text` | `text` |
| `flush` | `` tag inside a `text` message |
| `stop` | `end_of_stream` |
| `audio` (binary payload) | `audio` (base64 payload in JSON) |
| `finish` with `reason` | `end_of_stream` server message |
Gradium replies with `ready` after `setup`, streams `audio` messages,
and finishes with its own `end_of_stream`. For several logical requests
on one socket, add `client_req_id` to each message and set
`close_ws_on_eos: false`; see [Multiplexing](/guides/multiplexing).
## Voice Cloning Migration
Fish Audio clones a voice by creating a model from uploaded samples.
Gradium uses one `POST /voices/` call; the returned `uid` is the
`voice_id` for any TTS request.
```bash Gradium theme={null}
curl -X POST https://api.gradium.ai/api/voices/ \
-H "x-api-key: $GRADIUM_API_KEY" \
-F "audio_file=@voice_sample.wav" \
-F "name=My Custom Voice" \
-F "language=en"
```
| Fish Audio field | Gradium field |
| - | - |
| `voices` (one or more audio files) | `audio_file` (one file; about 10 seconds of clean speech) |
| `texts` (transcripts of the samples) | Not required |
| `title` | `name` |
| `train_mode` (`fast`) | Not required; extraction runs on upload |
| `visibility` | Not required; voices are private to your account |
| `type: "tts"` | Not required |
| `enhance_audio_quality` | Not required |
| Returned `_id` | Returned `uid`, used as `voice_id` |
Manage clones with `GET`, `PUT`, and `DELETE` on
`/voices/{voice_uid}`; see [Custom Voices](/guides/voices/custom-voices).
## STT Migration
Fish Audio's `/v1/asr` takes MessagePack or multipart uploads and
returns one JSON object. Gradium's POST STT endpoint takes raw audio
bytes as the request body and streams newline-delimited JSON messages.
```bash Gradium theme={null}
curl -X POST https://api.gradium.ai/api/post/speech/asr \
--url-query 'json_config={"language":"en"}' \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: audio/wav" \
--data-binary @recording.wav
```
| Fish Audio ASR concept | Gradium STT equivalent |
| - | - |
| MessagePack or multipart request body | Raw audio bytes with `Content-Type: audio/wav`, `audio/pcm`, or `audio/ogg` |
| `language` | `json_config.language` (always set it; `"any"` to auto-detect) |
| `text` in the response | Concatenate streamed `text` messages |
| `segments` with start and end times | Per-message segment timing |
| `ignore_timestamps` | Timing is always included in streamed messages |
| No streaming option | `wss://api.gradium.ai/api/speech/asr` with semantic VAD `step` messages |
For live transcription, use the WebSocket endpoint; it emits `step`
messages with `inactivity_prob` for turn-taking. See
[Speech-to-Text](/guides/speech-to-text).
## Adapter Checklist
* Replace Fish Audio URLs with the matching Gradium endpoints.
* Change auth from `Authorization: Bearer` to `x-api-key`, or
short-lived `?token=...` for browser WebSockets.
* Drop the `model` header; set `model_name` only for a specific model.
* Rename `reference_id` to `voice_id`, using flagship voices from the
[voice library](/guides/voices/flagship-voices).
* Collapse `format` and `sample_rate` into one `output_format` string;
replace MP3 with `wav` or `opus`.
* Replace MessagePack events (`start`, `text`, `stop`) with JSON
messages (`setup`, `text`, `end_of_stream`).
* Re-create cloned voices with `POST /voices/` and use the returned
`uid` values.
* Move prosody and sampling knobs into `json_config`; see
[Voice Settings](/guides/voice-settings).
* For batch STT, send raw audio bytes and read streamed JSON messages.
## Next steps
Streaming setup messages, audio messages, flush, and timestamps.
Create, update, and manage cloned voices from reference audio.
Speed, temperature and rewrite rules via json\_config.
Streaming transcription with semantic VAD and flush.
# Migrate to Gradium
Source: https://docs.gradium.ai/guides/migration/index
Move existing voice API integrations to Gradium with minimal code changes.
Most voice API migrations come down to the same small swap: point your
existing request code at Gradium, send your Gradium API key in the
`x-api-key` header, and use a Gradium `voice_id` or model setting.
The bigger win is that the same API covers realtime TTS, realtime STT,
semantic VAD, adaptive delay, browser-safe WebSocket tokens, and
custom voices.
Your app can keep the same shape:
* **POST** when you already have the full input.
* **WebSocket** when you want streaming input or low-latency output.
* **Audio bytes or streamed chunks** come back in the same places your
current provider integration already handles them.
* **Semantic VAD and adaptive delay** give voice agents first-class
turn-taking signals instead of forcing you to bolt on endpointing
heuristics.
* **Browser clients** should use short-lived Gradium tokens instead of
embedding API keys. See [Browser WebSockets](/guides/browser-websockets).
If you already wrapped ElevenLabs, Cartesia, Deepgram, or Fish Audio
behind a small provider adapter, migrating is usually just changing the
URL, auth header, and a few field names.
## Gradium POST example
For a complete text block, send one HTTP request and write the audio
response to a file:
```bash theme={null}
curl -L -X POST https://api.gradium.ai/api/post/speech/tts \
-H "x-api-key: your_api_key" \
-H "Content-Type: application/json" \
-d '{"text": "Hello from Gradium.", "voice_id": "YTpq7expH9539ERJ", "output_format": "wav", "only_audio": true}' \
> output.wav
```
That is the whole path for one-shot TTS: request body in, audio bytes
out. For the full schema, see [Text-to-Speech REST](/guides/text-to-speech-rest).
## Gradium WebSocket example
For streaming TTS, connect to the Gradium WebSocket, send `setup` once,
then send text:
```bash theme={null}
wscat -c "wss://api.gradium.ai/api/speech/tts" \
-H "x-api-key: your_api_key"
```
After the connection opens, send:
```json theme={null}
{"type":"setup","voice_id":"YTpq7expH9539ERJ","model_name":"default","output_format":"wav"}
{"type":"text","text":"Hello from Gradium."}
{"type":"end_of_stream"}
```
Gradium streams `audio` messages back with base64-encoded audio chunks.
For the full message contract, see [Text-to-Speech WebSocket](/guides/text-to-speech).
## Provider guides
Move existing TTS calls to Gradium REST and WebSocket endpoints.
Move TTS and STT adapters to Gradium request fields and message types.
Replace speech adapters with Gradium STT, TTS, semantic VAD, and flush.
Move TTS, voice cloning, and ASR calls to Gradium JSON APIs.
## What usually changes
| Area | Change |
| - | - |
| Base URL | Use `https://api.gradium.ai/api` for REST and `wss://api.gradium.ai/api` for WebSockets. |
| Auth | Send `x-api-key: your_api_key`. |
| TTS voice | Pass a Gradium `voice_id` in the request body or WebSocket `setup` message. |
| TTS output | Use `output_format`, for example `wav`, `pcm`, or `opus`. |
| Streaming start | Send a Gradium `setup` message first on WebSocket connections. |
| Streaming end | Send `{"type":"end_of_stream"}` when you are done sending input. |
| Browser auth | Generate a temporary token with `GET /api/api-keys/token`, then connect with `?token=...`. |
| Concurrent WebSocket requests | Use `client_req_id` and `close_ws_on_eos: false`. |
| Turn-taking | Use STT `step` messages, `inactivity_prob`, `delay_in_frames`, and `flush`. |
## Speech-to-text endpoints
If you are migrating an STT integration, use the same idea with the STT
routes:
| Flow | Gradium endpoint |
| - | - |
| Complete audio file | `POST https://api.gradium.ai/api/post/speech/asr` |
| Live audio stream | `wss://api.gradium.ai/api/speech/asr` |
See [Speech-to-Text REST](/guides/speech-to-text-rest) and
[Speech-to-Text WebSocket](/guides/speech-to-text) for the message
formats.
## Production Patterns
Setup, ready, input, flush, end-of-stream, multiplexing, and errors.
Issue short-lived tokens for browser and mobile WebSocket clients.
# Multiplexing
Source: https://docs.gradium.ai/guides/multiplexing
Send multiple independent requests over a single WebSocket connection
Multiplexing lets you send multiple independent requests over one
WebSocket connection. Each logical request gets a `client_req_id`; the
server copies that value onto responses so your client can route audio,
text, VAD, and end events back to the right caller.
This is useful when you want to avoid opening a new WebSocket for every
short utterance or when a server needs to process many low-latency TTS
requests in parallel.
## How Multiplexing Works
| Mode | How to use it | Behavior |
| - | - | - |
| Single-use | Send setup without `close_ws_on_eos: false` | The socket closes after `end_of_stream`. |
| Reusable sequential | Set `close_ws_on_eos: false`, omit `client_req_id` | Keep the socket open and process one request at a time. |
| Concurrent multiplexed | Set `close_ws_on_eos: false`, include `client_req_id` on every message | Multiple active requests share the socket. |
To run requests concurrently on the same socket:
1. Set `close_ws_on_eos: false` in every `setup`.
2. Generate a unique `client_req_id` per logical request.
3. Include that same `client_req_id` on `setup`, input messages, and
`end_of_stream`.
4. Route responses by `client_req_id`.
If you reuse a `client_req_id` while the previous request is still
active, the server returns a protocol error.
## TTS Example
```python theme={null}
import asyncio
import collections
import gradium
async def synthesize_many():
setup = {
"voice_id": "RI2y7oBdsQJmkgFF",
"output_format": "wav",
"close_ws_on_eos": False, # Enable multiplexing.
}
texts = [
"First request. Second part, last one.",
"Second request. Second part, last one again.",
]
client = gradium.client.GradiumClient(api_key="your-api-key")
async with client.tts_realtime(send_setup_on_start=False) as stream:
async def send_loop():
for idx, text in enumerate(texts):
stamp = {"client_req_id": f"req-{idx:02d}"}
await stream.send_setup(setup | stamp)
await stream.send_text(text, **stamp)
await stream.send_eos(**stamp)
async def recv_loop():
audio = collections.defaultdict(list)
num_eos = 0
async for msg in stream:
if msg["type"] == "audio":
audio[msg.get("client_req_id")].append(msg["audio"])
elif msg["type"] == "end_of_stream":
num_eos += 1
if num_eos == len(texts):
break
return audio
_, audio = await asyncio.gather(send_loop(), recv_loop())
return {k: b"".join(v) for k, v in audio.items()}
audio_by_request = asyncio.run(synthesize_many())
```
Each `audio` message contains decoded bytes when using the Python SDK.
If you talk to the WebSocket directly, `audio` is base64 encoded.
## Wire Transcript
```json Client theme={null}
{"type":"setup","voice_id":"RI2y7oBdsQJmkgFF","output_format":"pcm","close_ws_on_eos":false,"client_req_id":"req-a"}
{"type":"text","text":"First request.","client_req_id":"req-a"}
{"type":"end_of_stream","client_req_id":"req-a"}
{"type":"setup","voice_id":"RI2y7oBdsQJmkgFF","output_format":"pcm","close_ws_on_eos":false,"client_req_id":"req-b"}
{"type":"text","text":"Second request.","client_req_id":"req-b"}
{"type":"end_of_stream","client_req_id":"req-b"}
```
```json Server theme={null}
{"type":"ready","request_id":"...","client_req_id":"req-a","sample_rate":48000,"frame_size":3840}
{"type":"ready","request_id":"...","client_req_id":"req-b","sample_rate":48000,"frame_size":3840}
{"type":"audio","client_req_id":"req-b","audio":"...","start_s":0.0,"stop_s":0.08}
{"type":"audio","client_req_id":"req-a","audio":"...","start_s":0.0,"stop_s":0.08}
{"type":"end_of_stream","client_req_id":"req-b"}
{"type":"end_of_stream","client_req_id":"req-a"}
```
Responses may arrive interleaved. Do not assume the first request ends
before the second one starts returning audio.
## STT Notes
The same `client_req_id` mechanism exists on STT WebSockets. Use it
only when each audio source is a separate logical stream and your
client can route every audio chunk, flush, and `end_of_stream` to the
right request.
```json theme={null}
{"type":"setup","model_name":"default","input_format":"pcm","json_config":{"language":"en"},"close_ws_on_eos":false,"client_req_id":"caller-1"}
{"type":"audio","audio":"base64_audio","client_req_id":"caller-1"}
{"type":"flush","flush_id":1,"client_req_id":"caller-1"}
{"type":"end_of_stream","client_req_id":"caller-1"}
```
For most live microphone or telephony applications, one STT WebSocket
per live speaker stream is easier to reason about. Multiplex STT only
when connection overhead matters and you have strict routing tests.
## Closing a Reusable Socket
After all logical requests have finished, send an unscoped
`end_of_stream` to close the reusable socket:
```json theme={null}
{"type":"end_of_stream"}
```
If active requests are still running, the server closes after they
complete. If no requests are active, it closes immediately.
## Error Handling
Errors include `client_req_id` when the server can identify the
logical request:
```json theme={null}
{"type":"error","message":"Session already active (req-a).","code":1002,"client_req_id":"req-a"}
```
Treat an `error` as terminal for that logical request. Depending on the
error and whether other sessions are active, the WebSocket may close
after outstanding requests finish.
## Related
Setup, ready, input, flush, end, and errors.
Stream generated text while preserving prosody.
# Browser Microphone to STT
Source: https://docs.gradium.ai/guides/recipes/browser-microphone-stt
Capture microphone audio in a browser and stream it to Gradium STT
This recipe shows the moving pieces for browser microphone
transcription. In production, combine it with
[Browser WebSockets](/guides/browser-websockets) so the browser uses a
short-lived token instead of an API key.
## Flow
1. Ask the user for microphone permission.
2. Capture audio with the Web Audio API.
3. Convert float samples to 16-bit mono PCM.
4. Send base64 audio chunks over `wss://api.gradium.ai/api/speech/asr`.
5. Render `text` messages and use `step` messages for turn-taking.
## Browser Client
```js theme={null}
async function getGradiumToken() {
const response = await fetch("/gradium-token", {method: "POST"});
if (!response.ok) throw new Error("Could not get Gradium token");
return response.json();
}
function pcm16Base64(float32) {
const pcm = new Int16Array(float32.length);
for (let i = 0; i < float32.length; i += 1) {
const sample = Math.max(-1, Math.min(1, float32[i]));
pcm[i] = sample < 0 ? sample * 32768 : sample * 32767;
}
const bytes = new Uint8Array(pcm.buffer);
let binary = "";
for (const byte of bytes) binary += String.fromCharCode(byte);
return btoa(binary);
}
function appendTranscript(text) {
console.log("transcript", text);
}
function markPossibleTurnEnd() {
console.log("possible turn end");
}
async function startTranscription() {
const {token} = await getGradiumToken();
const url = new URL("wss://api.gradium.ai/api/speech/asr");
url.searchParams.set("token", token);
const stream = await navigator.mediaDevices.getUserMedia({audio: true});
const audioContext = new AudioContext({sampleRate: 24000});
const source = audioContext.createMediaStreamSource(stream);
const processor = audioContext.createScriptProcessor(2048, 1, 1);
const ws = new WebSocket(url);
ws.addEventListener("open", () => {
ws.send(JSON.stringify({
type: "setup",
model_name: "default",
input_format: "pcm",
json_config: {language: "en"},
}));
});
ws.addEventListener("message", (event) => {
const msg = JSON.parse(event.data);
if (msg.type === "text") {
appendTranscript(msg.text);
} else if (msg.type === "step") {
const horizon = msg.vad[msg.vad.length - 1];
if (horizon?.inactivity_prob > 0.5) markPossibleTurnEnd();
}
});
processor.onaudioprocess = (event) => {
if (ws.readyState !== WebSocket.OPEN) return;
const samples = event.inputBuffer.getChannelData(0);
ws.send(JSON.stringify({
type: "audio",
audio: pcm16Base64(samples),
}));
};
source.connect(processor);
processor.connect(audioContext.destination);
return {
stop() {
ws.send(JSON.stringify({type: "end_of_stream"}));
processor.disconnect();
source.disconnect();
stream.getTracks().forEach((track) => track.stop());
},
};
}
```
`ScriptProcessorNode` is easy to read but deprecated. For production,
prefer an `AudioWorklet` so audio capture stays reliable under UI load.
## Audio Format Notes
* `input_format: "pcm"` means 24 kHz, 16-bit signed mono PCM.
* If your browser audio graph runs at 48 kHz, either resample to 24 kHz
or send `input_format: "pcm_48000"`.
* Send small chunks, around 80-100 ms, to keep latency low.
* Do not send compressed browser formats unless you explicitly set a
supported Gradium input format such as `opus`.
## Related
Message types, VAD, flushing, and direct WebSocket examples.
Use semantic VAD to decide when a speaker has finished.
# Keyword Boosting
Source: https://docs.gradium.ai/guides/recipes/keyword-boosting
Recognize names, brands, and jargon by biasing STT toward a custom dictionary
Speech-to-Text is excellent on everyday language. Keyword boosting extends
that strength to the vocabulary that is specific to *your* product: player
names, brands, products, and channel names. Terms like `Mbappé` or
`Vinícius` come out exactly the way you expect.
You pass a dictionary of the terms you care about in `json_config`, and the
model gives them priority while decoding, in real time, with no retraining.
## The idea
Add a `keywords` object with the terms and a `boost` weight:
```python theme={null}
json_config = {
"language": "en",
"keywords": {
"words": ["Mbappé", "Haaland", "Vinícius", "Bellingham", "Lewandowski"],
"boost": 3,
},
}
```
`boost` is applied in log-probability space, so its effect is exponential: a
small value already shifts decoding noticeably. Use the range `-6` to `6`,
with `3` as a strong default. See
[Transcription Settings](/guides/transcription-settings#keyword-boosting) for
the full parameter reference.
## Build a good dictionary
Keywords are matched one token at a time, which means two things.
1. **Use single tokens, with no spaces.** Split multi-word names, so
`"Ferran Torres"` becomes `"Ferran"` and `"Torres"`.
2. **Match case and accents.** `"Mbappé"` and `"Mbappe"` are different keys,
so include the variants you expect.
You can boost up to **500 keywords**. Each variant of a word (lowercase,
uppercase, accented, etc.) is its own key and counts toward that limit, so
budget accordingly when expanding terms into variants.
A small helper that does both:
```python theme={null}
import unicodedata
def build_keywords(terms):
strip = lambda s: "".join(
c for c in unicodedata.normalize("NFKD", s) if not unicodedata.combining(c)
)
out = []
for term in terms:
for token in term.split(): # split phrases into tokens
for variant in (token, token.lower(), strip(token), strip(token).lower()):
if variant not in out:
out.append(variant)
return out
build_keywords(["Ferran Torres", "Mbappé"])
# ['Ferran', 'ferran', 'Torres', 'torres', 'Mbappé', 'mbappé', 'Mbappe', 'mbappe']
```
## Full example
```python theme={null}
import asyncio
import gradium
TERMS = ["Kylian Mbappé", "Erling Haaland", "Vinícius Júnior", "Jude Bellingham"]
async def transcribe(audio_source):
client = gradium.client.GradiumClient(api_key="your-api-key")
async with client.stt_realtime(
model_name="default",
input_format="pcm",
json_config={
"language": "en",
"delay_in_frames": 16,
"keywords": {"words": build_keywords(TERMS), "boost": 3},
},
) as stt:
transcript = []
async def producer():
async for chunk in audio_source:
await stt.send_audio(chunk)
await stt.send_eos()
async def consumer():
async for msg in stt:
if msg["type"] == "text":
transcript.append(msg["text"])
elif msg["type"] == "end_of_stream":
return
await asyncio.gather(producer(), consumer())
return " ".join(transcript).strip()
```
## Tuning
| Boost | Behaviour | Use it when |
| - | - | - |
| `1`-`2` | Gentle nudge, no risk | Fairly common vocabulary, where a tiebreak is enough. |
| `3` | Recommended default; recovers most rare names | Most cases. |
| `4` | Stronger recovery | Dense or foreign proper nouns. |
| `5`-`6` | Diminishing returns, and the decoder may start to loop (repeat a keyword) | Rarely, and watch the output. |
| negative | De-boosts a term | Only to remove a specific word you never want. |
Boosting is fully compatible with streaming, and it earns its keep most at low
`delay_in_frames`, where the model has less look-ahead and leans more on the
dictionary.
## Related
The `keywords` parameter reference and the rest of `json_config`.
STT message types, VAD details, and flushing.
# LLM Tokens to Streaming TTS
Source: https://docs.gradium.ai/guides/recipes/llm-to-tts
Stream generated text into TTS while preserving prosody and low latency
Use this pattern when an LLM produces text incrementally and you want
the user to hear audio before the full answer is finished.
The core idea:
1. Open one TTS WebSocket before the LLM starts.
2. Buffer LLM tokens into word or phrase chunks.
3. Send each chunk as a `text` message.
4. Add `` when the LLM finishes a sentence or the whole answer.
5. Read `audio` messages concurrently and stream them to playback.
## Streaming LLM Tokens to TTS
```python theme={null}
import asyncio
import gradium
async def play_pcm(chunk: bytes):
# Replace with your audio playback sink.
...
async def phrase_chunks(llm_tokens):
buffer = []
async for token in llm_tokens:
buffer.append(token)
text = "".join(buffer)
if text.endswith((".", "?", "!", ", ")) or len(text) > 80:
yield text
buffer = []
if buffer:
yield "".join(buffer) + " "
async def speak_llm(llm_tokens):
client = gradium.client.GradiumClient(api_key="your-api-key")
async with client.tts_realtime(
voice_id="YTpq7expH9539ERJ",
output_format="pcm",
) as tts:
async def sender():
async for chunk in phrase_chunks(llm_tokens):
await tts.send_text(chunk)
await tts.send_eos()
async def receiver():
async for msg in tts:
if msg["type"] == "audio":
await play_pcm(msg["audio"])
elif msg["type"] == "end_of_stream":
return
await asyncio.gather(sender(), receiver())
```
## Chunking Rules
* Send complete words, phrases, or sentences.
* Keep punctuation attached to the preceding word.
* Avoid sending one token per message unless tokens are already clean
word chunks.
* Add `` at natural boundaries, especially when the LLM has
completed the response.
* Avoid frequent flushes; they reduce the model's context and can make
prosody choppy.
## Handling Interruptions
If the user interrupts the agent:
1. Stop sending new text.
2. Stop playback locally.
3. Close the current WebSocket.
4. Open a fresh WebSocket for the next answer.
For several independent replies on one connection, use
[Multiplexing](/guides/multiplexing) and route audio by
`client_req_id`.
## Related
Full TTS streaming guide.
Setup, ready, input, flush, end, and errors.
# Telephony Audio Formats
Source: https://docs.gradium.ai/guides/recipes/telephony-audio
Use mu-law, A-law, and low-sample-rate PCM with Gradium voice APIs
Telephony providers commonly send 8 kHz mono audio encoded as mu-law
or A-law. Gradium supports those formats directly for STT and can
produce telephony-friendly TTS output.
## Recommended Formats
| Use case | Gradium format | Notes |
| - | - | - |
| Incoming PSTN audio to STT | `ulaw_8000` or `alaw_8000` | Use the encoding your telephony provider sends. |
| Incoming linear PCM to STT | `pcm_8000` or `pcm_16000` | 16-bit signed little-endian mono PCM. |
| Outgoing audio to PSTN | `ulaw_8000` or `alaw_8000` | Avoid resampling in your telephony bridge when possible. |
| Higher quality app audio | `pcm_24000`, `pcm_48000`, `wav`, or `opus` | Use outside PSTN constraints. |
## STT WebSocket Setup
```json theme={null}
{
"type": "setup",
"model_name": "default",
"input_format": "ulaw_8000",
"json_config": {"language": "en"}
}
```
Then send base64-encoded audio payloads:
```json theme={null}
{"type": "audio", "audio": "base64_encoded_ulaw_chunk"}
```
## TTS WebSocket Setup
```json theme={null}
{
"type": "setup",
"voice_id": "YTpq7expH9539ERJ",
"output_format": "ulaw_8000"
}
```
The `audio` messages contain base64-encoded mu-law chunks that can be
forwarded to a telephony media stream.
## Chunk Size Guidance
Many telephony media streams send 20 ms frames. Gradium accepts small
chunks, but batching to roughly 80 ms can reduce message overhead while
keeping latency low:
| Format | 20 ms payload | 80 ms payload |
| - | -: | -: |
| `ulaw_8000` / `alaw_8000` | 160 bytes | 640 bytes |
| `pcm_8000` | 320 bytes | 1280 bytes |
| `pcm_16000` | 640 bytes | 2560 bytes |
## Bridge Checklist
* Preserve the provider's stream/session ID in your own logs.
* Use one Gradium STT session per caller audio stream.
* Set `input_format` to the actual bytes you forward.
* For TTS, request an output format your provider can play directly.
* Use STT `step` messages or your provider's VAD to decide when to
send the user's turn to an agent.
* Close the WebSocket and start a new session if the call leg changes
format.
## Related
Real-time transcription with VAD and flush.
Stream generated audio back to your telephony bridge.
# Turn-Taking with Semantic VAD
Source: https://docs.gradium.ai/guides/recipes/turn-taking
Use STT semantic VAD and adaptive delay to decide when a user has finished speaking
Gradium STT emits `step` messages every 80 ms. Each `step` contains
semantic VAD predictions: the probability that the speaker will be
inactive at several future horizons. Use those probabilities together
with `delay_in_frames` and `flush` to build turn-taking that feels
responsive without cutting people off.
Use these signals to decide when an agent should respond.
## Basic Rule
Start with the longest horizon and a threshold around `0.5`:
```python theme={null}
def turn_has_probably_ended(msg):
if msg["type"] != "step" or not msg["vad"]:
return False
horizon = msg["vad"][-1]
return horizon["inactivity_prob"] > 0.5
```
This is a starting point, not a universal rule. Tune per application:
| Product feel | Suggested tuning |
| - | - |
| Fast assistant | Shorter horizon or lower threshold. |
| Careful assistant | Longer horizon or higher threshold. |
| Noisy telephony | Require several consecutive high-confidence steps. |
| Dictation | Prefer explicit user controls or longer silence. |
## Full Loop
```python theme={null}
import asyncio
import gradium
async def transcribe_turns(audio_source):
client = gradium.client.GradiumClient(api_key="your-api-key")
async with client.stt_realtime(
model_name="default",
input_format="pcm",
json_config={"language": "en", "delay_in_frames": 16},
) as stt:
transcript = []
high_vad_steps = 0
async def producer():
async for chunk in audio_source:
await stt.send_audio(chunk)
await stt.send_eos()
async def consumer():
nonlocal high_vad_steps, transcript
async for msg in stt:
if msg["type"] == "text":
transcript.append(msg["text"])
elif msg["type"] == "step":
inactivity = msg["vad"][-1]["inactivity_prob"]
high_vad_steps = high_vad_steps + 1 if inactivity > 0.5 else 0
if high_vad_steps >= 3 and transcript:
await stt.send_flush(flush_id=1)
elif msg["type"] == "flushed":
text = " ".join(transcript).strip()
transcript = []
high_vad_steps = 0
await handle_user_turn(text)
elif msg["type"] == "end_of_stream":
return
await asyncio.gather(producer(), consumer())
```
## Adaptive Delay
`delay_in_frames` controls how much context the STT model uses before
emitting text. Each frame is 80 ms. Larger values can improve text
quality but delay output. Smaller values are more reactive but may be
less stable. Supported values are `7`, `8`, `10`, `12`, `14`, `16`,
`20`, `24`, `32`, `36`, and `48`.
When the app decides a turn has ended, `send_flush()` asks the server
to process outstanding audio and then returns a matching `flushed`
message. Treat `flushed` as the point where the current turn is ready
for the next stage of your agent pipeline.
## Related
STT message types, VAD details, and flushing.
Tune `language`, `temp`, `padding_bonus`, and `delay_in_frames`.
# Earlier releases
Source: https://docs.gradium.ai/guides/release-notes/2026-02
Gradium releases from early 2026
We're excited to announce major updates to the Gradium API and Gradium Studio, delivering enhanced text-to-speech (TTS) and speech-to-text (STT) capabilities for developers and enterprises.
**Advanced TTS and STT Models**
Experience our latest text-to-speech model and speech recognition model with improved audio quality, accuracy, and natural-sounding voice generation. Perfect for voice applications, transcription services, and conversational AI. Now by default.
**Custom Pronunciation Dictionaries**
Take control of speech synthesis with pronunciation dictionaries. Ensure brand names, technical terminology, and industry-specific acronyms are pronounced correctly every time. Ideal for healthcare, finance, and industry specific applications.
**WebSocket Multiplexing for Real-Time Audio**
Boost performance with multiplexing support: process multiple TTS or STT requests simultaneously over a single WebSocket connection. Reduce latency, minimize connection overhead, and scale efficiently for high-volume applications.
**Gradium Referral Program**
Love Gradium? Join our referral program: share Gradium with your network and earn up to 9M API credits while your referrals get exclusive discounts. Win-win for developers and businesses alike.
**AWS Marketplace & SageMaker AI Deployment**
Deploy Gradium TTS and ASR models directly on AWS SageMaker with full bidirectional streaming support. Accelerate your deployment timeline while keeping your voice AI infrastructure secure within your private cloud network. Now available on AWS Marketplace.
Feedbacks are always welcome, do not hesitate to join our [discord channel](https://discord.com/invite/bcysuPRzXE)!
# June 2026
Source: https://docs.gradium.ai/guides/release-notes/2026-06
Gradium releases from June 2026
Gradium TTS now runs on an upgraded default model focused on more expressive speech and better reliable pronunciation for production voice agents.
Improved natural prosody, phrasing, intonation, and pacing. This is especially noticeable in conversational turns and longer passages.
The model is better at spelling out email addresses, reference numbers, confirmation codes, and acronyms such as `API`, `SQL`, and `URL`.
The upgraded model is now the default. If your integration uses the default model, you are already on the new Gradium TTS model with no code changes required.
Every voice from the previous model generation is available on the upgraded model, including custom voices. You do not need to re-clone or recreate voices.
Integrations pinned to a specific model can switch by changing the model identifier. To stay on the previous model temporarily, set `model_name` to `gradium-tts-202602`. The previous model will be gradually phased out over the next few weeks.
# July 2026
Source: https://docs.gradium.ai/guides/release-notes/2026-07
Gradium releases from July 2026
A new Gradium Text-to-Speech model is available in public beta. It handles complex text natively, with no pre-processing or normalization on your side: phone numbers in national and international formats, email addresses, IBAN numbers, time expressions, reference codes, measurements, URLs and ordinals.
The beta is available through the API and in Gradium Studio. To use it from the API, set `model_name` to `gradium-tts-beta` in your setup message, over both the SDK and the WebSocket API. Your existing voices, including custom voices, work as-is. On the API the current production model remains the default, so nothing changes unless you opt in.
See [Installation](/guides/installation) and the [TTS WebSocket setup](/api-reference/endpoint/tts-websocket#setup) to get started. Everyone who sends feedback during the beta receives 1M Gradium credits.
A new Speech-to-Text model is now live on the standard endpoint. Transcription is more robust on low volume audio, so soft speakers and far field microphones are recognised with improved reliability. Endpointing is more reactive, so the end of an utterance is detected sooner and voice agents can start responding with less delay.
See [Turn-Taking](/guides/recipes/turn-taking) for using semantic VAD signals to decide when a speaker has finished.
Get the vocabulary specific to your product transcribed correctly: brand and product names, people, places, drugs, and jargon. Pass a dictionary of the terms you care about in your STT `json_config`, and the model gives them priority while decoding, in real time, with no retraining. Rare, specific words that a general model rarely sees now come through correctly.
The `keywords` block takes a `words` dictionary and a `boost` weight. `boost` is applied in log-probability space, so its effect is exponential and a small value goes a long way. We recommend a boost of `3`. A dictionary holds up to 500 single-token entries, matched by exact case and accent.
Dictionaries can also be saved in Gradium Studio under Vocabulary, in the Keyword boost tab, then used in a session by passing their id as `keyword_boost_id`.
See the [Keyword Boosting recipe](/guides/recipes/keyword-boosting) and [Transcription Settings](/guides/transcription-settings) for details.
# August 2026
Source: https://docs.gradium.ai/guides/release-notes/2026-08
Gradium releases from August 2026
A new Gradium Text-to-Speech model is live and is now the default. Existing voices, including custom voices, work as-is.
The model reads the structured entities that break voice agents in production, such as phone numbers, email addresses, IBANs and reference codes, with no pre-processing or text normalization needed. On [Coval](https://benchmarks.coval.ai/tts), time to first audio P50 is 216 ms, 170 ms faster than the previous model.
Full benchmarks, audio samples and the [open-sourced evaluation set](https://huggingface.co/datasets/gradium/tts-eval-customer-support-202608) are in the [release post](https://gradium.ai/blog/gradium-tts-latency-and-accuracy). To get started, see [Text-to-Speech](/guides/text-to-speech).
Speech-to-Text requests now require a `language` in `json_config`. Our ASR gives its best results when the language is known, so set it whenever you can: `en`, `fr`, `es`, `pt`, or `de`. When the language is not known ahead of time, pass `any` and the model detects it from the audio. Requests that already set a language or `any` are unaffected. See [Transcription Settings](/guides/transcription-settings#languages).
The Resources menu on the website now links straight to these release notes.
* **Model selector in Gradium Studio**: the [Text-to-Speech](/guides/text-to-speech) page now has a Model dropdown to switch between the production model and `gradium-tts-beta`, showing each `model_name` with a copy button.
* **Shareable studio links**: pre-configure a session with `?voice=`, `?model_name=`, `?text=` and `?language=` and share the full setup as a single link. Links now survive login and signup, so they work for recipients who are not signed in yet; voice IDs with ready-made links are on [Flagship Voices](/guides/voices/flagship-voices).
* **Onboarding for new accounts**: signing up or accepting an invitation now starts with a short three-step setup to get you up and running quickly.
* **Contact and feedback without leaving the studio**: the profile menu now has Support > Contact and Support > Feedback, the same forms as on the website, with attachments.
* **System appearance**: a new System theme option in the studio follows your OS light or dark preference live.
* **Feedback page on the website**: [gradium.ai/feedback](https://gradium.ai/feedback) is a standalone form for product feedback, open to everyone.
* **Live Translation demo**: pick the target language and the output voice in one "Translate to ... with the voice of ..." row on the [website demo](https://gradium.ai).
Our live translation models (`stt-translate` for translated transcription and `s2s-translate` for Speech-to-Speech) now produce more natural, more accurate translations, with better context understanding across a conversation. The quality gains are most noticeable on conversational speech.
See [Speech-to-Speech](/guides/speech-to-speech) and the [Speech-to-Text overview](/guides/speech-to-text-overview) to get started.
# September 2026
Source: https://docs.gradium.ai/guides/release-notes/2026-09
Gradium releases from September 2026
Our new Text-to-Speech model is now the default, with more expressive speech and lower latency. Existing requests using `default` use the new model without a setup change.
`gradium-tts-beta` also routes to this model until the next TTS beta arrives. Keep that alias to follow future betas, or use `default` to stay on stable releases.
`cfg_coef` is deprecated for TTS. Requests that include it are still accepted, but the new model ignores its value. Remove it from `json_config`. See [Model selection](/guides/text-to-speech#model-selection) and [Voice Settings](/guides/voice-settings).
Clean up a voice you already have without changing its language or who is speaking. `POST /voice-generator/enhance` takes a flagship voice, a clone or a candidate and returns candidates with background noise reduced. Audition and convert them like Voice Design candidates.
Voice Enhance is in beta; output quality will keep improving. See the [Voice Enhance guide](https://docs.gradium.ai/guides/voices/voice-enhance) to get started.
Our new Text-to-Speech model entered public beta with lower latency and more expressive speech. At beta launch, we measured time to first audio under 50 ms, against 225 ms for the then-production model. Latency varies with network conditions and input.
During the beta, setting `model_name` to `"gradium-tts-beta"` selected the new model. Requests using `default` stayed on the previous production model. See the September 30 release above for the current default.
The beta ignored `cfg_coef`: requests that included it were accepted, but it did not adjust voice similarity. See [Voice Settings](/guides/voice-settings) for current TTS settings.
Gradium has achieved ISO 27001 certification, independently audited by A-LIGN.
Visit our [Trust Center](https://trust.gradium.ai/) for certification details and [security documentation](/guides/security).
[Studio](https://studio.gradium.ai) has a redesigned voice catalogue, new voice search controls in Text-to-Speech, and more ways to preview voices in Voice Design.
* **Voice catalogue:** Customize layout between tile or list views, group voices by language, and choose which properties to display. Studio saves your display preferences, and voice type labels distinguish catalogue, generated and cloned voices.
* **Filters and insights:** Narrow the catalogue by language, accent, age, gender and other voice attributes. Click a bar in the insights panel to filter the results, or copy a link to share a filtered view.
* **Voice details:** Open a voice's details beside the catalogue. For your own voices, edit the name and properties directly in the panel. For catalog voices, browse the voice attributes.
* **TTS voice selection:** Search and filter voices in a dedicated side panel. Describe what you need, such as "French Parisian voice", to get suggested smart filters. If nothing matches, Studio suggests a prompt you can take into Voice Design.
* **Voice Design previews:** Try a voice with three sample texts provided by default, regenerate the audio, and choose a different sample rate to hear how it sounds at your target rate.
* **Voice Cloning:** Find Instant and Pro cloning under one entry. Short recordings and upload errors are reported in the capture step.
* **Updated typography:** Studio uses a new typeface for improved readability.
Create a voice from a written description, no audio sample needed. Describe the accent, age and character you need, generate up to five candidates, and keep your favourite. The voice works with the same Text-to-Speech endpoint as any catalogue voice.
In our blind listening evaluation, native speakers judged how well voices matched accent prompts. Gradium achieved a 72.6% win rate against ElevenLabs, Inworld, MiniMax and Fish Audio, counting ties as half a win, and ranked first in English, French, Spanish, Portuguese and German.
See the [Voice Design guide](https://docs.gradium.ai/guides/voices/voice-design) to get started.
A session can last up to 3000 seconds. See [Limits](/guides/limits#session-duration).
# October 2026
Source: https://docs.gradium.ai/guides/release-notes/2026-10
Gradium releases from October 2026
Gradium has completed its SOC 2 Type II attestation.
Visit our [Trust Center](https://trust.gradium.ai/) for [security documentation](/guides/security).
Make a voice you already have speak another language, or another accent, without changing who is speaking. Point `POST /voice-generator/localize` at a flagship voice or a custom voice, pick an accent, and get candidates back in seconds. Audition and convert them like Voice Design candidates.
Also new: `GET /voice-generator/available-accents` lists the accents you can target, and the candidate listing now carries a per-kind config object (`generate_config`, `edit_language_config`, `enhance_config`). `prompt` is deprecated.
See the [Voice Localization guide](https://docs.gradium.ai/guides/voices/voice-localization) to get started.
Voice Enhance is out of beta: it is now a standard feature, with no change to the API. See the [Voice Enhance guide](https://docs.gradium.ai/guides/voices/voice-enhance).
# Security and compliance
Source: https://docs.gradium.ai/guides/security
SOC 2 Type II attestation, ISO 27001 certification and security resources for Gradium.
![A-LIGN SOC 2 Type 2 badge]()
SOC 2 Type II attestation
Gradium has completed its SOC 2 Type II attestation. Visit our Trust Center for security documentation.
View Trust Center →
![A-LIGN ISO 27001 certified badge]()
ISO 27001:2022 certified
Gradium has achieved ISO 27001 certification, independently audited by A-LIGN. Visit our Trust Center for certification details and security documentation.
View Trust Center →
For API data handling, see [data residency](/guides/data-residency) and [Zero Data Retention](/guides/faq#how-do-i-enable-zero-data-retention).
For security questions, contact [security@gradium.ai](mailto:security@gradium.ai). For a Data Processing Agreement, contact [support@gradium.ai](mailto:support@gradium.ai).
# Text-to-Speech on Baseten
Source: https://docs.gradium.ai/guides/self-hosted/baseten-tts
Stream Gradium TTS from a dedicated deployment hosted on Baseten
Your dedicated Gradium Text-to-Speech deployment runs on Baseten and
speaks the same streaming WebSocket protocol as the hosted Gradium API.
Only the connection details change: the URL and the authentication
header.
## Differences from the hosted API
| | Hosted Gradium API | Baseten deployment |
| - | - | - |
| WebSocket URL | `wss://api.gradium.ai/api/speech/tts` | The URL Baseten gives you, e.g. `wss-host.example.com/websocket` — the TTS WebSocket is served at that URL directly |
| Authentication | `x-api-key: ` | `Authorization: Api-Key ` |
| Voice catalog | Full voice library + your custom voices | The voices baked into the deployment image (see [Available voices](#available-voices)) |
| Pronunciation dictionaries (`pronunciation_id`) | Available | Not available — use inline [`rewrites`](#setup-parameters) instead |
## Prerequisites
* The WebSocket URL of your deployment (provided by Baseten / Gradium),
of the form `wss-host.example.com/websocket`.
* A Baseten API key with access to the deployment.
* Python 3.10+ with the Gradium SDK, version **0.6.1 or later**
(`pip install "gradium>=0.6.1"`). Earlier versions cannot override the
TTS route and will not reach a self-hosted deployment.
## Quickstart
The SDK repository ships a ready-made client for self-hosted
deployments:
[`examples/self_hosted/example_tts.py`](https://github.com/gradium-ai/gradium-py/blob/main/examples/self_hosted/example_tts.py).
It connects to the deployment, synthesizes the given text, and writes
the audio to a file:
```bash theme={null}
uv run examples/self_hosted/example_tts.py \
--base-url wss-host.example.com/websocket \
--api-key "" \
--text "Hello from Baseten!" \
--voice NbpkqMVS3CJeq2j8 \
--output-format wav \
--output out.wav
```
`--api-key` takes your **Baseten** API key — the example client sends
it in Baseten's `Authorization: Api-Key ...` header, not in the hosted
API's `x-api-key` header.
### Minimal client
The example boils down to two adjustments to the standard
`GradiumClient`, both worth copying into your own code:
1. **`tts_route=""`** — on the hosted API the client appends
`speech/tts` to the base URL; on Baseten the TTS WebSocket is served
at the deployment URL directly.
2. **Auth header override** — Baseten authenticates with
`Authorization: Api-Key `.
```python theme={null}
import asyncio
from gradium import GradiumClient
from gradium.client import SOURCE
class SelfHostedTTSClient(GradiumClient):
"""GradiumClient for a self-hosted / Baseten deployment."""
@property
def headers(self) -> dict:
return {
"Authorization": f"Api-Key {self._api_key}",
"x-api-source": SOURCE,
}
async def main():
client = SelfHostedTTSClient(
base_url="wss-host.example.com/websocket",
api_key="",
tts_route="", # the TTS WebSocket is served at the base URL directly
)
setup = {"voice_id": "NbpkqMVS3CJeq2j8", "output_format": "wav"}
stream = await client.tts_stream(setup, "Hello from Baseten!")
audio = b"".join([chunk async for chunk in stream.iter_bytes()])
with open("out.wav", "wb") as f:
f.write(audio)
asyncio.run(main())
```
All three SDK shapes work against the deployment exactly as described
in the [Text-to-Speech guide](/guides/text-to-speech): `tts_realtime`
for concurrent send/receive, `tts_stream` for an async iterator of
audio chunks, and `tts` for buffered one-shot synthesis.
## Setup parameters
* **`voice_id`**: The id of the voice to use, from
[Available voices](#available-voices) below. Omit it to use the
deployment's default voice.
* **`output_format`**: `"wav"` or `"pcm"`. PCM output is 48kHz,
16-bit signed integer, mono.
* **`json_config`**: Advanced voice settings such as temperature and
speed. See [Voice Settings](/guides/voice-settings).
* **`rewrites`**: Inline text rewrite rules, as a list of
`[original, rewrite, case_sensitive]` entries. See
[Text Rewriting](/guides/text-rewriting).
## Direct WebSocket
From a non-Python runtime, talk to the WebSocket directly. The message
flow is the same as the
[TTS WebSocket reference](/api-reference/endpoint/tts-websocket):
send `setup`, wait for `ready`, send `text` and `end_of_stream`,
receive `audio` / `text` messages until `end_of_stream`.
```bash theme={null}
wscat -c "wss:///websocket" \
-H "Authorization: Api-Key $BASETEN_API_KEY"
# After connection, type:
# {"type":"setup","voice_id":"NbpkqMVS3CJeq2j8","output_format":"wav"}
# {"type":"text","text":"Hello, world!"}
# {"type":"end_of_stream"}
```
The only difference from the hosted reference: `pronunciation_id` is
not available — use inline [`rewrites`](#setup-parameters) instead.
## Troubleshooting
| Symptom | Cause | Fix |
| - | - | - |
| Error: `voice_id '' not found` | The id isn't baked into the deployment image | Check the id against [Available voices](#available-voices) |
| Connection rejected (401/403) | Wrong auth header | Send `Authorization: Api-Key ` — not `x-api-key` |
| WebSocket closes right after connecting | The deployment replica is still warming up (\~3 min after a cold start or scale-up) | Retry with backoff |
## Available voices
The deployment image ships with **346 voices** plus a built-in default
voice (used when `voice_id` is omitted). All 66
[flagship voices](/guides/voices/flagship-voices) are included — the
tables below list them per language; the full id list follows. Pass
the id as the `voice_id` setup field — click an id to copy it. You can
also listen to and test the voices in the official
[Gradium Studio](https://studio.gradium.ai/).
Every id below is baked into the deployment image and usable as the
`voice_id` setup field, including the flagship voices listed above.
```text theme={null}
wGhY_zZCoQ5gB0ce
YHOBjtajNBEHUI_K
5UkFVe2B8OqLo-5R
KRo-uwfno-KcEgBM
BbLb4TxdlrldgpHI
Ow5IKhni2ED3Xxhl
ixaCTlZ5Xqf2XzQH
nlYB3lceu4gwj9Eu
KWJiFWu2O9nMPYcR
TJv-kucMsUo24VQe
74asmf7CXzjfopIX
4rdlkbxRv4m3UQTW
4u2uvwrHdTA2gRnZ
P0GYBrxlhTy5CC87
6PWnV0Nq4wu7RVBT
dEcrv3B8XGHoox2_
t-_TS1e-0GzDAX02
uem82D50GRv2Dwma
dME3IWyZBvmh1n1q
gDT1nz7Ie36ZhL-C
sVLgzKMqaptUdaY8
Eu9iL_CYe8N-Gkx_
7aEKz4P1ogZ0UsRP
fJDF4lEH590XplFv
kfzLbcdE_yXgLeUI
FchWfJlI2jr27eUg
7HhpTMy55D4HkXen
MhsYZQ4bIfcDpokF
MGiwMOFxVe4a2aSU
9VXl5t2IMagUQAzg
91EdXxJDbWICDBgz
NuUr_x5V90hSHzCJ
jvPx8j8zLGQ3utZz
p1fSBpcmVWngBqVd
vMYQUSzm6GRkJX6d
B36pbz5_UoWn4BDl
MZWrEHL2Fe_uc2Rv
pYcGZz9VOo4n2ynh
m3lIeODdTQ3bOh4z
ZeL1KGaZ4BZ2w0Np
LAmPTQZkwYJKRCKt
s_k3kLBbgeK9-xUg
A3UKMLXQUzknYpQa
4SZHfMpw-p46Ywgs
hP6WA-7ybEGApJ68
9UogMEa01dHR9Xbc
Qit9Oc9fEO9yXsVw
AByHrwi1S-yLzW-s
8sWSyTC7byLsbHkr
NvHEAMGiPT4u8iT-
vtG8ddh4IN32Otad
r2sIQdqqoqgRJuXw
L6OaiBybqikfCBk0
fTtFWGZZPge8OHrP
KUpE0JVhjiIzp1Fk
_6Aslh2DxfmnRLmP
jtEKaLYNn6iif5PR
Lxc7YlPC8ckLJA8H
7c5UOKm7AiBgJADg
GZkS2KENPVQjIadM
qkROcjqdyBhXWSmb
cLONiZ4hQ8VpQ4Sz
CF0NgaMwHMMrHZn0
Zd5POlBGSbD-JBXF
N8xxxD_d-ZinGVI4
m86j6D7UZpGzHsNu
_di_fkYdBWrbSGIL
GmGF_3ETsY2Zq7_w
kAoOc9Yb5EQDzA-N
Hdf5cdfaGrLDTD63
lP7D1y02OQFtffU3
WHINGSh2X5oidrhY
dh0EzP6jCroK6prq
ru5NfxcQb-xjB2ov
aHzItRuCKzZxzeK-
twLGV8mrH_ycNpUn
NbpkqMVS3CJeq2j8
w9V1722uEmTkWqnR
OHbzRwvaEWAp8ncr
biuhvu17TxVKOcyy
YKeBw3OV1RgpdhLh
POBHtemksfWQbng0
H3Rh9kJcd4gZidvN
8IWnaR9UcTNGRihW
Tek4tJXiX6_yvXq7
ifeL_LpqHFUEPI1Z
zyla-_bhVQtNTBdT
6MFfc37kq0sBjBjy
zUZZyO5dnCURpUJZ
b35yykvVppLXyw_l
25AzBFyp6svYnJsj
Y4iYxS8PBX-bazgX
xq0vDziADfAmg6Uh
iEu63s1rhn_kegTr
biPZlD1tJvi7Ixhq
J8c9KBRYAGGYwjns
axlOaUiFyOZhy4nv
ubuXFxVQwVYnZQhy
cc9VTN6fVa4s37K0
qTA0lxFpynJdoxx7
P4GqVY98hjQSvkiu
xu7iJ_fn2ElcWp2s
B09t5S64xLaKwXeW
GAYOBsMItSHuvInX
HvE1EoQDBLHnT7wd
6oIkS98REoVZ1dEw
ImBVnxSeLsdCfNIV
0y1VZjPabOBU3rWy
3hQIj8JOo7bU31Jw
QkmUhBH4hIV2_BkY
20zdyYrQPzKlCwkk
zba0owtqy4Gnewn9
LFZvm12tW_z0xfGo
Kf5m22mROozoMWj3
0LMAi0x_YVG_GLeM
QY_BJKHMElKDO12-
yyS1KYWs6mXoEw7D
jBULVCDhf05tOJN5
3ZKKapPOvuWFcw9f
lbpBQTVCOcOHJ5zS
YTpq7expH9539ERJ
QETTJoT4n_WmpL3w
nMNZ0sOWZVbKyjaI
aNiSRZ0BhQxO1FPx
p6Uutkyi3j2iNAUu
zIGaffB0kKEBG_8u
yHyu3PRfSmmiL2a4
6XwZeudK_gn719g5
Jlh1B0PKQJyup0sQ
SqFfhmAgR2XdN83R
Fmt16x6anKfMMeSx
uCqxlQCKi8sPHwG2
DUFZMTh4n-53Ly9O
ycGyxoEy9wLaX11R
-dOnYAX4N4GqSOee
YnWEONxJy7ptGhfb
fd7e1fLVAAJzzs8P
-uP9MuGtBqAvEyxI
M-FvVo9c-jGR4PgP
iNMTQyEFCzAONPit
73lMH7Zcc411nxJz
Ve1zknlflaRwcAQw
isyT17KHEj84P9w9
4ubKCfFxLeBg-cbl
KpDAXeGeen7P9Uri
ws0Wb0PZXl21_Bbz
24cfpJbYGXZLE39T
T4yRIRCLji61Fz-N
AaTW_13X1yYe_OnX
YE0-JPiElafJrZaC
B6aHVROMF8FuKR07
R3L8t75ZEoZCPUA9
yPxeHKlCzaHeKd_V
Mj0Pzs94jCw8oVOC
hx1RAC4Lqd9xyTAr
8QUaJGjSFdgHkuI8
mxcKXLymdLQCdlEq
EbIA5CIcQoa6NNd2
QZtWUy8jmIroWiOu
k2B3TJiffePxjeBn
1VAVLmmbQFDw7TMn
EzmLkNorEpZG_oNv
HBfu9XA3QfzAG1MN
D-IpHY1UI0iX9xQD
WWHSNJCSTm77dyGd
YUKEEk7Y4Igsj1Ts
zhH3lPUo-JxmlOJT
VAb2M8nKHlUUZBk4
W5htOuyiFI4Fwhxs
AroCL6f1qizjiZ_a
kw_VWSocR7vyA9Ty
rIYDMY3dLccdauWA
xynYWquoAsrvM7UY
IB53xJtufx1sbfbt
lPCVUcicz2XRaLE3
s0PhgjzOTRD5wo5L
h6qFHXR3-bqPg_PE
Bwl2KLUPxf82_ZaJ
k1wgs3k8-wRxTJO6
wT1bHy1Vq_0Bn73I
9c8MGnH1yQ4eN73X
8nsAoui8Y5RK9PYw
_cP-0vSYfMmzR4al
aCWBiYUiQ4VwW8_b
pdcyd1mLmo0fcg3O
eorxD0DWv--n7l3p
r5WB0b126tlHSrku
Gijj_GPBfJVcP-FZ
PqjKPYFyGNsg1YU-
6WgZTf6erRSiBlZi
zpmn3GOfiU_i5QGo
h39kz1iyoymcjcqh
-8ZoUJpVU98rxpv9
zdE2H9vw2vcMl_Pt
2AtP1urAQkZaeI2U
ynR4CAbXMiOv-vGC
auZu0iT-fniQ4cJd
Du_Dcv4fgXBDdubR
22YWyuFACaMHsPh5
SG3KnxbSOkkrY097
Yee42wDKxEFHi0BS
zyLIanWKViHkc6Wp
d9Fl9x8luXXX7u6E
2j8TWGsIiUl4G3kj
mn5sS7D8kYKETZXA
OceLYI_PPbqsdgdV
X-wgJsZwQKhfebgK
fggSYM_FGJ30QTTl
bDlMqRew31ZJwrD-
u8rA2xOF_0LRnNSb
gTAO-3xLZ8_WSfbm
m7fJRmVaJjG2TL1c
fs2Qj_X2Z2WvWJSU
QZMzHBlnJRjll_71
bvNlBZ3DWDoVy_Yc
T7UL6gmeDqqYiVe1
VeVmpxxbyJiWrGNG
knw-ddWDPNORRA4Z
IBVzgY91NZ1IJ0oP
i1kmq28cO60ia35K
SqHUVuEiTPSlIB5r
exG4bLr-lZ_bI0jF
3bIdO9CHnAh_pRAf
x69x43aS-5mVLCX2
CjQcj4yeIs6h0uAb
cuXxqSrGVntdhFpZ
QPHuXnvRPQ57oXYy
pxKsJ_4kEMid5XpZ
8dBmiTurwb7KcxLY
LqFNS0u6EII7VHBx
apU2CMobTyu92tZj
ikbJkd83GvuyoSLb
EfuzJVuTmw_mA7PC
AqRuVz8-e8u3BR00
n2Gv34jje2ZiiNzK
ptMwY_gvmFxXMmDf
aW5dxfdkzIFCIdXc
6Mp6PGnaCdb-US21
GJSxJhSTPAGIPDwy
8Tm8RKFEbnkRtkdA
MQC0U1yWvZXrppaF
UX3Hi2ZmK7tT0c3G
4nAcNUlNhEA_Kyjo
yU6yxQ3e8LKRwU84
2V3TjbyQGPlkY6ON
Cw79FL0p0J6UM9El
ZZb4X9ueHSdRlv9q
KNYHZTB8ZqdAZv5Q
-0MuXG9RcCsuSVtb
J2qsArcdozbto5Hn
wYY8mXKrKtwKsaXZ
t1Y_yKjku5R46F9t
3-pqEMoGtIq7wXtH
8eZwfGLoSF2N0RB3
-WFy9WtlQNE-dEV2
XnSnbQW98he4aULg
1D38wv1wp-H7QcyM
uF8PfAXrv6qU9UEM
HndphaVV7KTCfKQT
4NU5PqxX2BdMEtWe
ZOiGbnYdgKSBM_rH
RPw-aWdY8NBiIWeg
h2o5CDDhV5wE3Bwi
42-EbMFThYfhVB83
cu0XE3Cxmg_GmSJ3
D8iRHK1qJhqfE00v
H0GE4TqfCQGmpQhL
ApPgTz3nMHOsWxhK
wBgI9XmASQwvQ13w
XFttJvHwReWtWQNQ
78zAgQK6xmExb8wS
wPx6HPbUQkaUHGhq
9O8ZawShJ7UwURjK
6tFmjkrmrdhO2bXV
XJc-Y9tkSd1UA7s4
T2NDxsof9FHYxgJj
hOhCtzjR-cRG4T5T
df4Al5gt14Am4Qaf
--9DFXOPx8kJFsbe
uycTGmIXbw_Y83p9
v5lib8tjaosy5sxQ
xki1DK6Ks6tuDmcb
YHkMHL6WppbXd42a
Z5GIOZR45ieZ8M-W
lt88kyLfD8Mqemla
-qKylkN2UPxd7Mmg
lSVEPWl_N_7MtcHe
O0uTTRx5zcetDFX4
kr-Om35JRqmA3Hzq
56DcpvEI0Gawpidh
FOFDH8py3aghc5kb
hXjVvZ6oDDGQAQFj
WxHB2b5HxA0Kuq5u
dK5Glio51HTxdMu0
Abqwk2RWxlBEyv0j
KEMqb7dQlTCAEUx6
9LhjfdN9LOrygqDi
bauuigqCZbJFfk5q
ZsVFAOnjnEPxJVDI
AEJ61XaIaRill4cJ
9QHzSiOYUD-RzEzM
sz-H9BxaRaqxQ2S0
FRTqjB2TL-Ix9GXW
3mM3xaoFjNMQa22C
AySdCEnP2nqRo1WM
-_aUUFZaJ0CT1gks
s4CzgVHP5cEkB9LD
hAdJ9w9xBQkFgrRl
J4XbCGPYNMigXcfZ
NoJdNY6JTz-VJLwz
xB86uC_i8sO2U41-
L7890s1B44FqSiGC
vbg20SqFS_gBntTQ
VXA4-0_ZN4o8q3vK
lHL_BeE8kcYk_PFm
FvHwyEGJDAnhCOmy
IvU7qOuioP04a4eX
d1HpinVF79zaGnJY
cMvT_WqtiQCCf9c4
yk80SYvdMuQFF1MD
mweCx1v9-T6gDXH8
3YlG68yLpoTW8HEj
vzanWTXLIajkUaaT
IlWrXUM6C11vXm1S
8_M2uwvmyM-BadY9
aq7ltaIQ6ZJUY0jR
RhI-l8fGE2DtXgXV
HtgP9v8SoWbq_jxi
3jUdJyOi9pgbxBTK
2H4HY2CBNyJHBCrP
IIZIkBSZAmb9nFZb
PS7enm5lVZiIvEKV
b-1LP0pKWL1tNgml
c8BzreHTk1GG2R4z
mmLFHtCjt_6jw0vT
sX0PcxM_Ie2ctBGH
sBLwTd5womVX8JOw
TXbEUrHXNFlYBBKb
Xtp0vUDvtAfi1xkH
BPHlOW9jPs79KtW4
4NLtOv1m0azv9rGL
VDwnGxAo68C8U8vC
s58clrg2fe0MO7Y-
yHToO6ssaQHz5kIP
n7vovxcDTVG4gClo
tWll9uiMafMXfOGw
```
# Speech-to-Speech (WebSocket)
Source: https://docs.gradium.ai/guides/speech-to-speech
Real-time speech translation over WebSocket: transcribe, translate, and re-synthesize
Gradium Speech-to-Speech (S2S) streams input audio to the server over a
single duplex WebSocket and streams the translated output audio plus
its transcript back. The audio is transcribed, translated into
`target_language`, and re-synthesized with the voice you select. This
guide covers streaming via the Python SDK.
The overview compares `s2s_realtime`, `s2s_stream`, and the buffered
`s2s` helper.
## Quickstart
The shortest path: open an `s2s_realtime` context manager, push audio
chunks, and iterate to receive output audio and text.
```python theme={null}
import asyncio
import gradium
async def main():
client = gradium.client.GradiumClient(api_key="your-api-key")
async with client.s2s_realtime(
model_name="s2s-translate",
stt_model_name="stt-translate",
tts_model_name="default",
input_format="pcm",
output_format="pcm",
voice_id="YTpq7expH9539ERJ", # required; must be a voice in the target language
json_config={"target_language": "en"},
wait_for_ready_on_start=True,
) as s2s:
# Producer: push 80 ms PCM frames as they're captured.
async def producer():
async for chunk in microphone_chunks(): # 1920-sample frames
await s2s.send_audio(chunk)
await s2s.send_eos()
# Consumer: collect output audio, print the transcript as it arrives.
out = []
async def consumer():
async for msg in s2s:
if msg["type"] == "audio":
out.append(msg["audio"]) # already-decoded bytes
elif msg["type"] == "text":
print(msg["text"], end=" ", flush=True)
elif msg["type"] == "end_of_stream":
return
await asyncio.gather(producer(), consumer())
asyncio.run(main())
```
Expected input: 24 kHz, 16-bit signed mono PCM in 1920-sample (80 ms)
chunks. Output `pcm` is 48 kHz, 16-bit signed mono. Other formats are
supported, see [Setup Parameters](#setup-parameters).
## Choosing the SDK API
The Python SDK exposes three ways to drive S2S over the WebSocket
endpoint. All talk to `wss://api.gradium.ai/api/speech/s2s`: choose
based on how your audio source is shaped.
| Use case | Method |
| - | - |
| You're producing audio live (microphone, telephony, agent loop) and want output as it arrives | `s2s_realtime` |
| You can iterate the audio end-to-end (file, in-memory buffer, finite generator) and want to stream output back | `s2s_stream` |
| You have a complete file and just want the finished output audio plus transcript | `s2s` (buffered) |
Setup is **keyword arguments** for `s2s_realtime` and a **dict** for
`s2s_stream` / `s2s`. The wire-level `setup` message they produce is the
same.
## Real-time push: `s2s_realtime`
Use `s2s_realtime` when audio is being produced live (microphone,
telephony, conversational agent) and you need output audio and text as
they arrive. It's an async context manager, push audio with
`send_audio()` and iterate the same object to receive every message
type.
```python theme={null}
import asyncio
import gradium
async def main():
client = gradium.client.GradiumClient(api_key="your-api-key")
async with client.s2s_realtime(
model_name="s2s-translate",
stt_model_name="stt-translate",
tts_model_name="default",
input_format="pcm",
output_format="pcm",
voice_id="YTpq7expH9539ERJ", # required; must be a voice in the target language
json_config={"target_language": "en"},
wait_for_ready_on_start=True,
) as s2s:
print("Ready:", s2s.ready)
async def producer():
async for chunk in microphone_chunks(): # 80 ms PCM frames
await s2s.send_audio(chunk)
await s2s.send_eos()
out = []
async def consumer():
async for msg in s2s:
if msg["type"] == "audio":
out.append(msg["audio"])
elif msg["type"] == "text":
print(msg["text"], end=" ", flush=True)
elif msg["type"] == "end_of_stream":
return
await asyncio.gather(producer(), consumer())
if __name__ == "__main__":
asyncio.run(main())
```
`send_audio()` accepts `bytes` or a 1-D numpy array (`int16` or
`float32`) when `input_format` is a `pcm` variant. In the iterator,
`audio` messages already carry decoded `bytes` in `msg["audio"]`.
## Pull-based: `s2s_stream`
Use `s2s_stream` when the audio is already in hand (a file or a
complete in-memory buffer) but you still want to stream the output back
as it's produced. The function consumes your audio iterable and returns
an `S2SStream`.
```python theme={null}
import asyncio
import gradium
async def main():
client = gradium.client.GradiumClient(api_key="your-api-key")
async def audio_generator(audio_data, chunk_size=1920):
for i in range(0, len(audio_data), chunk_size):
yield audio_data[i : i + chunk_size]
setup = {
"model_name": "s2s-translate",
"stt_model_name": "stt-translate",
"tts_model_name": "default",
"input_format": "pcm",
"output_format": "wav",
"voice_id": "YTpq7expH9539ERJ",
"json_config": {"target_language": "en"},
}
stream = await client.s2s_stream(setup, audio_generator(audio_data))
with open("output.wav", "wb") as f:
async for chunk in stream.iter_audio():
f.write(chunk)
# Text segments are collected as audio is consumed.
for twt in stream._text_with_timestamps:
print(f"[{twt.start_s:.2f}s] {twt.text}")
if __name__ == "__main__":
asyncio.run(main())
```
`iter_audio()` yields output audio bytes and collects text segments
along the way. To handle every server message yourself (both `audio`
and `text` as dicts, with `audio` fields decoded), use `iter_events()`
instead.
## Buffered: `s2s`
Use `s2s` when you don't need results incrementally, hand it a complete
audio buffer and get back an `S2SResult` with the full output audio and
transcript.
```python theme={null}
import asyncio
import gradium
async def main():
client = gradium.client.GradiumClient(api_key="your-api-key")
setup = {
"model_name": "s2s-translate",
"stt_model_name": "stt-translate",
"tts_model_name": "default",
"input_format": "wav",
"output_format": "wav",
"voice_id": "YTpq7expH9539ERJ",
"json_config": {"target_language": "en"},
}
with open("speech.wav", "rb") as f:
result = await client.s2s(setup, f.read())
print(result.text)
with open("translated.wav", "wb") as f:
f.write(result.raw_data)
if __name__ == "__main__":
asyncio.run(main())
```
`audio` can be raw `bytes`, a 1-D numpy array (`int16`/`float32`, `pcm`
input only, `sample_rate=24000` required), or an async generator of
byte chunks. The returned `S2SResult` exposes `raw_data`,
`sample_rate`, `output_format`, `request_id`, `text`, and
`text_with_timestamps`; `pcm16()` / `pcm()` convert PCM output to numpy
arrays.
## Setup Parameters
* **`model_name`**: The S2S model to use. Use `"s2s-translate"` for live
translation, the only currently supported model.
* **`stt_model_name`**: Speech-to-text model used to transcribe the
input. Set to `"stt-translate"` for live translation.
* **`tts_model_name`**: Text-to-speech model used to synthesize the
output. Set to `"default"` for live translation.
* **`voice_id`** (required): Voice UID used for the synthesized output.
It must be a voice in the same language as `target_language`. See
[Voices](/guides/voices/overview).
* **`input_format`**: Audio format of the input. Supported values:
* `"pcm"`: raw PCM, 24 kHz, 16-bit signed little-endian, mono. Send
1920-sample (80 ms) chunks for best results.
* `"pcm_8000"`, `"pcm_16000"`, `"pcm_22050"`, `"pcm_24000"`,
`"pcm_44100"`, `"pcm_48000"`: raw PCM at the indicated sample rate
(16-bit signed little-endian, mono).
* `"wav"`: a valid WAV file with PCM data.
* `"opus"`: Ogg-wrapped Opus stream.
* `"ulaw_8000"` (alias `"mulaw_8000"`): mu-law encoded PCM at 8 kHz.
* `"alaw_8000"`: A-law encoded PCM at 8 kHz.
* **`output_format`**: Audio format of the output. Same value set as
`input_format`. For `"pcm"` the output is 48 kHz, 16-bit signed
little-endian, mono.
* **`json_config`**: Set `target_language` (e.g. `"en"`) to the
language to translate the speech into. The `voice_id` must be a voice
in this language. See
[Speech-to-Speech Settings](/guides/speech-to-speech-settings).
Unrecognised format values raise a server error.
## Message Types
When iterating over an `s2s_realtime` session you receive every message
the server sends:
* **`ready`**: connection accepted; carries `sample_rate` and
`frame_size` of the output. Available via `s2s.ready` when you pass
`wait_for_ready_on_start=True`.
* **`text`**: a translated transcript segment (in `target_language`)
with `start_s` / `stop_s` timestamps.
* **`audio`**: an output audio chunk. In the SDK iterator the `audio`
field is already decoded `bytes`.
* **`end_of_stream`**: terminal message after `send_eos()`.
For the wire-level schema (full field types and on-the-wire JSON), see
the [S2S WebSocket reference](/api-reference/endpoint/s2s-websocket).
## Direct WebSocket
If you don't want the gradium SDK, for example you're calling from a
non-Python runtime, or you want full control over the wire, you can
talk to the WebSocket protocol directly. The message shapes are
identical to what the SDK sends.
For interactive poking from a terminal, [`wscat`](https://github.com/websockets/wscat)
is the quickest way to confirm reachability and step through messages
by hand:
```bash theme={null}
wscat -c "wss://api.gradium.ai/api/speech/s2s" \
-H "x-api-key: your_api_key"
# After connection, type:
# {"type":"setup","model_name":"s2s-translate","stt_model_name":"stt-translate","tts_model_name":"default","input_format":"pcm","output_format":"pcm","voice_id":"YTpq7expH9539ERJ","json_config":{"target_language":"en"}}
```
For a client without the SDK, use any WebSocket library (here Python's
[`websockets`](https://websockets.readthedocs.io/)):
```python theme={null}
import asyncio
import base64
import json
import websockets
CHUNK_BYTES = 1920 * 2 # 80 ms at 24 kHz, 16-bit mono.
async def translate(api_key: str, pcm_audio: bytes):
setup = {
"type": "setup",
"model_name": "s2s-translate",
"stt_model_name": "stt-translate",
"tts_model_name": "default",
"input_format": "pcm",
"output_format": "pcm",
"voice_id": "YTpq7expH9539ERJ",
"json_config": {"target_language": "en"},
}
async with websockets.connect(
"wss://api.gradium.ai/api/speech/s2s",
additional_headers={"x-api-key": api_key},
) as ws:
await ws.send(json.dumps(setup))
ready = json.loads(await ws.recv())
assert ready["type"] == "ready"
async def producer():
for off in range(0, len(pcm_audio), CHUNK_BYTES):
chunk = pcm_audio[off : off + CHUNK_BYTES]
await ws.send(json.dumps({
"type": "audio",
"audio": base64.b64encode(chunk).decode(),
}))
await ws.send(json.dumps({"type": "end_of_stream"}))
out = []
async def consumer():
while True:
msg = json.loads(await ws.recv())
if msg["type"] == "audio":
out.append(base64.b64decode(msg["audio"]))
elif msg["type"] == "text":
print(msg["text"])
elif msg["type"] == "end_of_stream":
return
elif msg["type"] == "error":
raise RuntimeError(msg["message"])
await asyncio.gather(producer(), consumer())
asyncio.run(translate("your_api_key", open("input.pcm", "rb").read()))
```
For the full message schema (every field, every error code), see the
[S2S WebSocket reference](/api-reference/endpoint/s2s-websocket).
## Next steps
Complete wire-level schema: every message type, every field, every
error code.
`voice_id`, `target_language`, inner model selection, formats.
Pick the voice used for the synthesized output.
Error contracts across REST, WebSocket, and streamed responses.
# Speech-to-Speech Overview
Source: https://docs.gradium.ai/guides/speech-to-speech-overview
What Speech-to-Speech does and how to set it up
Gradium Speech-to-Speech (S2S) translates spoken audio from one language
into spoken audio in another, over a single duplex WebSocket. The input
audio is transcribed, translated into the target language, and
re-synthesized as output audio. You stream audio in and receive both the
synthesized output audio and the translated transcript back as they're
produced.
## Model
The only currently supported S2S model is **live translation**. To use
it, send a `setup` message with `model_name` set to `"s2s-translate"`,
`stt_model_name` set to `"stt-translate"`, and `tts_model_name` set to
`"default"`. Choose the target language
with `target_language` in `json_config`, and the output voice with
`voice_id` (required; it must be a voice in the target language).
```json theme={null}
{
"type": "setup",
"model_name": "s2s-translate",
"stt_model_name": "stt-translate",
"tts_model_name": "default",
"input_format": "pcm",
"output_format": "pcm",
"voice_id": "YTpq7expH9539ERJ",
"json_config": { "target_language": "en" }
}
```
For the full list of fields and formats, see
[Speech-to-Speech Settings](/guides/speech-to-speech-settings).
## Next steps
Stream audio in, output audio and transcript out, real-time or
buffered.
`voice_id`, `target_language`, inner model selection, formats.
Complete wire-level schema: every message type, every field, every
error code.
Pick the voice used for the synthesized output.
# Speech-to-Speech Settings
Source: https://docs.gradium.ai/guides/speech-to-speech-settings
Configure S2S: voice, translation target, inner models, and audio formats
A Speech-to-Speech request translates spoken audio into another
language. It is configured through the `setup` message: some fields
select the pipeline (models, voice, formats), and the translation
target is passed through `json_config`. In the Python SDK `json_config`
is a dict; the SDK serializes it to a JSON string on the wire for you.
These options apply to the [WebSocket](/guides/speech-to-speech) API
across all three SDK entry points (`s2s_realtime`, `s2s_stream`, and the
buffered `s2s`). For the STT-only and TTS-only knobs, see
[Transcription Settings](/guides/transcription-settings) and
[Voice Settings](/guides/voice-settings).
## Setup fields
| Field | Type | Required | Effect |
| - | - | - | - |
| `model_name` | string | Yes | S2S model alias. Use `"s2s-translate"` for live translation, the only currently supported model. |
| `stt_model_name` | string | No | Speech-to-text model used to transcribe the input. Set to `"stt-translate"` for live translation. |
| `tts_model_name` | string | No | Text-to-speech model used to synthesize the output. Set to `"default"` for live translation. |
| `voice_id` | string | Yes | Voice UID for the synthesized output. Must be a voice in the same language as `target_language`. See [Voices](/guides/voices/overview). |
| `input_format` | string | Yes | `pcm`, `wav`, `opus`, `ulaw_8000`, `alaw_8000`, or an explicit PCM rate such as `pcm_24000`. For `pcm`, input is 24 kHz, 16-bit signed mono. |
| `output_format` | string | Yes | Same value set as `input_format`. For `pcm`, output is 48 kHz, 16-bit signed mono. |
| `json_config` | object or string | No | Advanced pipeline settings, see below. |
| `client_req_id` | string | No | Correlates multiplexed requests. See [Multiplexing](/guides/multiplexing). |
| `close_ws_on_eos` | boolean | No | Defaults to `true`; set `false` to keep the socket open after `end_of_stream`. |
## `json_config` options
| Option | Type | Allowed values | Effect |
| - | - | - | - |
| `target_language` | string | `"en"`, `"fr"`, `"de"`, `"es"`, `"pt"` | Language to translate the speech into before synthesis. |
The transcribed text is translated into `target_language` before the
output audio is generated, and the `text` messages you receive back
carry the translated text. The `voice_id` you choose must be a voice in
this same language.
## Passing `json_config`
The same `json_config` payload is sent regardless of which SDK entry
point you use; only the call shape differs:
```python theme={null}
config = {"target_language": "en"}
# Real-time: setup is keyword arguments; json_config stays nested.
async with client.s2s_realtime(
model_name="s2s-translate",
stt_model_name="stt-translate",
tts_model_name="default",
input_format="pcm",
output_format="pcm",
voice_id="YTpq7expH9539ERJ",
json_config=config,
) as s2s:
...
# Pull-based / buffered: setup is a dict containing json_config.
setup = {
"model_name": "s2s-translate",
"stt_model_name": "stt-translate",
"tts_model_name": "default",
"input_format": "pcm",
"output_format": "wav",
"voice_id": "YTpq7expH9539ERJ",
"json_config": config,
}
stream = await client.s2s_stream(setup, audio_generator(audio_data))
```
## Next steps
Real-time, pull-based, and buffered S2S with the Python SDK.
Complete wire-level schema: every message type, every field, every
error code.
Pick the voice used for the synthesized output.
TTS-side options for the synthesis stage.
# Speech-to-Text (WebSocket)
Source: https://docs.gradium.ai/guides/speech-to-text
Real-time speech-to-text streaming over WebSocket with semantic VAD and flush
The Gradium Speech-to-Text model transcribes audio in real time over a
WebSocket, with semantic Voice Activity Detection (VAD) and on-demand
`flush` for low-latency turn-taking. This guide covers streaming via
the Python SDK.
For pre-recorded audio in hand, the REST POST endpoint is a single
HTTP call, see the REST guide.
## Quickstart
The shortest path: open an `stt_realtime` context manager, push audio
chunks, and iterate to receive transcripts.
```python theme={null}
import asyncio
import gradium
async def main():
client = gradium.client.GradiumClient(api_key="your-api-key")
stt_config = {"language": "en", "delay_in_frames": 16}
async with client.stt_realtime(
model_name="default",
input_format="pcm",
json_config=stt_config,
) as stt:
# Producer: push 80 ms PCM frames as they're captured.
async def producer():
async for chunk in microphone_chunks():
await stt.send_audio(chunk)
await stt.send_eos()
# Consumer: print transcripts as they arrive.
async def consumer():
async for msg in stt:
if msg["type"] == "text":
print(msg["text"], end=" ", flush=True)
elif msg["type"] == "end_of_stream":
return
await asyncio.gather(producer(), consumer())
asyncio.run(main())
```
Expected audio: 24 kHz, 16-bit signed mono PCM in 1920-sample (80 ms)
chunks. Other formats are supported, see [Setup
Parameters](#setup-parameters).
## Choosing the SDK API
The Python SDK exposes two ways to stream STT over the WebSocket
endpoint. Both talk to `wss://api.gradium.ai/api/speech/asr`: choose
based on how your audio source is shaped.
| Use case | Method |
| - | - |
| You can iterate the audio end-to-end (file, in-memory buffer, finite generator) and only need the transcript | `stt_stream` |
| You're producing audio live (microphone, telephony, agent loop) and want to react to VAD or flush mid-stream | `stt_realtime` |
`stt_stream` is a single async call that consumes your audio iterable
and returns an `STTStream` whose `iter_text()` yields
`TextWithTimestamps` segments. `stt_realtime` is an **async context
manager**: you push chunks with `send_audio()` and iterate the same
object to receive every message type, `text`, VAD `step`, `end_text`,
`flushed`.
Setup is a **dict** for `stt_stream` and **keyword arguments** for
`stt_realtime`. The wire-level `setup` message they produce is the
same.
## Pull-based: `stt_stream`
Use `stt_stream` when the audio is already in hand, for example when
transcribing a file or a complete in-memory buffer. The function
consumes your audio iterable and yields finalised text segments.
```python theme={null}
import asyncio
import gradium
async def main():
client = gradium.client.GradiumClient(api_key="your-api-key")
async def audio_generator(audio_data, chunk_size=1920):
for i in range(0, len(audio_data), chunk_size):
yield audio_data[i : i + chunk_size]
setup = {
"model_name": "default",
"input_format": "pcm",
"json_config": {"language": "en", "delay_in_frames": 16},
}
stream = await client.stt_stream(setup, audio_generator(audio_data))
async for segment in stream.iter_text():
print(f"[{segment.start_s:.2f}s] {segment.text}")
if __name__ == "__main__":
asyncio.run(main())
```
`iter_text()` filters to text segments only and yields
`TextWithTimestamps` objects (`text`, `start_s`, `stop_s`). To observe
VAD or `flushed` messages, use `stt_realtime` below.
## Real-time push: `stt_realtime`
Use `stt_realtime` when audio is being produced live (microphone,
telephony, conversational agent) and you need to react to VAD or flush
events as they arrive. It's an async context manager, push audio with
`send_audio()` and iterate the same object to receive every message
type.
```python theme={null}
import asyncio
import gradium
async def main():
client = gradium.client.GradiumClient(api_key="your-api-key")
async with client.stt_realtime(
model_name="default",
input_format="pcm",
json_config={"language": "en", "delay_in_frames": 16},
) as stt:
async def producer():
async for chunk in microphone_chunks(): # 80 ms PCM frames
await stt.send_audio(chunk)
await stt.send_eos()
async def consumer():
transcript = []
async for msg in stt:
if msg["type"] == "text":
transcript.append(msg["text"])
elif msg["type"] == "step":
if msg["vad"][2]["inactivity_prob"] > 0.5:
print(f"[turn] {' '.join(transcript)}")
transcript = []
elif msg["type"] == "end_of_stream":
return
await asyncio.gather(producer(), consumer())
if __name__ == "__main__":
asyncio.run(main())
```
Setup is passed as keyword arguments (`model_name`, `input_format`,
`json_config`, …). See [Setup Parameters](#setup-parameters) below.
## Setup Parameters
* **`model_name`**: The STT model to use (default: `"default"`).
* **`input_format`**: Audio format of the input. Supported values:
* `"pcm"`: raw PCM, 24 kHz, 16-bit signed little-endian, mono. Send
1920-sample (80 ms) chunks for best results.
* `"pcm_8000"`, `"pcm_16000"`, `"pcm_22050"`, `"pcm_24000"`,
`"pcm_44100"`, `"pcm_48000"`: raw PCM at the indicated sample rate
(16-bit signed little-endian, mono).
* `"wav"`: a valid WAV file with PCM data (`AudioFormat = 1` in the
WAV header). 16-, 24-, and 32-bit samples are supported.
* `"opus"`: Ogg-wrapped Opus stream.
* `"ulaw_8000"` (alias `"mulaw_8000"`): mu-law encoded PCM at 8 kHz.
* `"alaw_8000"`: A-law encoded PCM at 8 kHz.
* **`json_config`**: Advanced STT options. Pass a dict in the Python
SDK, or a JSON object/string in the raw WebSocket setup message.
Common keys are `language`, `target_language`, `temp`,
`padding_bonus`, and `delay_in_frames`.
Unrecognised values raise a server error.
## Language and delay example
Use `json_config.language` to set the audio language (`"any"` if unknown), and
`json_config.delay_in_frames` to choose the latency versus context
tradeoff. Each frame is 80 ms. Lower delay values are more reactive;
higher values give the model more context before emitting text.
```python Python SDK theme={null}
async with client.stt_realtime(
model_name="default",
input_format="pcm",
json_config={
"language": "en",
"delay_in_frames": 16,
},
) as stt:
...
```
```json WebSocket setup theme={null}
{
"type": "setup",
"model_name": "default",
"input_format": "pcm",
"json_config": {
"language": "en",
"delay_in_frames": 16
}
}
```
For translating STT models, use `target_language` or `language` to set
the output transcript language. See [Transcription
Settings](/guides/transcription-settings#languages).
## Message Types
When iterating over an `stt_realtime` session you receive every
message the server sends. The relevant types for transcription:
* **`text`**: a transcribed text segment with a `start_s` timestamp.
* **`end_text`**: finalises the previous `text` segment with a
`stop_s` timestamp. See [Timestamps](#timestamps).
* **`step`**: Voice Activity Detection information emitted every
80 ms. See [Voice Activity Detection (VAD)](#voice-activity-detection-vad).
* **`flushed`**: confirmation that all audio submitted before a
`send_flush()` has been processed. See [Flushing](#flushing).
* **`end_of_stream`**: terminal message after `send_eos()`.
For the wire-level schema (full field types and on-the-wire JSON), see
the [STT WebSocket reference](/api-reference/endpoint/stt-websocket).
```python theme={null}
async with client.stt_realtime(
model_name="default",
input_format="pcm",
json_config={"language": "en"},
) as stt:
# ... send audio in a producer task ...
async for msg in stt:
if msg["type"] == "text":
print(f"Transcription: {msg['text']}")
elif msg["type"] == "step":
# VAD steps arrive every 80 ms.
inactivity_prob = msg["vad"][2]["inactivity_prob"]
print(f"Inactivity probability: {inactivity_prob}")
elif msg["type"] == "flushed":
print(f"Flush complete: {msg['flush_id']}")
```
`stt_stream` exposes only finalised text segments via `iter_text()`. To
observe `step`, `end_text`, or `flushed` messages, use `stt_realtime`.
## Timestamps
Each `text` message carries a `start_s` field, the time in seconds
within the input audio at which the transcribed segment begins. When
the segment is finalised, the server emits an `end_text` message with
the corresponding `stop_s`:
```json theme={null}
{"type": "text", "text": "Hello world", "start_s": 0.5, "stream_id": 0}
{"type": "end_text", "stop_s": 2.5, "stream_id": 0}
```
Pair `text` and `end_text` by `stream_id` to recover `(text, start_s,
stop_s)` triples per segment.
Timestamps are emitted at **segment** granularity. For finer-grained
word or character offsets, derive them from the segment text.
## Voice Activity Detection (VAD)
Gradium's STT WebSocket emits a `step` message every 80 ms containing
**semantic VAD predictions across multiple horizons**: for each
horizon (in seconds) the model reports the probability that the
speaker will stay silent throughout that window. Longer horizons are
therefore a stronger claim. This is more informative than a simple
voiced/unvoiced flag and is designed for turn-taking in conversational
systems.
```json theme={null}
{
"type": "step",
"vad": [
{"horizon_s": 0.5, "inactivity_prob": 0.9140625},
{"horizon_s": 1.0, "inactivity_prob": 0.859375},
{"horizon_s": 2.0, "inactivity_prob": 0.53515625},
{"horizon_s": 3.0, "inactivity_prob": 0.40625}
],
"step_idx": 5,
"step_duration_s": 0.08,
"total_duration_s": 0.4
}
```
### Detecting end of turn
The recommended starting point is to look at the 1 s, 2 s, or 3 s
horizon and trigger when the inactivity probability crosses 0.5:
```python theme={null}
# vad is ordered by horizon: 0.5 s, 1.0 s, 2.0 s, 3.0 s
turn_ended = msg["vad"][3]["inactivity_prob"] > 0.5 # 3 s horizon
```
Tune the horizon and threshold per application: shorter horizons /
lower thresholds make the system more reactive but can cut speakers
off; longer horizons / higher thresholds are more conservative.
## Adaptive Delay
Gradium STT is intentionally not a fixed-latency black box. The
`delay_in_frames` option controls how much audio context the model
uses before emitting text. Each frame is 80 ms, and the server returns
the active value in the `ready.delay_in_frames` field.
Higher values give the model more context and can improve transcription
stability. Lower values reduce latency. Supported values are integers
from `7` through `55`. The default is `10` when you don't set it; `16`
is a good starting point when you want a little more stability for
interactive apps.
```python theme={null}
async with client.stt_realtime(
model_name="default",
input_format="pcm",
json_config={"language": "en", "delay_in_frames": 10},
wait_for_ready_on_start=True,
) as stt:
print(stt.ready["delay_in_frames"])
```
At a turn boundary, pair semantic VAD with `send_flush()`: when your
app decides the speaker is done, `flush` asks Gradium to process audio
already submitted and then returns `flushed` with the matching
`flush_id`. Internally, the worker tracks the adjusted delay after
flush events so segment timestamps remain aligned with the audio that
was processed.
VAD `step` messages are emitted on the WebSocket transport.
## Flushing
You can force the server to immediately process all buffered audio using `send_flush()`. This is useful when you need transcription results without waiting for the normal processing cadence, for example at a natural pause in conversation or before switching speakers.
The server will process all outstanding audio, return any pending text results, and then respond with a `flushed` message containing the same `flush_id` you provided.
```python theme={null}
async with client.stt_realtime(
model_name="default", input_format="pcm", json_config={"language": "en"}
) as stt:
# Send some audio
for i in range(0, len(audio_chunk), 1920):
await stt.send_audio(audio_chunk[i:i + 1920])
# Force processing of all buffered audio
await stt.send_flush(flush_id=1)
# Continue receiving - text results arrive, then a flushed confirmation
async for msg in stt:
if msg["type"] == "text":
print(f"Transcription: {msg['text']}")
elif msg["type"] == "flushed":
print(f"All audio up to flush {msg['flush_id']} has been processed")
break
```
## Tunable options
Models accept additional configuration via `json_config`, including
`language`, `temp`, `padding_bonus`, and `delay_in_frames`. Use
`delay_in_frames` as the adaptive delay control for your latency versus
quality tradeoff. See
[Transcription Settings](/guides/transcription-settings) for the full
table and how to pass `json_config` across the SDK and REST.
## Direct WebSocket
If you don't want the gradium SDK, for example you're calling from a
non-Python runtime, or you want full control over the wire, you can
talk to the WebSocket protocol directly. The message shapes are
identical to what the SDK sends.
For interactive poking from a terminal, [`wscat`](https://github.com/websockets/wscat)
is the quickest way to confirm reachability and step through messages
by hand:
```bash theme={null}
wscat -c "wss://api.gradium.ai/api/speech/asr" \
-H "x-api-key: your_api_key"
# After connection, type:
# {"type":"setup","model_name":"default","input_format":"pcm","json_config":{"language":"en","delay_in_frames":16}}
```
For a client without the SDK, use any WebSocket library
(here Python's [`websockets`](https://websockets.readthedocs.io/)):
```python theme={null}
import asyncio
import base64
import json
import websockets
CHUNK_BYTES = 1920 * 2 # 80 ms at 24 kHz, 16-bit mono.
async def transcribe(api_key: str, pcm_audio: bytes):
setup = {
"type": "setup",
"model_name": "default",
"input_format": "pcm",
"json_config": {
"language": "en",
"delay_in_frames": 16,
},
}
async with websockets.connect(
"wss://api.gradium.ai/api/speech/asr",
additional_headers={"x-api-key": api_key},
) as ws:
await ws.send(json.dumps(setup))
ready = json.loads(await ws.recv())
assert ready["type"] == "ready"
async def producer():
for off in range(0, len(pcm_audio), CHUNK_BYTES):
chunk = pcm_audio[off : off + CHUNK_BYTES]
await ws.send(json.dumps({
"type": "audio",
"audio": base64.b64encode(chunk).decode(),
}))
await ws.send(json.dumps({"type": "end_of_stream"}))
async def consumer():
while True:
msg = json.loads(await ws.recv())
if msg["type"] == "text":
print(msg["text"])
elif msg["type"] == "end_of_stream":
return
elif msg["type"] == "error":
raise RuntimeError(msg["message"])
await asyncio.gather(producer(), consumer())
asyncio.run(transcribe("your_api_key", open("input.pcm", "rb").read()))
```
For the full message schema (every field, every error code), see the
[STT WebSocket reference](/api-reference/endpoint/stt-websocket).
## Next steps
Complete wire-level schema: every message type, every field, every
error code.
Transcribe a complete audio file with a single HTTP request.
Browse complete voice-agent examples that use Gradium STT and TTS.
`language`, `temp`, `padding_bonus`, `delay_in_frames`, and how to
pass `json_config`.
Error contracts across REST, WebSocket, and streamed responses.
# Speech-to-Text Overview
Source: https://docs.gradium.ai/guides/speech-to-text-overview
When to use the WebSocket API vs the REST endpoint, and how the pieces fit together
Gradium exposes Speech-to-Text over two transports. They share the same
models; pick the transport that matches your audio source and latency
needs.
## Choosing a model
Two kinds of Speech-to-Text model are available, selected with `model_name`:
* **`"default"`** — a standard transcription model. The transcription is
produced in the language of the audio. Set `language` in `json_config`,
or pass `"any"` to auto-detect.
* **`"stt-translate"`** — a translating transcription model. The
transcription is generated in another language, which you choose with
`language` (or `target_language`). See [Transcription Settings](/guides/transcription-settings#languages).
Both kinds work identically over either transport.
## WebSocket vs REST
| If your audio is... | Use | Why |
| - | - | - |
| Live (microphone, telephony, agent loop) and you need sub-second turn-taking | [WebSocket](/guides/speech-to-text) | Lowest latency. Push audio as it's captured, get text and VAD signals as they're produced. React to end-of-turn in real time. |
| A complete file you already have on disk or in memory | [REST](/guides/speech-to-text-rest) | One HTTP POST with the audio in the body. No connection to manage; the server streams NDJSON results back as the response body. |
| In hand but you want VAD signals or to react to flush events | [WebSocket](/guides/speech-to-text) | Use `stt_stream` (pull-based) over the WebSocket. Same low latency, no manual connection management. |
| Browser microphone or mobile client audio | [Browser WebSockets](/guides/browser-websockets) | Use short-lived tokens instead of exposing an API key. |
| Telephony media streams | [Telephony Audio](/guides/recipes/telephony-audio) | Use `ulaw_8000`, `alaw_8000`, or low-sample-rate PCM directly. |
If you're transcribing pre-recorded audio and don't need VAD, REST is
the simpler path. Move to WebSocket when you need live audio,
turn-taking, or in-stream `flush`.
## What both transports share
* **Models**: same `model_name` works on both.
* **Input formats**: PCM (multiple sample rates), WAV, Opus, mu-law, A-law.
* **Tunable options**: `temp`, `language`, `padding_bonus`, `delay_in_frames`, and a `keywords` dictionary for [boosting names and jargon](/guides/transcription-settings#keyword-boosting), all via `json_config`. See [Transcription Settings](/guides/transcription-settings).
## What's transport-specific
* **WebSocket-only**: semantic VAD `step` messages every 80 ms, in-stream `send_flush()` for forced processing, adaptive delay controls (`delay_in_frames`), setup-message stream controls (`send_setup_on_start`, `wait_for_ready_on_start`), browser tokens. See [WebSocket Lifecycle](/guides/websocket-lifecycle).
* **REST-only**: send the full audio as the request body; receive NDJSON over a streaming response.
## Next steps
Streaming audio in, transcripts and VAD signals out, with flush
control.
One-shot transcription of a complete audio file.
Use semantic VAD and adaptive delay to decide when an agent should answer.
Recognize names, brands, and jargon by biasing STT toward a custom dictionary.
Browse complete voice-agent demos built with Gradium STT and TTS.
`language`, `temp`, `padding_bonus`, `delay_in_frames`.
Error contracts across REST, WebSocket, and streamed responses.
# Speech-to-Text (REST)
Source: https://docs.gradium.ai/guides/speech-to-text-rest
One-shot transcription of complete audio files via HTTP POST
Use the REST endpoint when you have a complete audio file in hand and
want a single HTTP request, no WebSocket to manage. The server reads
the full body, transcribes it, and streams newline-delimited JSON
(NDJSON) messages back as the body.
For live audio (microphone, telephony) or to react to VAD and flush
events in real time, use the SDK streaming guide instead.
## Quickstart
```python Python theme={null}
import json
import requests
with open("input.wav", "rb") as f:
audio = f.read()
with requests.post(
"https://api.gradium.ai/api/post/speech/asr",
params={"json_config": json.dumps({"language": "en"})},
data=audio,
headers={
"x-api-key": "your_api_key",
"Content-Type": "audio/wav",
},
stream=True,
) as resp:
resp.raise_for_status()
transcript = []
for line in resp.iter_lines(decode_unicode=True):
if not line:
continue
msg = json.loads(line)
if msg["type"] == "text":
transcript.append(msg["text"])
print(" ".join(transcript))
```
```bash cURL theme={null}
curl -L -X POST https://api.gradium.ai/api/post/speech/asr \
--url-query 'json_config={"language":"en"}' \
-H "x-api-key: your_api_key" \
-H "Content-Type: audio/wav" \
--data-binary @input.wav
```
## Response format
The response is `Content-Type: application/x-ndjson`. Each line is a
JSON object with a `type` field, `text` and `end_text` are the ones
you'll typically care about; `error` may appear if the pipeline fails
mid-stream. The body closes when transcription is complete.
For the full message schema, request body, query parameters, and error
shapes, see the [STT POST Endpoint reference](/api-reference/endpoint/stt-post).
## Passing `json_config`
Advanced options (`temp`, `language`, `padding_bonus`, `delay_in_frames`,
and a `keywords` dictionary for [boosting names and jargon](/guides/transcription-settings#keyword-boosting))
are passed as a JSON-encoded string in the `json_config` query
parameter. The example below sets the `language` and changes the
adaptive delay:
```bash cURL theme={null}
curl -L -X POST https://api.gradium.ai/api/post/speech/asr \
--url-query 'json_config={"language":"en","delay_in_frames":16}' \
-H "x-api-key: your_api_key" \
-H "Content-Type: audio/wav" \
--data-binary @input.wav
```
```python Python theme={null}
import json
import requests
with open("input.wav", "rb") as f:
audio = f.read()
config = {"language": "en", "delay_in_frames": 16}
with requests.post(
"https://api.gradium.ai/api/post/speech/asr",
params={"json_config": json.dumps(config)}, # requests URL-encodes for you
data=audio,
headers={
"x-api-key": "your_api_key",
"Content-Type": "audio/wav",
},
stream=True,
) as resp:
resp.raise_for_status()
for line in resp.iter_lines(decode_unicode=True):
if line:
print(json.loads(line))
```
The supported keys, allowed values, and validation rules are documented
in [Transcription Settings](/guides/transcription-settings).
## Next steps
Full request/response schema, query parameters, content types.
Low-latency, microphone-friendly transcription with VAD and flush.
# Text Rewriting Rules
Source: https://docs.gradium.ai/guides/text-rewriting
Normalize and expand text patterns for better TTS pronunciation
The text-to-speech API supports text rewriting rules that normalize and expand certain patterns in the input text before synthesis. These rules help the TTS model properly pronounce numbers, currency amounts, email addresses, URLs, and alphanumeric codes.
## Configuration
Rewrite rules are controlled by the `rewrite_rules` field of `json_config`. Its value is a single string: either a language or locale code (e.g. `"en"`), or `"none"` to disable rewriting.
`json_config` goes in the setup message for WebSocket sessions, or in the request body for the REST endpoint:
```python Python (streaming) theme={null}
import gradium
client = gradium.client.GradiumClient(api_key="your-api-key")
async with client.tts_realtime(
voice_id="Bla6SbVMczYnOhfK",
output_format="pcm",
json_config={"rewrite_rules": "en"},
) as tts:
await tts.send_text("The invoice came to $1500.")
await tts.send_eos()
...
```
```python Python (one-shot) theme={null}
import gradium
client = gradium.client.GradiumClient(api_key="your-api-key")
result = await client.tts(
setup={
"voice_id": "Bla6SbVMczYnOhfK",
"output_format": "wav",
"json_config": {"rewrite_rules": "en"},
},
text="The total is $1500, invoice sent to foo.bar@gmail.com.",
)
with open("output.wav", "wb") as f:
f.write(result.raw_data)
```
```js JavaScript (WebSocket) theme={null}
// Authentication uses a short-lived token; see the Browser WebSockets guide.
const url = new URL("wss://api.gradium.ai/api/speech/tts");
url.searchParams.set("token", token);
const ws = new WebSocket(url);
ws.addEventListener("open", () => {
ws.send(JSON.stringify({
type: "setup",
model_name: "default",
voice_id: "Bla6SbVMczYnOhfK",
output_format: "pcm",
json_config: {rewrite_rules: "en"},
}));
});
```
```bash cURL (REST) theme={null}
curl -L -X POST https://api.gradium.ai/api/post/speech/tts \
-H "x-api-key: your_api_key" \
-H "Content-Type: application/json" \
-d '{
"text": "Write to foo.bar@gmail.com about the 1500 euros.",
"voice_id": "Bla6SbVMczYnOhfK",
"output_format": "wav",
"only_audio": true,
"json_config": {"rewrite_rules": "en"}
}' > output.wav
```
### Default behavior
For both the production and beta models, when the selected voice has a language associated with it, the API enables that language's rules by default. So a session using an English voice behaves as if you had passed `"rewrite_rules": "en"`. Setting `rewrite_rules` yourself overrides this default.
### Disabling rewriting: `none`
Pass `"rewrite_rules": "none"` to turn all rewrite rules off. Because of the language default described above, this is the only way to get unrewritten text for a voice that has a language: omitting the field enables the voice language's rules, while `"none"` disables them.
```json theme={null}
{
"json_config": {
"rewrite_rules": "none"
}
}
```
With rules disabled, only minimal text normalization is applied before synthesis.
## Supported Languages and Locales
Each value enables the full set of rewriting rules for that language or locale:
| Value | Language |
| - | - |
| `en` | English |
| `fr` | French |
| `fr-be` | French (Belgium) |
| `fr-ch` | French (Switzerland) |
| `de` | German |
| `es` | Spanish |
| `pt` | Portuguese |
| `none` | Rewriting disabled |
Belgian and Swiss French share most of their rewriting with metropolitan French; the
differences are in how dates, numbers and currency amounts are spoken, so pick `fr-be` or
`fr-ch` when your audience is in those markets.
## What Gets Rewritten
With a language or locale selected, the following are expanded into the spoken form of that language:
* Numbers, including negative values and large magnitudes
* Currency amounts, with the symbol before or after the amount
* Email addresses
* URLs
* Alphanumeric codes and identifiers, such as reference or product codes
* Dashed digit sequences, such as phone and account numbers
Years and small numbers are kept as written, and punctuation around a rewritten pattern is preserved.
## Custom Rewrites: Pronunciation Dictionaries
Rewrite rules handle general patterns (numbers, currency amounts, emails, URLs). To rewrite specific words or phrases of your own (brand names, acronyms, domain jargon), use a [pronunciation dictionary](/guides/text-to-speech#pronunciation-dictionaries) instead.
The dictionary is not part of `rewrite_rules` or `json_config`: create it in the Gradium Studio (or via the [pronunciations API](/api-reference/endpoint/list-pronunciations)) and pass its ID as `pronunciation_id` at the top level of the setup message, next to `voice_id`:
```python theme={null}
result = await client.tts(
setup={
"voice_id": "Bla6SbVMczYnOhfK",
"output_format": "wav",
"pronunciation_id": "bb1ckYhNHCcIJjdK", # top level, not in json_config
"json_config": {"rewrite_rules": "en"}, # pattern rules still apply
},
text="The text you want to generate.",
)
```
Dictionary entries are applied to the text first, then the pattern rules run on the result. The two combine freely. For example, a dictionary that expands your product's acronym, plus `"en"` rules to read out prices and product codes. Pronunciation dictionaries apply to WebSocket sessions only; REST requests cannot use them, and only the pattern-based `rewrite_rules` are available there.
## Best Practices
1. **Match the value to the voice** - use the language or locale of the voice you synthesise with, and rely on the default when they already match
2. **Use the locale codes** - `fr-be` and `fr-ch` read dates, numbers and amounts the way those markets expect
3. **Disable deliberately** - pass `"none"` when your text is already written the way it should be spoken
# Text-to-Speech (WebSocket)
Source: https://docs.gradium.ai/guides/text-to-speech
Convert text to natural-sounding speech via the Gradium SDK and WebSocket
The Gradium Text-to-Speech model produces high-quality, natural speech
with low first-byte latency. This guide covers synthesis via the
Python SDK, including streaming output, custom voices, in-text control
tags, and pronunciation dictionaries.
For simple "text in, file out" use the REST POST endpoint, a single
HTTP call with no WebSocket to manage.
## Quickstart
Open a real-time TTS stream, send text, and read audio chunks as soon
as the model produces them.
```python theme={null}
import asyncio
import gradium
async def main():
client = gradium.client.GradiumClient(api_key="your-api-key")
audio_chunks = []
async with client.tts_realtime(
voice_id="YTpq7expH9539ERJ",
output_format="pcm",
) as tts:
async def sender():
await tts.send_text("Hello, world.")
await tts.send_eos()
async def receiver():
async for msg in tts:
if msg["type"] == "audio":
audio_chunks.append(msg["audio"])
elif msg["type"] == "end_of_stream":
return
await asyncio.gather(sender(), receiver())
with open("output.pcm", "wb") as f:
f.write(b"".join(audio_chunks))
asyncio.run(main())
```
For a lower-level protocol view, see
[WebSocket Lifecycle](/guides/websocket-lifecycle). For LLM-generated
text, see [LLM Tokens to Streaming TTS](/guides/recipes/llm-to-tts).
## Choosing the SDK API
The Python SDK exposes three TTS shapes. All WebSocket APIs talk to
`wss://api.gradium.ai/api/speech/tts`.
| Use case | Method |
| - | - |
| You are producing text live and want to send/receive concurrently | `tts_realtime` |
| You have a string, list, or async generator and want streamed audio chunks | `tts_stream` |
| You want the complete audio bytes after generation finishes | `tts` |
Use `tts_realtime` for voice agents, LLM token streaming, browser
bridges, telephony bridges, or any workflow where the sending and
receiving loops run at the same time. Use `tts_stream` when your input
is shaped like an iterable and you only need to consume bytes. Use
`tts` for scripts and tests where waiting for the full result is fine.
## Buffered convenience
```python theme={null}
import gradium
client = gradium.client.GradiumClient()
result = await client.tts(
setup={
"model_name": "default",
"voice_id": "YTpq7expH9539ERJ",
"output_format": "wav"
},
text="Hello, world!"
)
with open("output.wav", "wb") as f:
f.write(result.raw_data)
print(f"Sample rate: {result.sample_rate}")
print(f"Request ID: {result.request_id}")
```
`client.tts(...)` still uses the streaming protocol under the hood; it
buffers the audio chunks and returns a single `TTSResult`.
## Model selection
Set `model_name` at the top level of the TTS setup, alongside `voice_id`,
outside `json_config`.
The new TTS model is selected by `"default"`. You can omit
`model_name` to use it. `"gradium-tts-beta"` also routes to this model until the
next TTS beta arrives. Keep that alias to follow future beta models, or switch
to `"default"` to stay on stable releases.
In the Python SDK, pass `model_name` to `tts_realtime`, or inside `setup`
for `tts` and `tts_stream`. With a direct WebSocket, include it in the first
`setup` message. With [REST POST](/guides/text-to-speech-rest), include it
at the top level of the JSON request body.
## Setup Parameters
* **`model_name`**: The TTS model to use (default: `"default"`). See
[Model selection](#model-selection).
* **`voice_id`**: The voice id of the voice to be used. The voice id can be found in the voice library section of this documentation or in the studio.
* **`output_format`**: Audio format of the output data (supported: `"pcm"`,
`"wav"`, `"opus"`, ...)
* **`json_config`**: Advanced voice settings such as temperature,
speed and rewrite rules. See
[Voice Settings](/guides/voice-settings).
* **`pronunciation_id`**: Optional pronunciation dictionary to apply to
this request.
* **`client_req_id`** and **`close_ws_on_eos`**: Used for reusable and
multiplexed WebSocket connections. See
[Multiplexing](/guides/multiplexing).
When using `"pcm"` output format, the audio will adhere to the following
specifications:
* **Sample Rate**: 48000 Hz (48kHz)
* **Format**: PCM (Pulse Code Modulation)
* **Bit Depth**: 16-bit signed integer
* **Channels**: Single channel (mono)
* **Chunk Size**: 3840 samples per chunk (80ms at 48kHz)
When using the `"wav"` output format, the audio chunks are in WAV format,
at 48kHz, 16-bit signed integer mono.
When using the `"opus"` output format, the audio chunks use the Opus codec
wrapped in an Ogg container.
Alternative output formats include `"ulaw_8000"`, `"alaw_8000"`, `"pcm_8000"`,
`"pcm_16000"`, and `"pcm_24000"`.
## Streaming TTS
Use `tts_stream` when you have a string, list, or async generator and
want an async iterator of audio bytes.
```python theme={null}
stream = await client.tts_stream(
setup={
"model_name": "default",
"voice_id": "LFZvm12tW_z0xfGo",
"output_format": "pcm"
},
text="This is a longer text that will be streamed."
)
async for audio_chunk in stream.iter_bytes():
print(f"Received {len(audio_chunk)} bytes")
```
## Using Custom Voices
```python theme={null}
result = await client.tts(
setup={
"model_name": "default",
"voice_id": "YTpq7expH9539ERJ",
"output_format": "wav"
},
text="Hello with my custom voice!"
)
```
## Output Formats
```python theme={null}
# WAV format
result = await client.tts(setup={"voice_id": "YTpq7expH9539ERJ", "output_format": "wav"}, text="Hello")
# PCM format: the data is sampled at 48kHz, 16-bit signed integer, mono
result = await client.tts(setup={"voice_id": "YTpq7expH9539ERJ", "output_format": "pcm"}, text="Hello")
# Get numpy array from PCM
pcm_array = result.pcm()
pcm16_array = result.pcm16()
```
## Flushing and Pauses
The model only generates audio when it has enough context to do so, so generally
the audio lags a few words behind the text input. The `` tag can be
used to force the model to output the audio for all the text that has been
input so far.
```python theme={null}
sample_text = "Hello, this is a test from the Gradium Text to Speech system. We are testing the flush."
test_audio = await client.tts(
setup={'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav'},
text=sample_text,
)
```
Pauses can be generated by inserting a "break time" tag as shown below.
The break time is specified in seconds and should be between 0.1 and 2.0s.
The tag must be preceded and followed by a space.
```python theme={null}
sample_text = 'Hello, this is a test from the Gradium Text to Speech system.
**Split text chunks on whitespace, never in the middle of a word or
immediately before a punctuation mark.** When text is sent incrementally
(whether through successive `send_text` calls on the Python realtime
stream, through an async generator as shown above, or through multiple
`{"type": "text"}` messages over the WebSocket API), the server inserts
a single whitespace between the contents of consecutive messages.
So sending `"foo"` followed by `"bar"` is equivalent to sending
`"foo bar"` (a whitespace is added between them), **not** `"foobar"`.
Splitting a single word across two messages will change its pronunciation.
For the same reason, do not split punctuation into its own message:
sending `"foo"` followed by `"."` produces `"foo ."` rather than `"foo."`.
Keep trailing punctuation attached to the preceding word (e.g. send
`"foo."` as one message, or `"foo. "` followed by the next chunk).
## Pronunciation Dictionaries
Pronunciation dictionaries allow you to customize how specific words or phrases are pronounced in your TTS output. This is particularly useful for:
* Brand names, technical terms, or proper nouns
* Acronyms that should be pronounced in a specific way
* Words with non-standard pronunciations in your use case
The easiest way to create and manage pronunciation dictionaries is through the Gradium Studio, on the pronunciation page.
Once you have created a dictionary and obtained its ID, you can use it in your TTS requests by passing the `pronunciation_id` parameter in the setup message, similar to the way we pass the voice\_id:
```python theme={null}
import gradium
client = gradium.client.GradiumClient()
result = await client.tts(
setup={
"voice_id": "YTpq7expH9539ERJ",
"output_format": "wav",
"pronunciation_id": "bb1ckYhNHCcIJjdK", # Whatever your ID is.
},
text="The text you want to generate."
)
with open("output.wav", "wb") as f:
f.write(result.raw_data)
```
## Direct WebSocket
If you don't want the gradium SDK, for example you're calling from a
non-Python runtime, or you want full control over the wire, you can
talk to the WebSocket protocol directly. The message shapes are
identical to what the SDK sends.
For interactive poking from a terminal, [`wscat`](https://github.com/websockets/wscat)
is the quickest way to confirm reachability and step through messages
by hand:
```bash theme={null}
wscat -c "wss://api.gradium.ai/api/speech/tts" \
-H "x-api-key: your_api_key"
# After connection, type:
# {"type":"setup","voice_id":"YTpq7expH9539ERJ","model_name":"default","output_format":"wav"}
# {"type":"text","text":"Hello, world!"}
# {"type":"end_of_stream"}
```
For a client without the SDK, use any WebSocket library
(here Python's [`websockets`](https://websockets.readthedocs.io/)):
```python theme={null}
import asyncio
import base64
import json
import websockets
async def synthesise(api_key: str, voice_id: str, text: str) -> bytes:
setup = {
"type": "setup",
"voice_id": voice_id,
"model_name": "default",
"output_format": "wav",
}
audio_chunks = []
async with websockets.connect(
"wss://api.gradium.ai/api/speech/tts",
additional_headers={"x-api-key": api_key},
) as ws:
await ws.send(json.dumps(setup))
ready = json.loads(await ws.recv())
assert ready["type"] == "ready"
await ws.send(json.dumps({"type": "text", "text": text}))
await ws.send(json.dumps({"type": "end_of_stream"}))
while True:
msg = json.loads(await ws.recv())
if msg["type"] == "audio":
audio_chunks.append(base64.b64decode(msg["audio"]))
elif msg["type"] == "end_of_stream":
break
elif msg["type"] == "error":
raise RuntimeError(msg["message"])
return b"".join(audio_chunks)
audio = asyncio.run(synthesise("your_api_key", "YTpq7expH9539ERJ", "Hello, world!"))
with open("output.wav", "wb") as f:
f.write(audio)
```
For the full message schema (every field, every error code), see the
[TTS WebSocket reference](/api-reference/endpoint/tts-websocket).
## Next steps
Complete wire-level schema: every message type, every field, every
error code.
Synthesise a complete text block in a single HTTP request.
Run multiple concurrent synthesis requests over a single WebSocket
session.
Speed, temperature and rewrite rules.
# Text-to-Speech Overview
Source: https://docs.gradium.ai/guides/text-to-speech-overview
When to use the WebSocket API vs the REST endpoint, and how the pieces fit together
Gradium exposes Text-to-Speech over two transports. They share the same
models and voices; you pick the transport that matches your input shape
and latency needs.
## WebSocket vs REST
| If your text is... | Use | Why |
| - | - | - |
| Streamed in token-by-token (LLM output, agent loop, live UI) | [WebSocket](/guides/text-to-speech) | Lowest first-byte latency. Send text as it arrives, audio chunks come back as soon as they're generated. |
| A finished block in hand (article, scripted line, batch job) | [REST](/guides/text-to-speech-rest) | One HTTP call, no connection to manage. Save the response body to a file and you're done. |
| Many concurrent synthesis requests | [WebSocket + Multiplexing](/guides/multiplexing) | Run several requests over a single socket, matched by `client_req_id`. |
| Browser or mobile app audio | [Browser WebSockets](/guides/browser-websockets) | Use short-lived tokens instead of exposing an API key. |
If you're unsure, start with REST. It's a single POST and the response
body is the audio. Move to WebSocket once latency or streaming-input
concerns make REST the bottleneck.
## What both transports share
* **Voices**: any `voice_id` from the [Voice Library](/guides/voices/overview) works on both. [Voice Design](/guides/voices/voice-design) candidates are the exception: REST only, until converted.
* **Output formats**: PCM, WAV, Opus, and the telephony codecs (`ulaw_8000`, `alaw_8000`).
* **Voice settings**: `temp`, `padding_bonus` and `rewrite_rules`, all passed inside `json_config`. See [Voice Settings](/guides/voice-settings).
## What's transport-specific
* **`` and `
SDK and direct WebSocket usage, streaming output, flush, timestamps.
One-shot synthesis with a single HTTP POST.
Stream generated text into TTS while preserving natural prosody.
Speed, temperature and rewrite rules.
Browse flagship voices or create your own clones.
# Text-to-Speech (REST)
Source: https://docs.gradium.ai/guides/text-to-speech-rest
One-shot synthesis of a complete text block via HTTP POST
Use the REST endpoint when you have a finished text block and want a
single HTTP request, no WebSocket to manage. The response can either
be raw audio bytes (for direct file writes) or a JSON stream that
mirrors the WebSocket protocol.
For low-latency synthesis of long or generated text, streaming TTS
via the SDK gives you audio chunks as they're produced.
## Quickstart
`only_audio=true` returns the raw audio bytes directly, easiest for
"text in, file out".
```bash cURL theme={null}
curl -L -X POST https://api.gradium.ai/api/post/speech/tts \
-H "x-api-key: your_api_key" \
-H "Content-Type: application/json" \
-d '{"text": "Hello, world!", "voice_id": "YTpq7expH9539ERJ", "output_format": "wav", "only_audio": true}' \
> output.wav
```
```python Python theme={null}
import requests
resp = requests.post(
"https://api.gradium.ai/api/post/speech/tts",
json={
"text": "Hello, world!",
"voice_id": "YTpq7expH9539ERJ",
"output_format": "wav",
"only_audio": True,
},
headers={"x-api-key": "your_api_key"},
)
resp.raise_for_status()
with open("output.wav", "wb") as f:
f.write(resp.content)
```
To select a model, add `model_name` at the top level of the JSON body.
Omitting it uses `default`, which selects the latest production TTS model. See
[Model selection](/guides/text-to-speech#model-selection).
## Response modes
The `only_audio` field in the request body picks one of two response
shapes:
* **`only_audio: true`**: the response body is the raw audio in the
requested `output_format` (WAV, PCM, Opus, …). Save it directly to a
file or pipe it to a player. The `Content-Type` reflects the format
(`audio/wav`, `audio/ogg`, `audio/pcm`).
* **`only_audio: false`** (or omitted): the response is a JSON stream
using the same message format as the WebSocket endpoint, including
`audio` (base64), `text` (with timestamps), and `error`. Read the
body line-by-line until it closes.
### Streaming the JSON response
Pass `only_audio: false` and read the body as it arrives. With cURL,
the `-N` (`--no-buffer`) flag prints each line as soon as the server
sends it.
```bash cURL theme={null}
curl -N -L -X POST https://api.gradium.ai/api/post/speech/tts \
-H "x-api-key: your_api_key" \
-H "Content-Type: application/json" \
-d '{"text": "Hello, world!", "voice_id": "YTpq7expH9539ERJ", "output_format": "wav", "only_audio": false}'
```
```python Python theme={null}
import base64
import json
import requests
audio_chunks = []
text_segments = []
with requests.post(
"https://api.gradium.ai/api/post/speech/tts",
json={
"text": "Hello, world!",
"voice_id": "YTpq7expH9539ERJ",
"output_format": "wav",
"only_audio": False,
},
headers={"x-api-key": "your_api_key"},
stream=True,
) as resp:
resp.raise_for_status()
for line in resp.iter_lines(decode_unicode=True):
if not line:
continue
msg = json.loads(line)
if msg["type"] == "audio":
audio_chunks.append(base64.b64decode(msg["audio"]))
elif msg["type"] == "text":
text_segments.append(msg)
with open("output.wav", "wb") as f:
f.write(b"".join(audio_chunks))
```
For the full request schema, supported output formats, and error
shapes, see the [TTS POST Endpoint reference](/api-reference/endpoint/tts-post).
## Next steps
Full request body schema, output formats, error contracts.
Chunked audio output, custom voices, flush, timestamps.
# Transcription Settings
Source: https://docs.gradium.ai/guides/transcription-settings
Fine-tune STT: language, temperature, latency vs. quality tradeoffs
STT models accept additional configuration via the `json_config`
parameter. In the Python SDK, this is a dict mapping option name to a
JSON-serializable value. When using the REST endpoint, pass it as a
URL-encoded JSON string in the `json_config` query parameter.
These options apply to both the [WebSocket](/guides/speech-to-text)
and [REST](/guides/speech-to-text-rest) transports. For TTS settings,
see [Voice Settings](/guides/voice-settings).
## Quick reference
| Option | Type | Allowed values | Effect |
| - | - | - | - |
| `temp` | float | `0.0`-`1.5` | Sampling temperature for text generation. `0.0` is greedy; higher values produce more diverse output and can help when no text is being recognised. |
| `language` | string | `"en"`, `"fr"`, `"de"`, `"es"`, `"pt"`, `"any"` | Audio language for transcription models, or output language for translating models. Always set it; use `"any"` when the language is not known. See [Languages](#languages). |
| `target_language` | string | `"en"`, `"fr"`, `"de"`, `"es"`, `"pt"` | Output language for translating models; ignored otherwise. See [Languages](#languages). |
| `padding_bonus` | float | `-4.0`–`4.0` | Biases the model toward emitting text sooner (negative) or later (positive). Out-of-range values raise a server error. |
| `delay_in_frames` | int | `0`-`80` | Adaptive delay control. Each frame is 80 ms of context before text is emitted. Defaults to `10` when unset. Higher values improve quality at the cost of latency; lower values are more reactive. Values outside this range raise a server error. The legacy alias `delay_in_tokens` is also accepted. |
| `keywords` | object | `{ "words": string[], "boost": -6..6 }` | Bias transcription toward a custom dictionary of terms. `boost` is applied in log-space, so its effect is exponential; `3` is the recommended default. See [Keyword boosting](#keyword-boosting). |
Default values may evolve with new model releases. Pin the options
explicitly if you depend on a specific behaviour. The server validates
known keys with the constraints above; unknown keys are silently
ignored, so double-check spelling against the table.
## Languages
`json_config` with `language` set is required. Requests without it are
rejected. Pass the language of the audio when you know it, or `"any"` when you
do not. Requests that already set a language or `"any"` are unaffected.
For non-translating transcription models, `language` is the expected language
of the audio. It grounds the model to a single language for better transcription
quality, so set it whenever you know it. For audio that mixes languages, set the
dominant one. When no single language predominates, or the language is not known
ahead of time, pass `"any"` and the model detects it from the audio.
`target_language` has no effect on these models.
For translating transcription models, `language` and `target_language` specify
the language the transcription should be generated in, not the language of the
audio. The two are interchangeable; set either one.
## Keyword boosting
Adapt transcription to your own vocabulary: player names, brands,
products, and channel names. Pass a `keywords` dictionary and
the decoder gives those terms priority, so the vocabulary that matters
to your product is transcribed exactly as you expect, in real time and
without retraining.
```python theme={null}
config = {
"language": "en",
"keywords": {
"words": ["Mbappé", "Haaland", "Vinícius", "Bellingham"],
"boost": 3,
},
}
```
* **`words`**: the terms to boost. Matching is token-by-token, so
keywords must not contain spaces. Split multi-word names, so
`"Ferran Torres"` becomes `"Ferran"` and `"Torres"`. Matching is also
case and accent sensitive, so include the variants you expect
(`"Mbappé"` and `"Mbappe"`). You can boost up to **500 keywords**, and
every variant of a word (lowercase, uppercase, accented, etc.) counts
toward that limit.
* **`boost`**: how strongly to bias toward the dictionary, from `-6`
to `6`. The value is applied in log-probability space, so its effect
is *exponential*: small values already shift decoding noticeably.
**Choosing a boost.** `3` is the recommended default and works for most
cases; raise it to `4` for unusually rare or foreign vocabulary. Higher
values give diminishing returns and can make the decoder *loop*,
repeating a boosted word instead of following the audio, so avoid going
much beyond `4` to `5`.
**Negative values** de-boost a term. Use them only to *remove* a
specific word you never want in the output, not as a general accuracy
control.
Boosting helps most for rare, domain-specific vocabulary and in
low-latency (small `delay_in_frames`) streaming, where the model has
less context to disambiguate names on its own. See the
[Keyword boosting recipe](/guides/recipes/keyword-boosting) for an
end-to-end example.
## Passing `json_config`
The same `json_config` payload is sent regardless of which SDK API you
use; only the call shape differs:
```python theme={null}
config = {"language": "en", "temp": 0.3, "delay_in_frames": 16}
# Pull-based: setup is a dict containing json_config.
stream = await client.stt_stream(
{"model_name": "default", "input_format": "pcm", "json_config": config},
audio_generator(audio_data),
)
# Real-time: setup is keyword arguments; json_config stays nested.
async with client.stt_realtime(
model_name="default",
input_format="pcm",
json_config=config,
) as stt:
...
```
When calling the REST endpoints directly, pass `json_config` as a
URL-encoded JSON string in the query parameters, see the
[REST guide](/guides/speech-to-text-rest#passing-json_config).
## Semantic VAD and delay
`delay_in_frames` is most useful with the WebSocket STT stream. The
server emits semantic VAD `step` messages every 80 ms; each step
contains future horizons with `inactivity_prob` values. A common
starting point for voice agents is to watch the longest horizon and
flush when the inactivity probability stays above your threshold.
```python theme={null}
if msg["type"] == "step":
turn_done = msg["vad"][-1]["inactivity_prob"] > 0.5
if turn_done:
await stt.send_flush(flush_id=1)
```
Use lower `delay_in_frames` values for fast back-and-forth assistants,
and higher values for transcription quality when a little more latency
is acceptable. A value such as `16` is a balanced starting point; try
`8` when responsiveness matters more than context.
# Voice Settings
Source: https://docs.gradium.ai/guides/voice-settings
Fine-tune TTS output: speed, temperature, rewrite rules
TTS models accept advanced options via the `json_config` parameter. In
the Python SDK, this is a dict mapping option name to value (float or
string). For REST POST requests, pass it as an object in the JSON body.
These options apply to both the [WebSocket](/guides/text-to-speech) and
[REST](/guides/text-to-speech-rest) transports. For STT, see
[Transcription Settings](/guides/transcription-settings).
**Deprecated: `cfg_coef` for TTS**
The new TTS model accepts `cfg_coef` in `json_config` but ignores its value. Remove it
from TTS requests. Voice Design's `cfg_scale` is a separate generation setting
and remains available.
## Quick reference
**Range** is the span each option is tuned and tested across.
| Parameter | Range | Default | Effect |
| - | - | - | - |
| `temp` | `0.0`–`1.4` | `0.7` | Sampling temperature. `0.0` is deterministic; higher values produce more diverse output. |
| `padding_bonus` | `-4.0`–`4.0` | `0.0` | Speech speed. Negative values are faster, positive values are slower. |
| `rewrite_rules` | string | voice language | Text-rewriting rules applied before synthesis. Defaults to the voice's language; pass `"none"` to disable. See [Text Rewriting Rules](/guides/text-rewriting). |
Validation accepts a wider span than the table: `temp` to `1.5`,
`padding_bonus` from `-5.0` to `5.0`. Use the table's ranges when tuning.
For deterministic output, set `temp` to `0.0`. For multi-utterance
flows on a single session, see [Multiplexing](/guides/multiplexing).
For designed voices, see also
[Voice Design settings](/guides/voices/voice-design#synthesis-settings).
The TTS engine recognises the `` and ` not found.`
Pronunciation dictionaries apply to WebSocket sessions only. REST
requests cannot use them; the pattern-based `rewrite_rules` in
`json_config` are the only rewriting control there. See [Text
Rewriting Rules](/guides/text-rewriting).
```python theme={null}
# pronunciation_id sits beside voice_id; tuning options go in json_config.
audio = await client.tts(
setup={
"voice_id": "YTpq7expH9539ERJ",
"output_format": "wav",
"pronunciation_id": "bb1ckYhNHCcIJjdK",
"json_config": {"temp": 0.3},
},
text="Our SDK ships today.",
)
```
## Speed control
You can guide the speed of the model using the padding bonus
parameter. Default value is 0.0. Negative values mean that the speaker
will speak faster (values between -4.0 and -0.1). Positive values mean
that the speaker will speak slower (values between 0.1 and 4.0).
```python theme={null}
sample_text = "Hello, this is a test from the Gradium Text to Speech system. We are testing the speed."
slower_audio = await client.tts(
setup={'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav', 'json_config':{'padding_bonus':2.0}},
text=sample_text,
)
faster_audio = await client.tts(
setup={'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav', 'json_config':{'padding_bonus':-2.0}},
text=sample_text,
)
```
## Temperature control
The temperature for the generation can be set with values ranging from
0 to 1.4. A value of 0 corresponds to a deterministic generation, while
higher values lead to more diverse outputs. Default value is 0.7.
```python theme={null}
setup = {'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav', 'json_config':{'temp':0.3}}
audio = await client.tts(text=sample_text, setup=setup)
```
## Rewrite rules
The `rewrite_rules` parameter can be used to pass text rewriting rules
that are applied before the text is synthesized. The rules should be
passed as a string. More details on the rules themselves can be found
in the [Text Rewriting Rules](/guides/text-rewriting) guide. Values
such as `"en"`, `"fr"`, `"fr-be"`, `"fr-ch"`, `"de"`, `"es"`, `"pt"`
enable all the rewriting rules for a given language or locale. When the voice has a language, its
rules are enabled by default; pass `"none"` to disable rewriting entirely.
## Passing `json_config`
```python theme={null}
config = {"temp": 0.3, "padding_bonus": -1.0}
# Streaming over WebSocket: setup is keyword args; json_config stays nested.
async with client.tts_realtime(
voice_id="YTpq7expH9539ERJ",
output_format="pcm",
json_config=config,
) as stream:
...
# One-shot via SDK: setup is a dict containing json_config.
audio = await client.tts(
setup={"voice_id": "YTpq7expH9539ERJ", "output_format": "wav", "json_config": config},
text="Hello",
)
```
For REST, see the [TTS POST reference](/api-reference/endpoint/tts-post).
# Custom Voices
Source: https://docs.gradium.ai/guides/voices/custom-voices
Clone your own voice using a short audio sample
Create your own custom voice clones from audio samples. Custom voices are passed to TTS using the `voice_id` parameter (not `voice`).
## Create a Custom Voice
Upload an audio file to create a new voice clone. A minimum of 10 seconds of clear audio is recommended for best results.
```python theme={null}
import json
import gradium
client = gradium.client.GradiumClient()
voice = await gradium.voices.create(
client,
audio_file="my_voice_sample.wav",
name="My Custom Voice",
description="A voice created from my recording",
language="en",
start_s=0.0,
)
print(json.dumps(voice, indent=2))
```
### Parameters
| Parameter | Type | Required | Description |
| :- | :- | :-: | :- |
| `audio_file` | `string` | Yes | Path to the audio file (WAV, MP3, etc.) |
| `name` | `string` | Yes | A name for the voice |
| `description` | `string` | No | A description of the voice |
| `language` | `string` | Yes | Language spoken in the audio sample. One of `"en"`, `"fr"`, `"de"`, `"es"`, `"pt"`. Also what [Voice Localization](/guides/voices/voice-localization) uses as the source language, so it must match the recording. |
| `start_s` | `float` | No | Start time in seconds to begin sampling from the audio file (default `0.0`) |
| `timeout_s` | `float` | No | Maximum seconds to wait for embedding extraction (default `10.0`) |
| `input_format` | `string` | No | Audio format. If omitted, inferred from the file extension |
# Flagship Voices
Source: https://docs.gradium.ai/guides/voices/flagship-voices
Our top curated voices across multiple languages
Meet our flagship voices: the highest quality, most versatile picks in the Gradium library. Each is tuned for natural speech across any use case, spanning multiple languages with local region and accent coverage.
# Manage Voices
Source: https://docs.gradium.ai/guides/voices/manage-voices
List, update, and delete your custom voices
Manage your custom voice clones using the Gradium SDK.
## List All Custom Voices
Retrieve all custom voices for your organization.
```python theme={null}
import json
import gradium
client = gradium.client.GradiumClient()
all_custom_voices = await gradium.voices.get(client)
print(json.dumps(all_custom_voices, indent=2))
```
## Get a Specific Voice
Retrieve details for a single voice by its UID.
```python theme={null}
import json
import gradium
client = gradium.client.GradiumClient()
voice = await gradium.voices.get(client, voice_uid="abc123def456")
print(json.dumps(voice, indent=2))
```
## Update a Voice
Update the metadata of an existing voice. The `PUT /voices/{uid}` endpoint
accepts `name`, `description`, `language`, `start_s`, `tags`, and `rank`. The
SDK helper exposes a subset; for `language`, `tags`, or `rank` call the REST
endpoint directly.
[Voice Localization](/guides/voices/voice-localization) needs the source
voice's `language`. Clones created without one return `409` there; set it
here once, to the language actually spoken in the recording, and retry.
```python theme={null}
import gradium
client = gradium.client.GradiumClient()
await gradium.voices.update(
client,
voice_uid="abc123def456",
name="Updated Voice Name",
description="Updated description",
start_s=1.5
)
```
## Delete a Voice
Permanently delete a custom voice.
```python theme={null}
import gradium
client = gradium.client.GradiumClient()
await gradium.voices.delete(client, voice_uid="abc123def456")
```
# Overview
Source: https://docs.gradium.ai/guides/voices/overview
Explore the Gradium voice library
Gradium provides a curated library of flagship voices across five languages, plus the ability to create your own: clone a voice from a recording, design one from a written description, or localize any of them into another language or accent.
70+ flagship voices across five languages, hand-picked for quality and range.
Clone your own voice using a short audio sample.
Describe a voice in words and keep the candidate you like.
Clean up an existing voice without changing its language or speaker.
Make an existing voice speak another language or accent.
List, update, and delete your custom voices.
# Voice Design
Source: https://docs.gradium.ai/guides/voices/voice-design
Create a voice from a written description, audition it, and convert it into a voice you can use everywhere
Voice Design creates natural, realistic voices from a text description.
Describe the character you want for your voice agent or product, and Gradium
generates a few candidates. You listen, then convert the one you like into a
permanent voice. From then on it works like any other `voice_id`: REST,
WebSocket and Speech-to-Speech. No reference audio is needed.
Custom voices clone a speaker from a short audio sample.
## How it works
Send a description, get candidate voices back in seconds.
Audition each candidate on a short line of Text-to-Speech.
Promote your pick to a permanent `voice_id`.
Text-to-Speech, streaming, and Speech-to-Speech all accept it.
Describe the voice in one or two sentences, up to 500 characters, in English,
French, Spanish, Portuguese or German. Gradium expands the description into a
fuller specification and samples 1 to 5 complete voices from it, ready in a few
seconds. Audition each on a short line, then convert the one you want. Until you
convert it, a candidate cannot be used in production.
Two things to know. The model samples a new voice on every request, so if you
like a candidate, convert it: re-running the request will not bring it back. And
the candidates from one request are variations on one character. For a different
character, change the description.
## Access
Base URL `https://api.gradium.ai/api`, API key in the `x-api-key` header.
| Call | Endpoint |
| :- | :- |
| Generate candidates | `POST /voice-generator/generate` |
| Check readiness, or list candidates | `GET /voice-generator/embeddings` |
| Audition a candidate | `POST /post/speech/tts` |
| Convert a candidate | `POST /voices/from-embedding` |
| Delete a candidate | `DELETE /voice-generator/embeddings/{embedding_id}` |
## Quickstart
The steps run as one sequence, carrying ids in shell variables or Python names.
```bash cURL theme={null}
curl -s -X POST https://api.gradium.ai/api/voice-generator/generate \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"prompt": "A British female voice, 20 to 30, glossy and confident, with girly chatter, high pitch, fast pacing, high energy and bright sparkling resonance. Ideal for a friendly receptionist or assistant.",
"language": "en",
"n_samples": 3
}' > candidates.json
CAND0=$(jq -r '.embeddings[0].embedding_id' candidates.json)
CAND1=$(jq -r '.embeddings[1].embedding_id' candidates.json)
```
```python Python theme={null}
import os
import requests
API_KEY = os.environ["GRADIUM_API_KEY"]
BASE = "https://api.gradium.ai/api"
resp = requests.post(
f"{BASE}/voice-generator/generate",
json={
"prompt": (
"A British female voice, 20 to 30, glossy and confident, with girly "
"chatter, high pitch, fast pacing, high energy and bright sparkling "
"resonance. Ideal for a friendly receptionist or assistant."
),
"language": "en",
"n_samples": 3,
},
headers={"x-api-key": API_KEY},
)
resp.raise_for_status()
candidates = [e["embedding_id"] for e in resp.json()["embeddings"]]
```
```json 201 Created theme={null}
{
"embeddings": [
{ "embedding_id": "vox_emb_njsiEgpj5NjHKdZc", "ready": false, "expires_at": "2026-10-03T11:00:00Z" },
{ "embedding_id": "vox_emb_39vb1XVZpnSF5Rt4", "ready": false, "expires_at": "2026-10-03T11:00:00Z" },
{ "embedding_id": "vox_emb_99mdAvteTpRBRBdW", "ready": false, "expires_at": "2026-10-03T11:00:00Z" }
]
}
```
Generation runs in the background, so the ids come back with `ready: false`.
Every request mints new ids. Store them. To recover ids you did not store,
list your candidates with `GET /voice-generator/embeddings`.
```bash cURL theme={null}
until curl -s -H "x-api-key: $GRADIUM_API_KEY" \
"https://api.gradium.ai/api/voice-generator/embeddings?embedding_id=$CAND0" \
| jq -e '.embeddings[0].ready' > /dev/null; do
sleep 2
done
```
```python Python theme={null}
import time
def wait_until_ready(embedding_id, timeout_s=120):
deadline = time.monotonic() + timeout_s
while time.monotonic() < deadline:
resp = requests.get(
f"{BASE}/voice-generator/embeddings",
params={"embedding_id": embedding_id},
headers={"x-api-key": API_KEY},
)
resp.raise_for_status()
found = resp.json()["embeddings"]
if found and found[0]["ready"]:
return found[0]
time.sleep(2)
raise TimeoutError(embedding_id)
for embedding_id in candidates:
wait_until_ready(embedding_id)
```
```json 200 OK theme={null}
{
"embeddings": [{
"embedding_id": "vox_emb_njsiEgpj5NjHKdZc",
"ready": true,
"prompt": "A British female voice, 20 to 30, ...",
"language": "en",
"created_at": "2026-09-03T11:00:00.968734",
"expires_at": "2026-10-03T11:00:00.959626"
}]
}
```
Three candidates typically take three to five seconds. Bound the loop at
about two minutes and offer a retry.
An unknown id returns `200` with an empty `embeddings` list, not `404`, so
check the list before indexing into it. Timestamps are UTC with or without a
trailing `Z`.
Pass the candidate id as `voice_id` on the normal Text-to-Speech endpoint. The
`vox_emb_` prefix is how the API tells a candidate from a converted voice.
```bash cURL theme={null}
curl -s -X POST https://api.gradium.ai/api/post/speech/tts \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"text\": \"Hi there, thanks so much for calling! How can I help you today?\",
\"voice_id\": \"$CAND0\",
\"model_name\": \"default\",
\"output_format\": \"wav\",
\"only_audio\": true
}" --output candidate-0.wav
```
```python Python theme={null}
resp = requests.post(
f"{BASE}/post/speech/tts",
json={
"text": "Hi there, thanks so much for calling! How can I help you today?",
"voice_id": candidates[0],
"model_name": "default",
"output_format": "wav",
"only_audio": True,
},
headers={"x-api-key": API_KEY},
)
resp.raise_for_status()
with open("candidate-0.wav", "wb") as f:
f.write(resp.content)
```
The response body is the audio, 48 kHz mono. Repeat for the other candidates.
Candidates have three restrictions that converted voices do not:
* Text is capped at 100 characters (`400 input text too long`).
* REST only. The TTS WebSocket and Speech-to-Speech reject candidate ids with error `1011`.
* A candidate that is not ready, unknown, or from another account returns `404 Embedding not found`.
Audition with the `model_name` and `json_config` you will ship, so the voice
you approve is the voice you get.
A candidate is a draft, deleted 30 days after generation. Converting it
creates a permanent voice in your library.
```bash cURL theme={null}
curl -s -X POST https://api.gradium.ai/api/voices/from-embedding \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"voxium_embedding_id\": \"$CAND0\",
\"name\": \"Receptionist EN\",
\"description\": \"Front desk and appointment booking\"
}" > voice.json
VOICE=$(jq -r '.uid' voice.json)
```
```python Python theme={null}
resp = requests.post(
f"{BASE}/voices/from-embedding",
json={
"voxium_embedding_id": candidates[0],
"name": "Receptionist EN",
"description": "Front desk and appointment booking",
},
headers={"x-api-key": API_KEY},
)
resp.raise_for_status()
voice_id = resp.json()["uid"]
```
```json 201 Created theme={null}
{
"uid": "b0ntuVzgFdUGoSPc",
"name": "Receptionist EN",
"description": "Front desk and appointment booking",
"filename": "vox_emb_njsiEgpj5NjHKdZc",
"start_s": 0.0,
"is_catalog": false,
"is_pro_clone": false,
"language": "en",
"tags": []
}
```
`uid` is your permanent `voice_id`. Store this one. It is the same value as
`voxium_embedding_id` in the request and `voice_id` everywhere else.
* Converting is free. The voice uses one custom-voice slot, shared with [voice clones](/guides/voices/custom-voices). Over the allowance: `409 Custom voice limit reached`.
* It clears the candidate's expiry.
* Converting the same candidate twice: `409 A voice was already created from this embedding: `. Store the candidate-to-voice mapping yourself rather than parsing this.
* Not ready or unknown: `409 Embedding is not ready yet`.
```bash cURL theme={null}
curl -s -X POST https://api.gradium.ai/api/post/speech/tts \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"text\": \"Good morning, and welcome to Northgate! Do you have an appointment with us today, or shall I get you booked in?\",
\"voice_id\": \"$VOICE\",
\"model_name\": \"default\",
\"output_format\": \"wav\",
\"only_audio\": true
}" --output line-0.wav
```
```python Python theme={null}
resp = requests.post(
f"{BASE}/post/speech/tts",
json={
"text": (
"Good morning, and welcome to Northgate! Do you have an appointment "
"with us today, or shall I get you booked in?"
),
"voice_id": voice_id,
"model_name": "default",
"output_format": "wav",
"only_audio": True,
},
headers={"x-api-key": API_KEY},
)
resp.raise_for_status()
with open("line-0.wav", "wb") as f:
f.write(resp.content)
```
No length cap here. The same `voice_id` works with the
[Python SDK](/guides/text-to-speech), the
[TTS WebSocket](/api-reference/endpoint/tts-websocket) and
[Speech-to-Speech](/guides/speech-to-speech).
Candidates you do not convert are removed after 30 days. To remove one sooner:
```bash cURL theme={null}
curl -s -X DELETE \
-H "x-api-key: $GRADIUM_API_KEY" \
"https://api.gradium.ai/api/voice-generator/embeddings/$CAND1"
```
Safe at any time: a converted voice holds its own copy. An id that is already
gone returns `404`.
## Complete example
The whole flow in one file. Set `GRADIUM_API_KEY` and run it. It creates a
real voice in your account.
```python voice_design.py theme={null}
import os
import time
import requests
BASE_URL = "https://api.gradium.ai/api"
HEADERS = {"x-api-key": os.environ["GRADIUM_API_KEY"]}
DESCRIPTION = (
"A British female voice, 20 to 30, glossy and confident, with high pitch, "
"fast pacing, high energy and bright sparkling resonance. Ideal for a "
"friendly receptionist or assistant."
)
AUDITION_LINE = "Hi there, thanks so much for calling Northgate! How can I help you today?"
def check(resp):
if not resp.ok:
raise RuntimeError(f"HTTP {resp.status_code}: {resp.text}")
return resp
def generate(prompt, language="en", n_samples=3, json_config=None):
body = {"prompt": prompt, "language": language, "n_samples": n_samples}
if json_config:
body["json_config"] = json_config
resp = check(requests.post(f"{BASE_URL}/voice-generator/generate", headers=HEADERS, json=body))
return [c["embedding_id"] for c in resp.json()["embeddings"]]
def wait_until_ready(candidate_ids, timeout_s=120.0):
deadline = time.monotonic() + timeout_s
pending = set(candidate_ids)
while pending:
for candidate_id in sorted(pending):
resp = check(requests.get(
f"{BASE_URL}/voice-generator/embeddings",
headers=HEADERS,
params={"embedding_id": candidate_id},
))
embeddings = resp.json()["embeddings"]
if not embeddings:
raise RuntimeError(f"candidate {candidate_id} not found")
if embeddings[0]["ready"]:
pending.discard(candidate_id)
if pending:
if time.monotonic() > deadline:
raise TimeoutError(f"still not ready: {sorted(pending)}")
time.sleep(2.0)
def synthesise(voice_id, text, path):
resp = check(requests.post(
f"{BASE_URL}/post/speech/tts",
headers=HEADERS,
json={"text": text, "voice_id": voice_id, "output_format": "wav", "only_audio": True},
))
with open(path, "wb") as f:
f.write(resp.content)
def convert(candidate_id, name, description=None):
resp = check(requests.post(
f"{BASE_URL}/voices/from-embedding",
headers=HEADERS,
json={"voxium_embedding_id": candidate_id, "name": name, "description": description},
))
return resp.json()["uid"]
def delete_candidate(candidate_id):
requests.delete(f"{BASE_URL}/voice-generator/embeddings/{candidate_id}", headers=HEADERS)
if __name__ == "__main__":
print("[1/5] generating candidates")
candidate_ids = generate(DESCRIPTION, language="en", n_samples=3)
print("[2/5] waiting until ready")
wait_until_ready(candidate_ids)
print("[3/5] auditioning")
for i, candidate_id in enumerate(candidate_ids):
synthesise(candidate_id, AUDITION_LINE, f"candidate-{i}.wav")
print(f" candidate-{i}.wav {candidate_id}")
# Listen to the files, then pick one. This example keeps the first.
chosen = candidate_ids[0]
print("[4/5] converting the chosen candidate into a voice")
voice_id = convert(chosen, "Receptionist EN", "Front desk and appointment booking")
print(f" voice_id={voice_id}")
print("[5/5] synthesising with the new voice")
synthesise(
voice_id,
"Good morning, and welcome to Northgate! Do you have an appointment with us "
"today, or shall I get you booked in?",
"welcome.wav",
)
for candidate_id in candidate_ids:
if candidate_id != chosen:
delete_candidate(candidate_id)
print("done")
```
```javascript voice_design.mjs theme={null}
import { writeFile } from "node:fs/promises";
const BASE_URL = "https://api.gradium.ai/api";
const HEADERS = {
"x-api-key": process.env.GRADIUM_API_KEY,
"Content-Type": "application/json",
};
const DESCRIPTION =
"A British female voice, 20 to 30, glossy and confident, with high pitch, " +
"fast pacing, high energy and bright sparkling resonance. Ideal for a " +
"friendly receptionist or assistant.";
const AUDITION_LINE =
"Hi there, thanks so much for calling Northgate! How can I help you today?";
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
async function check(resp) {
if (!resp.ok) throw new Error(`HTTP ${resp.status}: ${await resp.text()}`);
return resp;
}
async function generate(prompt, { language = "en", nSamples = 3, jsonConfig } = {}) {
const body = { prompt, language, n_samples: nSamples };
if (jsonConfig) body.json_config = jsonConfig;
const resp = await check(
await fetch(`${BASE_URL}/voice-generator/generate`, {
method: "POST",
headers: HEADERS,
body: JSON.stringify(body),
}),
);
return (await resp.json()).embeddings.map((c) => c.embedding_id);
}
async function waitUntilReady(ids, timeoutMs = 120_000) {
const deadline = Date.now() + timeoutMs;
const pending = new Set(ids);
while (pending.size > 0) {
for (const id of [...pending]) {
const resp = await check(
await fetch(`${BASE_URL}/voice-generator/embeddings?embedding_id=${id}`, {
headers: HEADERS,
}),
);
const { embeddings } = await resp.json();
if (embeddings.length === 0) throw new Error(`candidate ${id} not found`);
if (embeddings[0].ready) pending.delete(id);
}
if (pending.size > 0) {
if (Date.now() > deadline) throw new Error(`still not ready: ${[...pending]}`);
await sleep(2000);
}
}
}
async function synthesise(voiceId, text, path) {
const resp = await check(
await fetch(`${BASE_URL}/post/speech/tts`, {
method: "POST",
headers: HEADERS,
body: JSON.stringify({ text, voice_id: voiceId, output_format: "wav", only_audio: true }),
}),
);
await writeFile(path, Buffer.from(await resp.arrayBuffer()));
}
async function convert(candidateId, name, description) {
const resp = await check(
await fetch(`${BASE_URL}/voices/from-embedding`, {
method: "POST",
headers: HEADERS,
body: JSON.stringify({ voxium_embedding_id: candidateId, name, description }),
}),
);
return (await resp.json()).uid;
}
async function deleteCandidate(candidateId) {
await fetch(`${BASE_URL}/voice-generator/embeddings/${candidateId}`, {
method: "DELETE",
headers: HEADERS,
});
}
console.log("[1/5] generating candidates");
const candidateIds = await generate(DESCRIPTION, { language: "en", nSamples: 3 });
console.log("[2/5] waiting until ready");
await waitUntilReady(candidateIds);
console.log("[3/5] auditioning");
for (const [i, candidateId] of candidateIds.entries()) {
await synthesise(candidateId, AUDITION_LINE, `candidate-${i}.wav`);
console.log(` candidate-${i}.wav ${candidateId}`);
}
// Listen to the files, then pick one. This example keeps the first.
const chosen = candidateIds[0];
console.log("[4/5] converting the chosen candidate into a voice");
const voiceId = await convert(chosen, "Receptionist EN", "Front desk and appointment booking");
console.log(` voice_id=${voiceId}`);
console.log("[5/5] synthesising with the new voice");
await synthesise(
voiceId,
"Good morning, and welcome to Northgate! Do you have an appointment with us " +
"today, or shall I get you booked in?",
"welcome.wav",
);
for (const candidateId of candidateIds) {
if (candidateId !== chosen) await deleteCandidate(candidateId);
}
console.log("done");
```
Audio is streamed, so WAV headers carry placeholder lengths. Readers that trust the header, such as Python's `wave`, report a wrong duration. Rewrite the RIFF and `data` sizes after saving if that matters to you.
## Writing a description
The description is the only input the model has. It responds to the attributes a
casting brief would carry, and naming more of them gives a tighter result.
| Attribute | Examples |
| - | - |
| Gender | female, male, androgynous |
| Age | 20 to 30, middle aged, elderly |
| Accent or origin | British, Southern US, Parisian, Brazilian |
| Pitch | high pitch, deep, mid range |
| Pace | fast pacing, measured, unhurried |
| Energy | high energy, calm, subdued |
| Timbre and resonance | glossy, gravelly, breathy, bright sparkling resonance, warm |
| Register and manner | confident, girly chatter, formal, conspiratorial |
| Intended use | a friendly receptionist, an audiobook narrator, a news anchor |
End with the intended use. It steers delivery and register, not only the colour
of the voice.
> A British female voice, 20 to 30, glossy and confident, with girly
> chatter, high pitch, fast pacing, high energy and bright sparkling
> resonance. Ideal for a friendly receptionist or assistant.
### Rules of thumb
* **Use the space.** Up to 500 characters, all usable. Full sentences beat a list of adjectives.
* **Describe the voice, not the script.** Words to be spoken go in the audition line.
* **Set `language` to the language the voice will speak.** It shapes the accent and the delivery, not only the words. Regional accents still go in the description: "Bristolian" or "Parisian" has to be written out.
* **Concrete beats evaluative.** "Low pitch, slow pacing, gravelly" gives the model more than "a great narrator voice".
### Examples
| Voice | Language | Made for |
| :- | :- | :- |
| **Pirate** | English, Bristol | Character narration |
| **Fionn** | English, Ireland | Customer service |
| **Freya** | English, GB | Receptionist |
| **Desmond** | English, US | Long-form narration |
The descriptions as sent:
**Pirate**
> A gruff Bristolian English male pirate voice, 45 to 60, for game and character
> narration: weathered low pitch, gravelly timbre with heavy vocal fry, strong
> projection, at a steady, unhurried pace, boisterous and commanding energy.
**Fionn**
> An Irish English male voice, 40 to 55, for customer service: calm and crisp,
> with mid-low pitch, steady natural pacing, medium energy and warm rounded
> resonance. Ideal for reassuring walkthroughs, empathic de-escalation and
> complex IT support.
**Freya**
> A British female voice, 20 to 30, glossy and confident, with girly chatter,
> high pitch, fast pacing, high energy and bright sparkling resonance. Ideal for
> a friendly receptionist or assistant.
**Desmond**
> An American English male voice, 55 to 65: clean, deliberate and precise, with
> low pitch, slow pacing and low-to-mid energy, resonant timbre and a gentle
> low-to-high flow. Ideal for projecting academic authority.
### Iterating
Edit the attribute that is off rather than rewriting the description.
| Candidates sound... | Change |
| - | - |
| Too young or old | An explicit range: "40 to 55" rather than "middle aged" |
| Too fast or slow | "unhurried", "measured" or "fast pacing" |
| Flat | An energy word and a manner word |
| Too polished | A timbre word: "gravelly", "breathy", "weathered" |
| Right voice, wrong delivery | An intended-use clause at the end |
| Inconsistent with each other | A higher [`cfg_scale`](#cfg_scale) |
### Building this into a product
If you expose voice design to your own users, prefill a form over the
attributes above and compose the sentence from it, with a free-text box as the
escape hatch. Show the 500 character limit. Offer "keep this one" as the way to
hold onto a voice, and make regenerate send a new description rather than more
samples.
## Generation settings
Generation settings shape the voice and go in `json_config` on
`POST /voice-generator/generate`. [Synthesis settings](#synthesis-settings)
shape each utterance and go in `json_config` on the TTS request. Voice Design
still supports `cfg_scale` to control how literally the voice follows your
description.
All generation settings are optional. The defaults are tuned, so send only the
keys you have a reason to change.
```json theme={null}
{
"prompt": "A calm Irish male voice, 40 to 55, warm and steady, for customer support.",
"language": "en",
"n_samples": 3,
"json_config": { "cfg_scale": 8.0 }
}
```
| Key | Range | Default | Effect |
| - | - | - | - |
| `cfg_scale` | `1.0` to `20.0` | `5.0` | How closely the voice follows the description. Higher is more literal, less varied. |
### `cfg_scale`
* **`5.0`**, the default, when exploring with 3 to 5 candidates and you want them to differ.
* **`8.0` to `12.0`** when candidates drift from the description, or when generating one candidate at a time. Conversational designers that generate one candidate per turn typically use `10.0`.
* **Above `12.0`** the voice follows the description ever more literally at the cost of naturalness, and candidates converge on each other.
`cfg_scale` cannot add an attribute the description does not name. Fix the
[description](#writing-a-description) first.
### The `json_config` allow-list
Send only `cfg_scale`. **An unknown key or out-of-range value is not rejected.** The request returns `201` and the candidates never become `ready`. Bound your polling loop and treat a timeout as a bad request.
The usual causes: a synthesis setting such as `temp`, or `cfg_scale` outside its range.
## Synthesis settings
Auditioning a candidate or synthesising with a converted voice is an ordinary
TTS request, so everything in [Voice Settings](/guides/voice-settings) applies.
See [Model selection](/guides/text-to-speech#model-selection) for TTS model access.
**Deprecated: `cfg_coef` for TTS**
The new TTS model ignores `cfg_coef` when synthesising with a designed voice. Remove it
from TTS requests. `cfg_scale` still applies when generating the voice.
```json theme={null}
{
"text": "Hi there, thanks so much for calling Northgate!",
"voice_id": "vox_emb_njsiEgpj5NjHKdZc",
"output_format": "wav",
"only_audio": true,
"json_config": { "temp": 0.5 }
}
```
### Temperature (`temp`)
`0.0` to `1.4`, default `0.7`.
* **Audition at the default.** `0.0` makes a lively voice sound flatter than it will in production.
* **`0.3` to `0.5`** for scripted, high-volume output where takes must match: IVR prompts, fixed announcements.
* **Default or slightly above** for conversational agents and narration.
## Supported tags
The models read plain text. Two inline tags are recognised, on candidates and
converted voices alike. **Anything else is spoken aloud as text.**
| Tag | Effect |
| - | - |
| `` | Emit audio for all text received so far. Only meaningful when streaming over the WebSocket. |
On a candidate, tags count toward the 100-character limit.
### Not supported
SSML is not interpreted. `Hello there. ` | `padding_bonus`, see [Speed control](/guides/voice-settings#speed-control) |
| ``, `` | The description. Pitch and energy are attributes of the voice. |
| `` | Punctuation and sentence structure |
| ``, ``, `` | [`rewrite_rules`](/guides/text-rewriting) or a [pronunciation dictionary](/api-reference/endpoint/create-pronunciation) |
| `` | A separate request with a different `voice_id` |
Markdown and HTML are read as text too. Strip formatting before sending.
## Candidate lifecycle
| | |
| - | - |
| **Retention** | Candidates are removed 30 days after generation. Converting one clears its expiry. |
| **Accumulation** | Generating never evicts earlier candidates. Holding them is free. |
| **Cleanup** | `DELETE /voice-generator/embeddings/{embedding_id}`. Always safe: a converted voice holds its own copy. Unknown id: `404`. |
| **Deleting a voice** | `DELETE /voices/{voice_id}` leaves the candidate intact, with no expiry. It can be converted again. |
| **Lost an id** | `GET /voice-generator/embeddings` lists every candidate, newest first, with its description. Page with `skip` and `limit` (1 to 1000, default 100). `expires_at: null` marks converted candidates. |
## Limits
| Limit | Value |
| - | - |
| Description | 1 to 500 characters |
| Candidates per request | 1 to 5, default 1 |
| Audition text | 100 characters, REST only |
| Languages | `en`, `fr`, `es`, `pt`, `de` |
| Candidate retention | 30 days, cleared once converted |
| Converted voices | Count against the custom-voice allowance |
## Errors
Errors are `{"detail": "..."}`, except `422`, which carries a list of field errors.
| Status | Where | Meaning |
| - | - | - |
| `400` | Audition | Text over 100 characters |
| `401` | Any | Invalid or expired API key |
| `404` | Audition, delete | Unknown, not ready, or another account's id |
| `409` | Convert | Already converted, not ready, or custom-voice allowance reached. `detail` says which. |
| `422` | Generate, list | Invalid `language`, `n_samples`, `prompt` or `limit` |
A candidate that never becomes ready is almost always a `json_config` value out
of range or a key the API does not recognise. See [the
allow-list](#the-json_config-allow-list).
## Next steps
Make a candidate or converted voice speak another language or accent.
List, update, and delete the voices you convert.
Speed, temperature and other synthesis options.
Use your new voice on the low-latency WebSocket.
The Voice Design endpoints in full.
# Voice Enhance
Source: https://docs.gradium.ai/guides/voices/voice-enhance
Clean up a voice you already have, without changing its language or who is speaking
Voice Enhance takes an existing voice, a flagship voice, a clone or a designed
voice, and produces cleaner candidates of the same speaker in the same
language. A clone made from a noisy recording comes back with the background
noise reduced and a quality target applied; the speaker, the language and the
delivery stay. You audition the candidates, convert the one you like, and it
works like any other `voice_id`: REST, WebSocket and Speech-to-Speech.
Audition every candidate against its source before you convert it.
Clone the speaker first, then enhance the clone.
Voice Design creates a new voice from words. Enhance cleans up a voice you already have.
## How it works
A flagship voice, one of your clones, or a Voice Design candidate.
One request, no other settings. Candidates are ready in about twenty seconds.
Audition each candidate against the source on the same line of Text-to-Speech.
Promote your pick to a permanent `voice_id`.
The source is never modified. Each request creates `n_samples` new
candidates, drafts with a `vox_emb_` id, exactly like Voice Design candidates:
you audition them on `POST /post/speech/tts`, convert them with
`POST /voices/from-embedding`, and they expire after 30 days unless converted.
## Access
Base URL `https://api.gradium.ai/api`, API key in the `x-api-key` header.
| Call | Endpoint |
| :- | :- |
| Enhance a voice | `POST /voice-generator/enhance` |
| Check readiness, or list candidates | `GET /voice-generator/embeddings` |
| Audition a candidate | `POST /post/speech/tts` |
| Convert a candidate | `POST /voices/from-embedding` |
| Delete a candidate | `DELETE /voice-generator/embeddings/{embedding_id}` |
## Quickstart
The example enhances a French clone. Replace `$SOURCE` with any voice id or
candidate id you own, or a flagship voice id.
```bash cURL theme={null}
curl -s -X POST https://api.gradium.ai/api/voice-generator/enhance \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"src_voice\": \"$SOURCE\",
\"n_samples\": 2
}" > candidates.json
CAND0=$(jq -r '.embeddings[0].embedding_id' candidates.json)
```
```python Python theme={null}
import os
import requests
API_KEY = os.environ["GRADIUM_API_KEY"]
BASE = "https://api.gradium.ai/api"
SOURCE = os.environ["SOURCE"] # a voice id or a vox_emb_ candidate id
resp = requests.post(
f"{BASE}/voice-generator/enhance",
json={"src_voice": SOURCE, "n_samples": 2},
headers={"x-api-key": API_KEY},
)
resp.raise_for_status()
candidates = [e["embedding_id"] for e in resp.json()["embeddings"]]
```
```json 201 Created theme={null}
{
"embeddings": [
{ "embedding_id": "vox_emb_XeBcXDgoigTfL9Cv", "ready": false, "expires_at": "2026-10-21T13:03:52Z" },
{ "embedding_id": "vox_emb_ni8lehVHTb35siXI", "ready": false, "expires_at": "2026-10-21T13:03:53Z" }
]
}
```
| Field | | |
| :- | :- | :- |
| `src_voice` | required | A voice id (your clone, a converted candidate, or a flagship voice) or a `vox_emb_` candidate id. |
| `n_samples` | optional | 1 to 5 candidates, default 1. |
There is nothing else to set: the language comes from the source voice, and
so does the speaker. The request is checked before anything is queued, so
an unknown source or a source that cannot be enhanced fails here with
`404` or `409` and nothing is billed.
```bash cURL theme={null}
until curl -s -H "x-api-key: $GRADIUM_API_KEY" \
"https://api.gradium.ai/api/voice-generator/embeddings?embedding_id=$CAND0" \
| jq -e '.embeddings[0].ready' > /dev/null; do
sleep 2
done
```
```python Python theme={null}
import time
def wait_until_ready(embedding_id, timeout_s=120):
deadline = time.monotonic() + timeout_s
while time.monotonic() < deadline:
resp = requests.get(
f"{BASE}/voice-generator/embeddings",
params={"embedding_id": embedding_id},
headers={"x-api-key": API_KEY},
)
resp.raise_for_status()
found = resp.json()["embeddings"]
if found and found[0]["ready"]:
return found[0]
time.sleep(2)
raise TimeoutError(embedding_id)
for embedding_id in candidates:
wait_until_ready(embedding_id)
```
```json 200 OK theme={null}
{
"embeddings": [{
"embedding_id": "vox_emb_XeBcXDgoigTfL9Cv",
"ready": true,
"kind": "enhance",
"language": "fr",
"created_at": "2026-09-21T13:03:52.591128",
"expires_at": "2026-10-21T13:03:52.495534",
"enhance_config": {
"src_voice": "H5oin0KHqRTAxaeL",
"src_language": "fr"
},
"prompt": null
}]
}
```
Enhanced candidates are typically ready in fifteen to twenty seconds. In
the listing, `kind` is `enhance` and `enhance_config` holds the source id
and its language. `language` is the source voice's language, which the
candidate keeps. `prompt` is empty for an enhanced candidate.
Pass the candidate id as `voice_id` on the Text-to-Speech endpoint, with
text in the voice's language. Synthesise the same line with the source
voice too: the difference between the two files is what Enhance did.
```bash cURL theme={null}
for VOICE in "$CAND0" "$SOURCE"; do
curl -s -X POST https://api.gradium.ai/api/post/speech/tts \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"text\": \"Bonjour, c'est Constance. Même voix, un peu plus propre, j'espère.\",
\"voice_id\": \"$VOICE\",
\"model_name\": \"default\",
\"output_format\": \"wav\",
\"only_audio\": true
}" --output "$VOICE.wav"
done
```
```python Python theme={null}
LINE = "Bonjour, c'est Constance. Même voix, un peu plus propre, j'espère."
for voice_id, path in [(candidates[0], "candidate-0.wav"), (SOURCE, "source.wav")]:
resp = requests.post(
f"{BASE}/post/speech/tts",
json={
"text": LINE,
"voice_id": voice_id,
"model_name": "default",
"output_format": "wav",
"only_audio": True,
},
headers={"x-api-key": API_KEY},
)
resp.raise_for_status()
with open(path, "wb") as f:
f.write(resp.content)
```
Candidates keep the same restrictions as Voice Design candidates: REST
only, capped audition text, and an opaque `404 Embedding not found` until
ready.
```bash cURL theme={null}
curl -s -X POST https://api.gradium.ai/api/voices/from-embedding \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"voxium_embedding_id\": \"$CAND0\",
\"name\": \"Constance (enhanced)\",
\"description\": \"French support voice, enhanced\"
}" > voice.json
VOICE=$(jq -r '.uid' voice.json)
```
```python Python theme={null}
resp = requests.post(
f"{BASE}/voices/from-embedding",
json={
"voxium_embedding_id": candidates[0],
"name": "Constance (enhanced)",
"description": "French support voice, enhanced",
},
headers={"x-api-key": API_KEY},
)
resp.raise_for_status()
voice_id = resp.json()["uid"]
```
```json 201 Created theme={null}
{
"uid": "w0F73K2fcf3X2Vcn",
"name": "Constance (enhanced)",
"description": "French support voice, enhanced",
"filename": "vox_emb_XeBcXDgoigTfL9Cv",
"start_s": 0.0,
"is_catalog": false,
"is_pro_clone": false,
"language": "fr",
"tags": []
}
```
`uid` is your permanent `voice_id`. Store it. The new voice keeps the
source's `language`. Converting is free, uses one custom-voice slot, and
clears the candidate's expiry. Candidates from one request are variations:
if you like one, convert it rather than re-running the request. The source
voice is untouched, so you can keep both or delete the source once you are
happy with the enhanced one.
## Complete example
Set `GRADIUM_API_KEY` and `SOURCE`, then run it. It creates a real voice in
your account.
```python voice_enhance.py theme={null}
import os
import time
import requests
BASE_URL = "https://api.gradium.ai/api"
HEADERS = {"x-api-key": os.environ["GRADIUM_API_KEY"]}
SOURCE = os.environ["SOURCE"]
LINE = "Hi, this is your assistant. How can I help you today?" # in the source voice's language
N_SAMPLES = 3
def check(resp):
if not resp.ok:
raise RuntimeError(f"HTTP {resp.status_code}: {resp.text}")
return resp
def enhance(src_voice, n_samples=1):
resp = check(requests.post(
f"{BASE_URL}/voice-generator/enhance",
headers=HEADERS,
json={"src_voice": src_voice, "n_samples": n_samples},
))
return [c["embedding_id"] for c in resp.json()["embeddings"]]
def wait_until_ready(candidate_ids, timeout_s=120.0):
deadline = time.monotonic() + timeout_s
pending = set(candidate_ids)
while pending:
for candidate_id in sorted(pending):
resp = check(requests.get(
f"{BASE_URL}/voice-generator/embeddings",
headers=HEADERS,
params={"embedding_id": candidate_id},
))
embeddings = resp.json()["embeddings"]
if not embeddings:
raise RuntimeError(f"candidate {candidate_id} not found")
if embeddings[0]["ready"]:
pending.discard(candidate_id)
if pending:
if time.monotonic() > deadline:
raise TimeoutError(f"still not ready: {sorted(pending)}")
time.sleep(2.0)
def synthesise(voice_id, text, path):
resp = check(requests.post(
f"{BASE_URL}/post/speech/tts",
headers=HEADERS,
json={"text": text, "voice_id": voice_id, "output_format": "wav", "only_audio": True},
))
with open(path, "wb") as f:
f.write(resp.content)
def convert(candidate_id, name, description=None):
resp = check(requests.post(
f"{BASE_URL}/voices/from-embedding",
headers=HEADERS,
json={"voxium_embedding_id": candidate_id, "name": name, "description": description},
))
return resp.json()["uid"]
if __name__ == "__main__":
print("[1/4] enhancing")
candidates = enhance(SOURCE, N_SAMPLES)
print("[2/4] waiting until ready")
wait_until_ready(candidates)
print("[3/4] auditioning")
synthesise(SOURCE, LINE, "source.wav") # the same line without enhancement
for i, candidate_id in enumerate(candidates):
synthesise(candidate_id, LINE, f"candidate-{i}.wav")
print(f" candidate-{i}.wav vs source.wav")
# Listen, then convert the one you keep. This example keeps the first.
print("[4/4] converting")
voice_id = convert(candidates[0], "Assistant (enhanced)", f"Enhanced from {SOURCE}")
print(f" voice_id={voice_id}")
print("done")
```
## Sources you can enhance
| Source | Works | Notes |
| :- | :- | :- |
| Flagship voice | Yes | Any id from [Flagship Voices](/guides/voices/flagship-voices). |
| Instant clone | Yes, if it has a language | Clones created without a `language` return `409 The source voice has no language`. Set it once with [`PUT /voices/{voice_id}`](/guides/voices/manage-voices#update-a-voice) and retry. The language must be the one spoken in the recording: a wrong tag is accepted and quietly degrades the result. |
| Voice Design candidate or converted voice | Yes | Use the `vox_emb_` id or the converted `voice_id`. |
| Enhanced candidate | Yes | The candidate must be ready (`409` otherwise). |
| Pro clone | No | `409 This voice has no embedding the voice generator can edit.` |
| Another account's voice | No | Opaque `404`. |
## Tips from testing
* **Compare against the source reading the same text.** Enhance changes how
the voice sounds, not what it says or in which language. Put the source and
the candidate side by side in your review UI, on the same line.
* **Start with a clone.** Flagship voices are produced from clean recordings
already; the clone you wish sounded cleaner is the natural source.
* **Candidates vary.** Five samples from one request, or five separate
requests, give five slightly different results. Convert the one you like;
do not expect a re-run to reproduce it.
* **Keep the source.** Converting a candidate does not replace the source
voice. Switch your `voice_id` over once you have listened, and delete the
old one when you no longer need it.
* **Budget the wait.** Enhanced candidates take fifteen to twenty seconds,
longer than Voice Design candidates. Poll rather than block a request on it.
## Candidate lifecycle
Enhanced candidates follow the [Voice Design candidate
lifecycle](/guides/voices/voice-design#candidate-lifecycle): 30 days retention,
cleared on conversion, `DELETE /voice-generator/embeddings/{embedding_id}` to
remove one early. Deleting the source voice or candidate after the `201` never
breaks the request: each candidate holds its own copy of the source.
## Limits
| Limit | Value |
| - | - |
| Candidates per request | 1 to 5, default 1 |
| Languages | The source voice's language: `en`, `fr`, `es`, `pt` or `de` |
| Audition text | REST only, same cap as Voice Design candidates |
| Candidate retention | 30 days, cleared once converted |
| Converted voices | Count against the custom-voice allowance |
## Errors
Errors are `{"detail": "..."}`, except `422`, which carries a list of field
errors.
| Status | Where | Meaning |
| - | - | - |
| `401` | Any | Invalid or expired API key |
| `404` | Enhance | Unknown source, or another account's voice or candidate |
| `404` | Audition | Candidate not ready, unknown, or deleted |
| `409` | Enhance | Source has no language, source is a candidate that is not ready, or source is a pro clone. `detail` says which. |
| `409` | Convert | Already converted, not ready, or custom-voice allowance reached |
| `422` | Enhance | Invalid `n_samples`, or an empty `src_voice` |
| `503` | Enhance | Source audio could not be copied, or no voice generator is available. Retry. |
## Next steps
Clone a speaker, with a language, so the clone can be enhanced.
Create a voice from a description instead.
Set a language on an older clone, list and delete voices.
The enhance endpoint in full.
# Voice Localization
Source: https://docs.gradium.ai/guides/voices/voice-localization
Adapt an existing voice to another language or accent
Voice Localization is available through the API only.
Listen to each candidate before saving it as a permanent voice.
Choose a flagship voice, a clone or a designed voice, then select a target
language and accent. Voice Localization creates candidates for you to preview
and save to your voice library.
Voice Design creates a new voice from words. Localization edits a voice you already have.
Clone the speaker first, then localize the clone.
## How it works
A flagship voice, one of your clones, or a Voice Design candidate.
Choose a target language and an accent to generate candidates.
Audition each candidate on a line of Text-to-Speech in the target language.
Save a candidate as a permanent voice in your library.
Each request creates `n_samples` candidates without modifying the source.
A candidate is a temporary voice with a `vox_emb_` ID. Preview it with REST
Text-to-Speech, then use `POST /voices/from-embedding` to save it as a permanent
voice. The response's `uid` is the `voice_id` you use with REST, WebSocket and
Speech-to-Speech. Unsaved candidates expire after 30 days.
To change only the accent, keep the source language as `target_language` and
choose another accent. For example, use `en` and `Australian` for a British
English source.
## Access
Base URL `https://api.gradium.ai/api`, API key in the `x-api-key` header.
| Call | Endpoint |
| :- | :- |
| List the accents you can target | `GET /voice-generator/available-accents` |
| Localize a voice | `POST /voice-generator/localize` |
| Check readiness, or list candidates | `GET /voice-generator/embeddings` |
| Audition a candidate | `POST /post/speech/tts` |
| Convert a candidate | `POST /voices/from-embedding` |
| Delete a candidate | `DELETE /voice-generator/embeddings/{embedding_id}` |
## Quickstart
The example localizes a French clone into American English. Set
`GRADIUM_API_KEY` and `SOURCE` in your environment. `SOURCE` can be a voice or
ready candidate you own, or a flagship voice ID. Shell examples require `jq`;
Python examples require `requests`. Run the steps in order using one language.
```bash cURL theme={null}
curl --fail --silent --show-error --max-time 30 -H "x-api-key: $GRADIUM_API_KEY" \
https://api.gradium.ai/api/voice-generator/available-accents
```
```json 200 OK (abbreviated) theme={null}
{
"languages": {
"en": ["General American", "Standard British", "Irish", "..."],
"fr": ["General French", "Quebecois French"],
"...": ["..."]
}
}
```
Read the list from this endpoint at runtime rather than copying it from
these docs: it will grow. You can cache it. The first accent of each
language is the default when a request omits `accent`. Spell accents as
listed; matching is case-insensitive and ignores surrounding spaces, but
"British", "Quebecois" or "Québécois French" are rejected.
```bash cURL theme={null}
curl --fail --silent --show-error --max-time 30 -X POST https://api.gradium.ai/api/voice-generator/localize \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"src_voice\": \"$SOURCE\",
\"target_language\": \"en\",
\"accent\": \"General American\",
\"gender\": \"female\",
\"n_samples\": 2
}" > candidates.json
CAND0=$(jq -r '.embeddings[0].embedding_id' candidates.json)
```
```python Python theme={null}
import os
import requests
API_KEY = os.environ["GRADIUM_API_KEY"]
BASE = "https://api.gradium.ai/api"
SOURCE = os.environ["SOURCE"] # a voice id or a vox_emb_ candidate id
resp = requests.post(
f"{BASE}/voice-generator/localize",
json={
"src_voice": SOURCE,
"target_language": "en",
"accent": "General American",
"gender": "female",
"n_samples": 2,
},
headers={"x-api-key": API_KEY},
timeout=30,
)
resp.raise_for_status()
candidates = [e["embedding_id"] for e in resp.json()["embeddings"]]
```
```json 201 Created theme={null}
{
"embeddings": [
{ "embedding_id": "vox_emb_XeBcXDgoigTfL9Cv", "ready": false, "expires_at": "2026-10-21T13:03:52Z" },
{ "embedding_id": "vox_emb_ni8lehVHTb35siXI", "ready": false, "expires_at": "2026-10-21T13:03:53Z" }
]
}
```
| Field | Required | Description |
| :- | :- | :- |
| `src_voice` | required | A voice id (your clone, a converted candidate, or a flagship voice) or a `vox_emb_` candidate id. |
| `target_language` | required | `en`, `fr`, `es`, `pt` or `de`. May equal the source language for an accent change. |
| `accent` | optional | One of the accents listed for `target_language`. Defaults to the first one. |
| `gender` | optional | `male` or `female`. See [Gender](#gender). |
| `n_samples` | optional | 1 to 5 candidates, default 1. |
The request is checked before anything is queued, so a bad accent, an
unknown source or a source that cannot be localized fails here with `422`,
`404` or `409` and nothing is billed.
```bash cURL theme={null}
wait_for_candidate() {
local deadline=$((SECONDS + 120))
local result
while (( SECONDS < deadline )); do
result=$(curl --fail --silent --show-error --max-time 10 \
-H "x-api-key: $GRADIUM_API_KEY" \
"https://api.gradium.ai/api/voice-generator/embeddings?embedding_id=$1") || return 1
if ! jq -e '.embeddings | length > 0' <<< "$result" > /dev/null; then
echo "Candidate not found: $1" >&2
return 1
fi
if jq -e '.embeddings[0].ready' <<< "$result" > /dev/null; then
return 0
fi
sleep 2
done
echo "Timed out waiting for candidate: $1" >&2
return 1
}
wait_for_candidate "$CAND0"
```
```python Python theme={null}
import time
def wait_until_ready(embedding_id, timeout_s=120):
deadline = time.monotonic() + timeout_s
while time.monotonic() < deadline:
resp = requests.get(
f"{BASE}/voice-generator/embeddings",
params={"embedding_id": embedding_id},
headers={"x-api-key": API_KEY},
timeout=min(10, max(0.1, deadline - time.monotonic())),
)
resp.raise_for_status()
found = resp.json()["embeddings"]
if not found:
raise RuntimeError(f"candidate {embedding_id} not found")
if found and found[0]["ready"]:
return found[0]
time.sleep(2)
raise TimeoutError(embedding_id)
for embedding_id in candidates:
wait_until_ready(embedding_id)
```
```json 200 OK theme={null}
{
"embeddings": [{
"embedding_id": "vox_emb_XeBcXDgoigTfL9Cv",
"ready": true,
"kind": "edit_language",
"language": "en",
"created_at": "2026-09-21T13:03:52.591128",
"expires_at": "2026-10-21T13:03:52.495534",
"generate_config": null,
"edit_language_config": {
"src_voice": "H5oin0KHqRTAxaeL",
"src_language": "fr",
"target_language": "en",
"accent": "General American",
"gender": "female"
},
"enhance_config": null,
"prompt": "Feminine. General American accent."
}]
}
```
Localized candidates are typically ready in two to eight seconds. Poll
until `ready` is `true` before requesting audio. An unknown candidate
returns an empty `embeddings` list. The shell example waits for the first
candidate; repeat it for any others you want to preview.
`kind` is `edit_language` for a localized candidate (`generate` and `enhance`
for the others). The matching config object, `edit_language_config` here,
contains the request settings, and `language` is the candidate's target
language. Read the config objects instead of the deprecated `prompt` field.
Pass the candidate id as `voice_id` on the Text-to-Speech endpoint, with
text **in the target language**.
```bash cURL theme={null}
curl --fail --silent --show-error --max-time 30 -X POST https://api.gradium.ai/api/post/speech/tts \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"text\": \"Hi, this is Constance. Same voice, new language, and hopefully the same personality.\",
\"voice_id\": \"$CAND0\",
\"model_name\": \"default\",
\"output_format\": \"wav\",
\"only_audio\": true
}" --output candidate-0.wav
```
```python Python theme={null}
resp = requests.post(
f"{BASE}/post/speech/tts",
json={
"text": "Hi, this is Constance. Same voice, new language, and hopefully the same personality.",
"voice_id": candidates[0],
"model_name": "default",
"output_format": "wav",
"only_audio": True,
},
headers={"x-api-key": API_KEY},
timeout=30,
)
resp.raise_for_status()
with open("candidate-0.wav", "wb") as f:
f.write(resp.content)
```
Compare the candidate with the source reading the same text. Candidate
previews use REST only and accept up to 300 characters (`400 input text
too long` beyond). A candidate that is not ready yet returns
`404 Embedding not found`; an unknown or deleted candidate returns `400`.
Use text in the candidate's target language. Text in another language can
produce poor pronunciation. Keep one localized voice per language.
Conversion saves the candidate as a permanent voice in your library.
Use the response's `uid` as `voice_id` for subsequent speech requests.
```bash cURL theme={null}
curl --fail --silent --show-error --max-time 30 -X POST https://api.gradium.ai/api/voices/from-embedding \
-H "x-api-key: $GRADIUM_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"voxium_embedding_id\": \"$CAND0\",
\"name\": \"Constance EN\",
\"description\": \"French support voice, localized to General American\"
}" > voice.json
VOICE=$(jq -r '.uid' voice.json)
```
```python Python theme={null}
resp = requests.post(
f"{BASE}/voices/from-embedding",
json={
"voxium_embedding_id": candidates[0],
"name": "Constance EN",
"description": "French support voice, localized to General American",
},
headers={"x-api-key": API_KEY},
timeout=30,
)
resp.raise_for_status()
voice_id = resp.json()["uid"]
```
```json 201 Created theme={null}
{
"uid": "w0F73K2fcf3X2Vcn",
"name": "Constance EN",
"description": "French support voice, localized to General American",
"filename": "vox_emb_XeBcXDgoigTfL9Cv",
"start_s": 0.0,
"is_catalog": false,
"is_pro_clone": false,
"language": "en",
"tags": []
}
```
The saved voice keeps the target `language` and can be localized again.
Conversion is free, uses one custom-voice slot, and removes the candidate's
expiry. Save a candidate you like; a new request produces different results.
## Complete example
Set `GRADIUM_API_KEY` and `SOURCE`, then run this example to generate candidates
and comparison audio in three languages. Listen to the files before choosing
which candidates to save. Each saved voice uses a custom-voice slot.
```python voice_localization.py theme={null}
import os
import time
import requests
BASE_URL = "https://api.gradium.ai/api"
HEADERS = {"x-api-key": os.environ["GRADIUM_API_KEY"]}
SOURCE = os.environ["SOURCE"]
TARGETS = [ # (language, accent, audition line)
("en", "General American", "Hi, this is your assistant. How can I help you today?"),
("de", "Standard German", "Hallo, hier ist Ihre Assistentin. Wie kann ich Ihnen heute helfen?"),
("es", "Mexican Spanish", "Hola, soy tu asistente. ¿En qué puedo ayudarte hoy?"),
]
def check(resp):
if not resp.ok:
raise RuntimeError(f"HTTP {resp.status_code}: {resp.text}")
return resp
def localize(src_voice, language, accent, gender=None, n_samples=1):
body = {"src_voice": src_voice, "target_language": language, "accent": accent, "n_samples": n_samples}
if gender:
body["gender"] = gender
resp = check(requests.post(f"{BASE_URL}/voice-generator/localize", headers=HEADERS, json=body, timeout=30))
return [c["embedding_id"] for c in resp.json()["embeddings"]]
def wait_until_ready(candidate_ids, timeout_s=120.0):
deadline = time.monotonic() + timeout_s
pending = set(candidate_ids)
while pending:
for candidate_id in sorted(pending):
resp = check(requests.get(
f"{BASE_URL}/voice-generator/embeddings",
headers=HEADERS,
params={"embedding_id": candidate_id},
timeout=10,
))
embeddings = resp.json()["embeddings"]
if not embeddings:
raise RuntimeError(f"candidate {candidate_id} not found")
if embeddings[0]["ready"]:
pending.discard(candidate_id)
if pending:
if time.monotonic() > deadline:
raise TimeoutError(f"still not ready: {sorted(pending)}")
time.sleep(2.0)
def synthesise(voice_id, text, path):
resp = check(requests.post(
f"{BASE_URL}/post/speech/tts",
headers=HEADERS,
timeout=30,
json={"text": text, "voice_id": voice_id, "output_format": "wav", "only_audio": True},
))
with open(path, "wb") as f:
f.write(resp.content)
def convert(candidate_id, name, description=None):
resp = check(requests.post(
f"{BASE_URL}/voices/from-embedding",
headers=HEADERS,
timeout=30,
json={"voxium_embedding_id": candidate_id, "name": name, "description": description},
))
return resp.json()["uid"]
if __name__ == "__main__":
accents = check(requests.get(f"{BASE_URL}/voice-generator/available-accents", headers=HEADERS, timeout=30)).json()["languages"]
for language, accent, _ in TARGETS:
assert accent in accents[language], f"{accent} is not an accent for {language}"
print("[1/4] localizing")
plan = {language: localize(SOURCE, language, accent)[0] for language, accent, _ in TARGETS}
print("[2/4] waiting until ready")
wait_until_ready(plan.values())
print("[3/4] auditioning")
for language, accent, line in TARGETS:
synthesise(plan[language], line, f"candidate-{language}.wav")
synthesise(SOURCE, line, f"source-{language}.wav") # the same line without localization
print(f" candidate-{language}.wav vs source-{language}.wav ({accent})")
print("[4/4] listen to the files, then choose which voices to save")
for language, accent, _ in TARGETS:
if input(f"Save the {language} candidate as a permanent voice? [y/N] ").strip().lower() != "y":
continue
voice_id = convert(plan[language], f"Assistant {language.upper()}", f"Localized from {SOURCE}: {accent}")
print(f" {language}: voice_id={voice_id}")
print("done")
```
## Choosing an accent
`GET /voice-generator/available-accents` returns the accents you can target,
grouped by language, with the default first. At the time of writing it covers
the five supported languages with between two and six accents each, regional
variants such as Quebecois French, Mexican Spanish or Swiss German included.
Read the list from the endpoint rather than hardcoding it, and show it to your
users as a picker: free-text accents are not accepted. Regional accents the
list does not offer, "Bristolian" or "Texan", belong to
[Voice Design](/guides/voices/voice-design), where they go in the description.
## Gender
Use `gender` to specify `male` or `female`. When you omit it:
1. A flagship voice uses its own gender tag.
2. A candidate reuses the gender its own localization was made with.
3. Otherwise no gender is specified. Clones rarely carry a gender tag,
and converted voices do not, so send `gender` for those sources.
You can request a gender different from the source. Listen to the result to
check whether it still matches the speaker you want.
## Sources you can localize
| Source | Works | Notes |
| :- | :- | :- |
| Flagship voice | Yes | Any id from [Flagship Voices](/guides/voices/flagship-voices). |
| Instant clone | Yes, if it has a language | Clones created without a `language` return `409 The source voice has no language`. Set it once with [`PUT /voices/{voice_id}`](/guides/voices/manage-voices#update-a-voice) and retry. The language must be the one spoken in the recording: a wrong tag is accepted and quietly degrades the result. |
| Voice Design candidate or converted voice | Yes | Use the `vox_emb_` id or the converted `voice_id`. |
| Localized candidate | Yes | Localizations chain: a French candidate made from an English voice can be localized to German. The candidate must be ready (`409` otherwise). |
| Pro clone | No | `409 This voice has no embedding the voice generator can edit.` |
| Another account's voice | No | Opaque `404`. |
## Evaluating candidates
* Compare the source and candidate reading the same target-language text.
Listen for pronunciation, accent and whether you still recognize the speaker.
* For an accent-only comparison, keep the source language and explicitly choose
the target accent. The default accent may differ from the source's accent.
* Compare multiple candidates before saving one. Repeating a request does not
reproduce a previous candidate.
* Use text in the candidate's target language. Keep a separate localized voice
for each language you need.
* To clean up a noisy source rather than change its accent, use
[Voice Enhance](/guides/voices/voice-enhance).
## Candidate lifecycle
Candidates expire after 30 days unless you save them as permanent voices with
`POST /voices/from-embedding`. Saving a candidate removes its expiry and returns
the permanent voice's `uid`. See the [conversion step](#quickstart) for the request
and response.
To remove a candidate early, use
`DELETE /voice-generator/embeddings/{embedding_id}`. A converted voice holds its
own copy. Localized candidates also keep a copy of their source, so deleting
the source after the request is accepted does not cancel generation. See the
[Voice Design candidate lifecycle](/guides/voices/voice-design#candidate-lifecycle)
for listing and cleanup details.
## Limits
| Limit | Value |
| - | - |
| Candidates per request | 1 to 5, default 1 |
| Target languages | `en`, `fr`, `es`, `pt`, `de` |
| Accents | Live list, read `GET /voice-generator/available-accents` |
| `gender` | `male` or `female` |
| Audition text | 300 characters, REST only |
| Candidate retention | 30 days, cleared once converted |
| Converted voices | Count against the custom-voice allowance |
## Errors
Errors are `{"detail": "..."}`, except `422`, which carries a list of field
errors including the valid accents for the language you asked for.
| Status | Where | Meaning |
| - | - | - |
| `401` | Any | Invalid or expired API key |
| `404` | Localize | Unknown source, or another account's voice or candidate |
| `404` | Audition | Candidate not ready yet |
| `400` | Audition | Unknown or deleted candidate, or text over 300 characters |
| `409` | Localize | Source has no language, source is a candidate that is not ready, source is a pro clone or a voice speaking an unsupported language. `detail` says which. |
| `409` | Convert | Already converted, not ready, or custom-voice allowance reached |
| `422` | Localize | Invalid `target_language`, `accent`, `gender`, `n_samples`, or an empty `src_voice` |
| `503` | Localize | Source audio could not be copied, or no voice generator is available. Retry. |
## Next steps
Create the source voice from a description instead.
Clone a speaker, with a language, so the clone can be localized.
Set a language on an older clone, list and delete voices.
The localization endpoints in full.
# WebSocket Lifecycle
Source: https://docs.gradium.ai/guides/websocket-lifecycle
Connection setup, ready messages, input, flush, end-of-stream, multiplexing, and errors
Gradium's real-time APIs use the same WebSocket lifecycle for TTS and
STT:
1. Connect with authentication.
2. Send a `setup` message.
3. Wait for, or lazily receive, `ready`.
4. Send input messages (`text` for TTS, `audio` for STT).
5. Optionally flush buffered input.
6. Send `end_of_stream`.
7. Read output until the server sends `end_of_stream` or `error`.
The Python SDK handles the connection for you, but the lifecycle is the
same if you use the wire protocol directly.
## Endpoints
| Product | WebSocket endpoint | Input | Output |
| - | - | - | - |
| TTS | `wss://api.gradium.ai/api/speech/tts` | `text` messages | `audio`, `text`, `ready`, `end_of_stream`, `error` |
| STT | `wss://api.gradium.ai/api/speech/asr` | `audio`, `flush` messages | `text`, `end_text`, `step`, `flushed`, `ready`, `end_of_stream`, `error` |
## Authentication
Server-side clients should send the API key in the `x-api-key` header:
```bash theme={null}
wscat -c "wss://api.gradium.ai/api/speech/tts" \
-H "x-api-key: your_api_key"
```
Browser clients should not expose API keys. Generate a short-lived,
single-use token on your server and connect with `?token=...`; see
[Browser WebSockets](/guides/browser-websockets).
## Setup
The first logical message for every request is `setup`.
```json TTS setup theme={null}
{
"type": "setup",
"model_name": "default",
"voice_id": "YTpq7expH9539ERJ",
"output_format": "pcm"
}
```
```json STT setup theme={null}
{
"type": "setup",
"model_name": "default",
"input_format": "pcm",
"json_config": {"language": "en", "delay_in_frames": 16}
}
```
Shared setup fields:
| Field | Applies to | Purpose |
| - | - | - |
| `model_name` | TTS, STT | For TTS, defaults to `"default"` (stable). Use `"gradium-tts-beta"` to follow betas. See [Model selection](/guides/text-to-speech#model-selection). |
| `json_config` | TTS, STT | Advanced model settings. SDK calls accept a dict; raw WebSocket clients may send an object or JSON string. |
| `client_req_id` | TTS, STT | Correlates messages when running multiple requests on one socket. |
| `close_ws_on_eos` | TTS, STT | Defaults to `true`. Set `false` to keep the socket open after a request. |
| `retry_for_s` | TTS, STT | Optional setup retry window for transient worker allocation failures. |
TTS-specific setup fields:
| Field | Purpose |
| - | - |
| `voice_id` | Voice library or custom voice ID. Prefer this for production. |
| `voice` | Voice name fallback, defaulting to `"default"` when no `voice_id` is provided. |
| `output_format` | `wav`, `pcm`, `opus`, `ulaw_8000`, `alaw_8000`, or explicit PCM rates such as `pcm_16000`. |
| `pronunciation_id` | Pronunciation dictionary to apply to this request. |
STT-specific setup fields:
| Field | Purpose |
| - | - |
| `input_format` | `pcm`, `wav`, `opus`, `ulaw_8000`, `alaw_8000`, or explicit PCM rates such as `pcm_16000`. |
## Ready
After setup, the server sends `ready`. You can wait for this before
sending input, or start sending immediately and let the SDK capture it
while receiving.
```json TTS ready theme={null}
{
"type": "ready",
"request_id": "req_...",
"model_name": "default",
"model_ext": "resolved-model",
"sample_rate": 48000,
"frame_size": 3840,
"audio_stream_names": [],
"text_stream_names": []
}
```
```json STT ready theme={null}
{
"type": "ready",
"request_id": "req_...",
"model_name": "default",
"sample_rate": 24000,
"frame_size": 1920,
"delay_in_frames": 16,
"text_stream_names": []
}
```
Use `request_id` in logs and support tickets. For STT, use
`delay_in_frames` when tuning turn-taking or forced flush behavior.
## Input
TTS accepts text messages:
```json theme={null}
{"type": "text", "text": "Hello, world."}
```
When streaming text from an LLM, split on whitespace or sentence
boundaries. Do not split inside a word or separate punctuation into a
standalone message; the server treats successive text messages as
separate chunks and inserts spacing between them.
STT accepts base64-encoded audio messages:
```json theme={null}
{"type": "audio", "audio": "base64_encoded_audio"}
```
For raw PCM, use 80 ms chunks when possible:
| Format | Sample rate | Samples per 80 ms | Bytes per chunk |
| - | -: | -: | -: |
| `pcm` | 24 kHz | 1920 | 3840 |
| `pcm_8000` | 8 kHz | 640 | 1280 |
| `pcm_16000` | 16 kHz | 1280 | 2560 |
| `pcm_48000` | 48 kHz | 3840 | 7680 |
## Flush
TTS supports model-level flushing with the `` tag inside text:
```json theme={null}
{"type": "text", "text": "The answer is ready. "}
```
Use this when an upstream LLM has finished a thought and you want the
model to emit remaining buffered audio without waiting for more text.
Avoid flushing after every token; small text fragments reduce prosody.
STT supports a `flush` message:
```json theme={null}
{"type": "flush", "flush_id": 1}
```
The server processes outstanding audio and responds with:
```json theme={null}
{"type": "flushed", "flush_id": 1}
```
Use STT flush when your application has detected a turn boundary and
needs any pending transcript before passing the turn to an agent.
## End
Send `end_of_stream` when you are done sending input for a request:
```json theme={null}
{"type": "end_of_stream"}
```
For a single-use connection, the server sends final output and closes
the WebSocket. For a reusable or multiplexed connection, set
`close_ws_on_eos: false` in setup and keep sending new setup/input
groups.
## Multiplexing
To run multiple logical requests over one socket:
1. Set `close_ws_on_eos: false`.
2. Attach a unique `client_req_id` to every message for a request.
3. Route every response by its matching `client_req_id`.
See [Multiplexing](/guides/multiplexing) for full examples.
## Errors
WebSocket errors are sent as JSON and then the socket closes:
```json theme={null}
{"type": "error", "message": "Session not found. Send setup first.", "code": 1002}
```
Treat `error` as terminal for that socket. Open a new connection when
retrying. Common codes:
| Code | Meaning |
| -: | - |
| `1002` | Protocol error, such as sending input before setup or reusing an active `client_req_id`. |
| `1008` | Policy violation, such as invalid auth, missing subscription, or invalid request policy. |
| `1011` | Internal server error or unexpected session failure. |
For REST and WebSocket error contracts, see [Errors](/guides/errors).
## Next steps
Stream text in and receive audio chunks back.
Stream audio in and receive text, VAD, and flush events.
Run several logical requests on one WebSocket.
Use short-lived tokens without exposing API keys.
# WebSocket Stream Options
Source: https://docs.gradium.ai/guides/websocket-stream-options
Connection initialisation flags shared by tts_realtime and stt_realtime
When using the real-time WebSocket APIs (`tts_realtime` or
`stt_realtime`), two parameters control how the connection initialises.
Both work identically across TTS and STT.
For the full protocol lifecycle, including raw WebSocket messages,
browser tokens, multiplexing, and error behavior, see
[WebSocket Lifecycle](/guides/websocket-lifecycle).
## `send_setup_on_start`
Controls whether the setup message is automatically sent when the
context manager is entered. Defaults to `True`.
Set this to `False` when you need to send setup manually, for example
when using [multiplexing](/guides/multiplexing) where each request has
its own setup with a unique `client_req_id`.
```python theme={null}
# Setup sent automatically (default)
async with client.tts_realtime(voice_id="YTpq7expH9539ERJ", output_format="pcm") as stream:
await stream.send_text("Hello")
# Setup sent manually
async with client.tts_realtime(send_setup_on_start=False) as stream:
await stream.send_setup({"voice_id": "YTpq7expH9539ERJ", "output_format": "pcm"})
await stream.send_text("Hello")
```
## `wait_for_ready_on_start`
Controls whether the client blocks waiting for the server's `ready`
message after sending setup. Defaults to `False`.
When set to `False`, the `ready` message is captured lazily during the
normal receive loop. This reduces connection latency since you can
start sending data immediately after setup without waiting for a
round-trip.
When set to `True`, `stream.ready` is guaranteed to be populated before
you start sending data, which can be useful if you need
server-provided metadata (like sample rate) before proceeding.
```python theme={null}
# Non-blocking (default), ready captured lazily during recv
async with client.tts_realtime(
voice_id="YTpq7expH9539ERJ",
output_format="pcm"
) as stream:
# stream.ready is None here, start sending immediately
await stream.send_text("Hello")
await stream.send_eos()
async for msg in stream:
# stream.ready gets populated when the ready message arrives
if msg["type"] == "audio":
process_audio(msg["audio"])
# Blocking, wait for ready before sending
async with client.tts_realtime(
voice_id="YTpq7expH9539ERJ",
output_format="pcm",
wait_for_ready_on_start=True
) as stream:
# stream.ready is populated here
print(f"Server ready: {stream.ready}")
await stream.send_text("Hello")
```
Both parameters work identically for `stt_realtime`:
```python theme={null}
async with client.stt_realtime(
model_name="default",
input_format="pcm",
json_config={"language": "en"},
wait_for_ready_on_start=True
) as stream:
print(f"Server ready: {stream.ready}")
await stream.send_audio(audio_data)
```
# How we use your feedback
Source: https://docs.gradium.ai/how-your-data-is-used-to-improve-model-performance
What you share with Gradium when you submit feedback, and how we use it.
Your feedback helps us investigate quality issues and improve the Gradium product. We do not use feedback submitted through this form to train models.
## What you share
When you click **Submit**, we collect:
* Your rating, selected categories, and optional comment.
* Identifiers linking your feedback to the relevant generation or session and your account.
* For Speech-to-Text, whether you uploaded a file or used your microphone.
* For Translation, the session’s mode and target language.
Anything you write in the comment is included, so please avoid sharing sensitive information.
## Sharing feedback is optional
You can use Gradium without submitting feedback. Closing the form without submitting does not send your comment or selected categories.
## If you’ve opted out of data retention
If your organization has opted out of data retention, nothing is collected through this feedback form, and we will not receive your feedback.
To share feedback with us, please [contact customer support](https://studio.gradium.ai/support/contact).
## Questions and removal requests
To ask how your feedback is handled or request its removal, please [contact customer support](https://studio.gradium.ai/support/contact).
# Gradium API Documentation
Source: https://docs.gradium.ai/index
Build voice apps with Gradium's low-latency Text-to-Speech and Speech-to-Text APIs, SDK guides, streaming examples, voices, and pronunciation tools.
## Get Started
Build voice-powered applications with the Gradium API. Our TTS and STT models deliver natural-sounding output with best-in-class accuracy, low latency, semantic VAD for turn-taking, and adaptive delay controls for realtime agents.
Install the Python SDK and make your first API call.
Convert text to natural-sounding speech with streaming support.
Transcribe audio in real time with semantic VAD and flush.
Explore the full REST and WebSocket API reference.
## Explore
Browse available voices or create custom voice clones.
Control speed, temperature, and voice similarity.
Use semantic VAD and adaptive delay for conversational agents.
Browse complete voice-agent demos built with Gradium STT and TTS.
# Gradbot
Source: https://docs.gradium.ai/integrations/agent-frameworks/gradbot
Prototype voice agents quickly with Gradium APIs
Gradbot is Gradium's open-source framework for prototyping voice agents. It gives you the event loop for low-latency speech interactions: streaming audio, VAD, turn-taking, interruptions, tool calls, and OpenAI-compatible LLM orchestration.
Explore the Gradbot overview and live demos.
Fork the open-source Gradbot repository.
Gradbot is built for fast prototyping, demos, and experimentation. For production scenarios, use Gradium APIs directly or through mature agent integrations such as [Pipecat](/integrations/agent-frameworks/pipecat).
## Install
```bash theme={null}
pip install gradbot
```
## Basic agent
Create a `SessionConfig`, pick a Gradium voice, and let Gradbot run the realtime voice loop.
```python theme={null}
import gradbot
voice = gradbot.flagship_voice("Emma")
config = gradbot.SessionConfig(
voice_id=voice.voice_id,
instructions="You are a helpful assistant.",
language=voice.language,
)
input_handle, output_handle = await gradbot.run(
session_config=config,
input_format=gradbot.AudioFormat.OggOpus,
output_format=gradbot.AudioFormat.OggOpus,
)
```
## When to use Gradbot
* **Voice agent prototypes**: get from idea to a working spoken demo quickly.
* **Interactive demos**: showcase Gradium voices, interruptions, and tool calls in a browser.
* **Internal experiments**: test prompts, workflows, and voice UX before hardening the stack.
* **Hackathons and templates**: start with a small app instead of wiring every realtime primitive yourself.
## Production path
Once the interaction pattern is validated, move production workloads to Gradium's lower-level APIs or production-oriented integrations. [Pipecat](/integrations/agent-frameworks/pipecat) is a good fit when you need a mature pipeline framework for transports, processors, service composition, deployment, and observability.
# LiveKit
Source: https://docs.gradium.ai/integrations/agent-frameworks/livekit
Use Gradium speech models in LiveKit Agents
LiveKit Agents is a framework for realtime voice AI over WebRTC, SIP, and streaming media. The Gradium plugin lets a LiveKit agent use Gradium for speech-to-text, text-to-speech, or both inside the same `AgentSession`.
LiveKit's guide to Gradium text-to-speech.
LiveKit's guide to Gradium speech-to-text.
Python API reference for `livekit.plugins.gradium`.
End-to-end Gradium and LiveKit voice agent walkthrough.
## Install
Install LiveKit Agents with the Gradium plugin extra:
```bash theme={null}
pip install "livekit-agents[gradium]~=1.5"
```
Set the required credentials in your environment:
```bash theme={null}
export LIVEKIT_URL=wss://your-project.livekit.cloud
export LIVEKIT_API_KEY=your-livekit-api-key
export LIVEKIT_API_SECRET=your-livekit-api-secret
export GRADIUM_API_KEY=gd_your_api_key_here
```
## Basic agent session
Use `gradium.STT()` to transcribe user audio and `gradium.TTS()` to synthesize agent replies.
```python theme={null}
from livekit.agents import AgentSession
from livekit.plugins import gradium
session = AgentSession(
stt=gradium.STT(language="en"),
tts=gradium.TTS(
voice_id="YTpq7expH9539ERJ",
),
# ... llm, vad, tools, and other session options
)
```
You can also use only one side of Gradium. For example, keep an existing STT provider and use Gradium just for speech output:
```python theme={null}
session = AgentSession(
tts=gradium.TTS(),
# ... llm, stt, etc.
)
```
## Configuration
Common Gradium options in LiveKit Agents:
| Setting | Applies to | Description |
| - | - | - |
| `api_key` | STT, TTS | Gradium API key. Defaults to `GRADIUM_API_KEY`. |
| `model_endpoint` | STT, TTS | Gradium WebSocket endpoint. Defaults to `api.gradium.ai`; requests are automatically routed to the nearest regional cluster. |
| `model_name` | STT, TTS | Gradium model name. Defaults to `default`. |
| `voice_id` | TTS | Gradium voice ID for synthesized replies. Defaults to Emma. |
| `pronunciation_id` | TTS | Optional Gradium pronunciation dictionary ID. |
| `language` | STT | Transcription language: `en`, `fr`, `es`, `pt`, or `de`. Always set it. |
| `json_config` | TTS | Advanced TTS settings such as speed and rewrite rules. |
| `vad_threshold` | STT | Semantic VAD threshold for detecting when the user has finished speaking. |
| `vad_bucket` | STT | Semantic VAD timing bucket. |
## Custom pronunciation
Pass a Gradium pronunciation dictionary ID into the TTS plugin when an agent needs brand names, product names, or domain-specific terms spoken consistently.
```python theme={null}
tts = gradium.TTS(
voice_id="YTpq7expH9539ERJ",
pronunciation_id="pronunciation_dictionary_id",
)
```
For lower-level pronunciation and rewrite behavior, see [Pronunciation Dictionaries](/guides/text-to-speech#pronunciation-dictionaries) and [Voice Settings](/guides/voice-settings).
## When to use LiveKit with Gradium
* **Realtime voice agents**: run a complete STT, LLM, and TTS loop in `AgentSession`.
* **WebRTC applications**: connect browser and mobile users to Gradium-powered speech through LiveKit rooms.
* **Telephony agents**: use LiveKit SIP while keeping Gradium as the speech model layer.
* **Production deployments**: deploy LiveKit agents with managed rooms, credentials, and worker processes.
# OpenClaw
Source: https://docs.gradium.ai/integrations/agent-frameworks/openclaw
Use Gradium text-to-speech in OpenClaw agents
OpenClaw is a self-hosted agent gateway for messaging channels, agent sessions, and tool-driven workflows. Its bundled Gradium speech provider lets OpenClaw agents turn replies into audio, voice-note-compatible Opus output, or 8 kHz u-law audio for telephony surfaces.
OpenClaw's provider reference for Gradium TTS.
View the OpenClaw Gradium extension source.
Gradium WebSocket TTS setup and streaming behavior.
## Setup
Create a Gradium API key, then expose it to OpenClaw with an environment variable:
```bash theme={null}
export GRADIUM_API_KEY=gd_your_api_key_here
```
OpenClaw can also read the key from `messages.tts.providers.gradium.apiKey` in `~/.openclaw/openclaw.json`.
## Basic config
Set `gradium` as the TTS provider and choose a Gradium voice ID.
```json theme={null}
{
"messages": {
"tts": {
"auto": "always",
"provider": "gradium",
"providers": {
"gradium": {
"voiceId": "YTpq7expH9539ERJ",
"apiKey": "${GRADIUM_API_KEY}"
}
}
}
}
}
```
Use `auto: "always"` when every eligible agent reply should include audio. Use OpenClaw's session commands or per-agent config when you need more targeted behavior.
## Provider settings
| Setting | Description |
| - | - |
| `messages.tts.provider` | Set to `gradium` to make Gradium the primary TTS provider. |
| `messages.tts.providers.gradium.apiKey` | Gradium API key. You can use `${GRADIUM_API_KEY}` or rely on the environment variable. |
| `messages.tts.providers.gradium.voiceId` | Default Gradium voice ID for replies. |
| `messages.tts.providers.gradium.baseUrl` | Optional Gradium API origin override. Defaults to `https://api.gradium.ai`. |
## Voices
OpenClaw's Gradium provider defaults to Emma.
| Voice | Voice ID |
| - | - |
| Emma | `YTpq7expH9539ERJ` |
| Kent | `LFZvm12tW_z0xfGo` |
| Tiffany | `Eu9iL_CYe8N-Gkx_` |
| Christina | `2H4HY2CBNyJHBCrP` |
| Sydney | `jtEKaLYNn6iif5PR` |
| John | `KWJiFWu2O9nMPYcR` |
| Arthur | `3jUdJyOi9pgbxBTK` |
## Output behavior
OpenClaw chooses the Gradium output format from the destination surface:
| Destination | Gradium output |
| - | - |
| Standard audio reply | WAV audio file |
| Voice-note channel | Opus voice-compatible audio |
| Telephony surface | `ulaw_8000` at 8 kHz |
## When to use OpenClaw with Gradium
* **Messaging-channel agents**: speak replies across OpenClaw-supported channels without building per-channel audio delivery.
* **Voice-note workflows**: send spoken responses as voice-compatible channel media.
* **Telephony surfaces**: synthesize 8 kHz u-law audio for phone-oriented integrations.
* **Self-hosted agent stacks**: keep OpenClaw's local gateway model while using Gradium for speech output.
# Pipecat
Source: https://docs.gradium.ai/integrations/agent-frameworks/pipecat
Use Gradium streaming text-to-speech in Pipecat voice agents
Pipecat is a framework for building real-time voice and multimodal agents. Its `GradiumTTSService` lets a Pipecat pipeline stream synthesized speech through Gradium's WebSocket API.
API reference for `GradiumTTSService`.
Complete Pipecat example using Gradium services.
Pipecat's multiplayer game, using Gradium for TTS by default.
Gradium WebSocket TTS setup and streaming behavior.
## Installation
Install Pipecat with the Gradium extra:
```bash theme={null}
uv add "pipecat-ai[gradium]"
```
Set your Gradium API key in the environment:
```bash theme={null}
export GRADIUM_API_KEY=gd_your_api_key_here
```
## Basic setup
Create a `GradiumTTSService` with your API key and a Gradium voice ID. Pipecat's current pattern is to pass runtime options through `GradiumTTSService.Settings(...)`.
```python theme={null}
import os
from pipecat.services.gradium.tts import GradiumTTSService
tts = GradiumTTSService(
api_key=os.environ["GRADIUM_API_KEY"],
settings=GradiumTTSService.Settings(
voice="YTpq7expH9539ERJ",
),
)
```
Then place the service in the audio pipeline after the LLM and before the transport output:
```python theme={null}
pipeline = Pipeline(
[
transport.input(),
stt,
user_aggregator,
llm,
tts,
transport.output(),
assistant_aggregator,
]
)
```
## Configuration
Common settings:
| Setting | Type | Description |
| - | - | - |
| `api_key` | `str` | Gradium API key. |
| `settings.voice` | `str` | Gradium voice ID to synthesize with. |
| `settings.model` | `str` | Optional Gradium model name. |
| `settings.language` | `Language \| str` | Optional synthesis language. |
| `url` | `str` | Optional Gradium WebSocket endpoint override. |
| `json_config` | `str` | Optional JSON string for additional model configuration. |
Gradium TTS in Pipecat outputs audio at 48kHz. Pipecat sets this sample rate automatically.
## Runtime behavior
* **Streaming audio**: Gradium emits audio over a WebSocket, which Pipecat forwards through the active transport.
* **Word timestamps**: Gradium timestamp events can support synchronized text display.
* **Voice switching**: Updating the `voice` setting at runtime reconnects the Gradium WebSocket with the new voice.
* **Connection events**: `on_connected`, `on_disconnected`, and `on_connection_error` are available for logging and health checks.
```python theme={null}
@tts.event_handler("on_connected")
async def on_connected(service):
print("Connected to Gradium")
```
## Example project
[Gradient Bang](https://github.com/pipecat-ai/gradient-bang) is Pipecat's multiplayer game and uses Gradium as the default TTS provider. Its bot environment includes `TTS_PROVIDER=gradium`, making it a useful larger reference for a real voice-agent application built on Pipecat.
# Vapi
Source: https://docs.gradium.ai/integrations/agent-frameworks/vapi
Use Gradium speech models in Vapi voice agents
Vapi is a platform for building and deploying voice agents, handling telephony, web transport, LLM calls, and turn-taking. Gradium is not a built-in Vapi provider, so you connect it through Vapi's custom endpoints: a `custom-voice` webhook for speech output and a `custom-transcriber` WebSocket for speech input. You run a small bridge service, and Vapi keeps managing the rest of the call.
Gradium WebSocket TTS setup and streaming behavior.
Gradium WebSocket STT, VAD, and flushing.
Tune semantic VAD horizons and thresholds.
Sign up for Vapi and create an assistant.
You need a [Vapi account](https://dashboard.vapi.ai) and a Gradium API key. This
page is the reference for the Gradium side of the integration: the endpoint
contracts, the bridge code, and the tuning options.
## How Gradium connects
| Vapi endpoint | Transport | What your bridge does |
| - | - | - |
| `custom-voice` | HTTP POST | Vapi posts the text to speak; you stream raw PCM back from `tts_realtime`. |
| `custom-transcriber` | WebSocket | Vapi streams call audio; you return transcripts from `stt_realtime`. |
You can use either side on its own. Keeping Vapi's built-in transcriber while using Gradium for speech output is a valid setup, and vice versa.
## Install
```bash theme={null}
pip install gradium
```
Set your Gradium API key in the environment:
```bash theme={null}
export GRADIUM_API_KEY=gd_your_api_key_here
```
## Text to speech
Vapi posts one request per sentence, carrying the text and the sample rate it expects:
```json theme={null}
{
"message": {
"type": "voice-request",
"text": "Your table is booked for seven o'clock.",
"sampleRate": 24000
}
}
```
Read `message.sampleRate` and pass it straight through as Gradium's `output_format`. Vapi expects raw 16-bit little-endian mono PCM at exactly the rate it asked for, streamed as it is produced.
```python theme={null}
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse
from gradium.client import GradiumClient
app = FastAPI()
client = GradiumClient(api_key=GRADIUM_API_KEY)
@app.post("/vapi/tts")
async def vapi_tts(request: Request) -> StreamingResponse:
message = (await request.json())["message"]
text, rate = message["text"], message["sampleRate"]
async def pcm():
async with client.tts_realtime(
voice_id="YTpq7expH9539ERJ",
output_format=f"pcm_{rate}",
) as tts:
await tts.send_text(text)
await tts.send_eos()
async for msg in tts:
if msg["type"] == "audio":
yield msg["audio"]
elif msg["type"] == "end_of_stream":
break
return StreamingResponse(pcm(), media_type="application/octet-stream")
```
Every sample rate Vapi requests has a matching Gradium PCM format, so no resampling is needed. See [Limits](/guides/limits) for the full format list.
Point the assistant's `voice` at the endpoint:
```json theme={null}
{
"voice": {
"provider": "custom-voice",
"server": { "url": "https://your-server.com/vapi/tts" }
}
}
```
## Speech to text
Vapi opens a WebSocket, sends a JSON `start` frame, then streams interleaved 16-bit PCM. When `channels` is `2`, channel 0 carries the caller and channel 1 carries the assistant — forward **only channel 0**, or the agent transcribes its own speech.
```python theme={null}
import asyncio, json
from fastapi import FastAPI, WebSocket
from gradium.client import GradiumClient
app = FastAPI()
client = GradiumClient(api_key=GRADIUM_API_KEY)
def caller_channel(pcm: bytes, channels: int) -> bytes:
"""Keep channel 0 from interleaved 16-bit PCM."""
if channels == 1:
return pcm
stride = channels * 2
return b"".join(pcm[i:i + 2] for i in range(0, len(pcm) - stride + 1, stride))
@app.websocket("/vapi/transcriber")
async def vapi_transcriber(ws: WebSocket) -> None:
await ws.accept()
start = json.loads(await ws.receive_text())
channels = int(start.get("channels", 2))
transcript, high_vad_steps = [], 0
async def send(text: str, transcript_type: str) -> None:
await ws.send_text(json.dumps({
"type": "transcriber-response",
"transcription": text,
"channel": "customer",
"transcriptType": transcript_type,
}))
async with client.stt_realtime(
input_format=f"pcm_{start['sampleRate']}",
json_config={"language": "en", "delay_in_frames": 16},
) as stt:
async def to_gradium() -> None:
while True:
pcm = await ws.receive_bytes()
await stt.send_audio(caller_channel(pcm, channels))
async def to_vapi() -> None:
nonlocal transcript, high_vad_steps
async for msg in stt:
if msg["type"] == "text":
transcript.append(msg["text"])
await send(" ".join(transcript).strip(), "partial")
elif msg["type"] == "step":
inactivity = msg["vad"][-1]["inactivity_prob"]
high_vad_steps = high_vad_steps + 1 if inactivity > 0.5 else 0
if high_vad_steps >= 3 and transcript:
await stt.send_flush(flush_id=1)
elif msg["type"] == "flushed":
await send(" ".join(transcript).strip(), "final")
transcript, high_vad_steps = [], 0
# Stop as soon as either side ends, so the Gradium session and the
# audio pump are torn down when the caller hangs up.
tasks = [asyncio.create_task(to_gradium()),
asyncio.create_task(to_vapi())]
try:
await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
finally:
for task in tasks:
task.cancel()
```
Send `partial` transcripts as well as `final` ones — Vapi needs to see speech while it is happening for barge-in to work. Treat `flushed` as the end of the turn rather than finalizing the moment the VAD threshold is crossed, because Gradium is still holding audio in its decoding window. See [Turn-Taking with Semantic VAD](/guides/recipes/turn-taking) for tuning the horizon and threshold.
Point the assistant's `transcriber` at the bridge over `wss`:
```json theme={null}
{
"transcriber": {
"provider": "custom-transcriber",
"server": { "url": "wss://your-server.com/vapi/transcriber" }
}
}
```
## Secure the endpoints
Both endpoints are public HTTP surfaces. Set a `secret` on the `server` object
in the assistant config, and Vapi sends it as an `X-Vapi-Secret` header on every
request. Reject anything that does not match:
```python theme={null}
if request.headers.get("x-vapi-secret") != VAPI_WEBHOOK_SECRET:
raise HTTPException(status_code=401, detail="bad secret")
```
## Configuration
| Setting | Applies to | Description |
| - | - | - |
| `api_key` | STT, TTS | Gradium API key. Defaults to `GRADIUM_API_KEY`. |
| `model_name` | STT, TTS | Gradium model name. Defaults to `default`. |
| `voice_id` | TTS | Gradium voice ID for synthesized replies. |
| `output_format` | TTS | Set to `pcm_` using Vapi's `message.sampleRate`. |
| `pronunciation_id` | TTS | Optional pronunciation dictionary ID. |
| `input_format` | STT | Set to `pcm_` using the `start` frame's `sampleRate`. |
| `json_config.language` | STT | Transcription language: `en`, `fr`, `es`, `pt`, `de`, or `any` to auto-detect. Always set it. |
| `json_config.delay_in_frames` | STT | Context before text is emitted. Each frame is 80 ms. |
| `json_config.keywords` | STT | Bias recognition toward names and domain vocabulary. |
## Reduce latency with multiplexing
Vapi posts one `custom-voice` request per sentence, so a fresh WebSocket per request pays connection setup several times per agent reply. Keep one connection open per sample rate and route concurrent requests over it with `close_ws_on_eos: false` and a per-request `client_req_id`.
Recycle pooled connections before the session limit, and fall back to a single-use connection if a pooled socket closes before it produces audio. See [Multiplexing](/guides/multiplexing) for the full contract.
## Let Gradium decide when the turn ends
By default Vapi runs its own endpointing on your partial transcripts, which can race Gradium's VAD and make the assistant reply before the flushed final arrives. Hand the decision to Gradium with a custom endpointing model:
```json theme={null}
{
"startSpeakingPlan": {
"waitSeconds": 0.25,
"smartEndpointingPlan": {
"provider": "custom-endpointing-model",
"server": { "url": "https://your-server.com/vapi/endpointing", "timeoutSeconds": 3 }
}
}
}
```
Vapi then posts a `call.endpointing.request` on every transcript update, and your server answers with how long to keep waiting based on the live Gradium session:
```python theme={null}
@app.post("/vapi/endpointing")
async def endpointing(body: dict) -> dict:
if body.get("message", {}).get("type") != "call.endpointing.request":
return {"ok": True}
state = gradium_turn_state() # your live STT session
if not state["active"]:
return {"timeoutSeconds": 1.0} # no session, keep the call moving
if state["seconds_since_final"] < 2.5:
return {"timeoutSeconds": 0.05} # Gradium flushed, the turn is over
if state["inactivity_prob"] < 0.5:
return {"timeoutSeconds": 5.0} # caller is audibly mid-sentence
return {"timeoutSeconds": 3.0} # a pause; the final should arrive
```
Clear your reference to the Gradium session when a call ends. If a finished session stays registered, the first endpointing requests of the next call are answered from stale turn state.
## Long calls
A single Gradium session lasts up to 300 seconds. Reconnect the Gradium session transparently when one ends while keeping Vapi's socket open, so calls of any length keep transcribing. See [Limits](/guides/limits).
## When to use Vapi with Gradium
* **Telephony agents**: let Vapi manage phone numbers, SIP, and call routing while Gradium handles speech.
* **Swapping speech providers**: keep an existing Vapi assistant, LLM, and tools, and change only the voice or transcriber layer.
* **Custom voices**: use an instantly cloned Gradium voice in a Vapi agent. See [Voices](/guides/voices/overview).
* **Domain vocabulary**: improve recognition of menu items, brand names, and product terms with [Keyword Boosting](/guides/recipes/keyword-boosting) and [Text Rewriting Rules](/guides/text-rewriting).
# Integrations
Source: https://docs.gradium.ai/integrations/index
Connect Gradium voice applications to tools, data sources, and agent workflows
Integrations help Gradium voice agents take action outside the conversation. Use them to connect spoken requests to search providers, application APIs, workflow tools, and agent frameworks.
## Agent Frameworks
Use Gradium inside agent frameworks that manage real-time transport, turn-taking, LLM calls, and audio pipelines.
Start with Gradbot for prototypes and demos, use LiveKit, Pipecat, or Vapi for production realtime voice agents, or connect messaging-channel agents through OpenClaw.
Prototype voice agents quickly with Gradium's open-source framework.
Use Gradium STT and TTS in LiveKit Agents.
Add Gradium TTS to OpenClaw messaging-channel agents.
Add Gradium streaming TTS to a Pipecat voice pipeline.
Connect Gradium STT and TTS to Vapi voice agents.
## Tools
Give voice agents fresh context and action surfaces through external tools.
### Websearch
Use websearch when a voice agent needs current web context before it speaks.
Answer spoken questions with Keenable realtime web search.
Book hotels and search the live web with Linkup.
Add real-time web search to a Gradium voice agent.
# Keenable
Source: https://docs.gradium.ai/integrations/web-search/keenable
Use Keenable realtime web search with Gradium voice AI agents
Keenable provides web search and content access built for AI agents. In a Gradium voice agent, Keenable acts as the live web layer: the user asks a question out loud, the agent turns it into a search query, Keenable returns current web results in realtime mode, and Gradium speaks the answer back while the sources stream to the screen.
Explore Keenable's search API, CLI, and MCP server.
See a Gradbot demo that answers spoken questions with live web search.
## Instant voice search
A voice search agent built on Gradium and Keenable follows this loop:
1. The user asks a spoken question, such as "What did the Fed announce today?"
2. Gradium Speech-to-Text transcribes the request in real time.
3. The agent calls a `web_search` tool with a focused query, silently and as its first action.
4. Keenable returns the top results with titles, sources, and snippets.
5. The agent answers in one to three short sentences, citing the source naturally.
6. Gradium Text-to-Speech streams the answer back to the user.
```
mic → Gradium STT → LLM (tool-capable) → Keenable realtime search → Gradium TTS → speaker
```
Because Keenable's realtime mode returns results in well under a second, the search fits inside a natural conversational pause. The caller hears an answer, not a progress report.
## Core wiring
The demo exposes search as a single voice-agent tool. Gradium handles the live STT and TTS session; the agent decides when to call `web_search`.
### Search tool definition
```python theme={null}
gradbot.ToolDef(
name="web_search",
description=(
"Search the live web for current or factual information. Use this "
"for any question about recent events, facts, prices, people, "
"products, news, or anything you are unsure about. Returns the top "
"results with title, source site, and a snippet."
),
parameters_json=json.dumps({
"type": "object",
"properties": {
"query": {
"type": "string",
"description": (
"The search query as a clear natural-language "
"description of what to find."
),
}
},
"required": ["query"],
}),
)
```
### Calling the Keenable Search API
When the model calls the tool, the handler runs a Keenable realtime search. The API takes a POST request with an `X-API-Key` header and returns a list of results.
```python theme={null}
import httpx
KEENABLE_API_URL = "https://api.keenable.ai/v1/search"
async def keenable_search(query: str, limit: int = 5) -> list[dict]:
payload = {"query": query, "mode": "realtime"}
headers = {
"X-API-Key": os.environ["KEENABLE_API_KEY"],
"Content-Type": "application/json",
}
async with httpx.AsyncClient() as client:
r = await client.post(
KEENABLE_API_URL, json=payload, headers=headers, timeout=30.0
)
r.raise_for_status()
data = r.json()
rows = []
for item in data.get("results") or []:
rows.append({
"title": item.get("title") or "Untitled",
"url": item.get("url"),
"snippet": item.get("snippet") or item.get("description") or "",
})
if len(rows) >= limit:
break
return rows
```
Each result includes a `title`, `url`, and `description`, plus an extended `snippet` and `published_at` timestamp when available. The request body also accepts optional filters such as `site` to restrict results to one domain, and `published_after` or `published_before` to bound results by publication date.
### Returning results to the agent
The tool handler sends a compact text summary back to the agent, along with instructions for how to speak the answer. Keeping the payload small keeps the LLM leg of the turn fast.
```python theme={null}
async def handle_tool_call(handle, input_handle, websocket):
if handle.name != "web_search":
await handle.send_error(f"Unknown tool: {handle.name}")
return
query = (handle.args.get("query") or "").strip()
results = await keenable_search(query, limit=5)
summary = "\n".join(
f"- {r['title']} ({r['url']}): {r['snippet'][:300]}" for r in results
)
await handle.send(json.dumps({
"success": True,
"query": query,
"results_summary": summary,
"message": (
"Answer the caller's question in 1-3 short spoken sentences "
"using these results. Do NOT search again for this."
),
}))
```
### Agent session config
The session config wires the tool into a live Gradium voice session.
```python theme={null}
def make_config(msg: dict) -> gradbot.SessionConfig:
return gradbot.SessionConfig(
voice_id="_6Aslh2DxfmnRLmP",
language=gradbot.LANGUAGES["en"],
instructions=SYSTEM_PROMPT,
tools=[WEB_SEARCH_TOOL],
assistant_speaks_first=True,
silence_timeout_s=0.0,
)
await gradbot.websocket.handle_session(
websocket,
config=cfg,
on_start=make_config,
on_tool_call=handle_tool_call,
)
```
## Keep the voice loop fast
Latency is what makes voice search feel magical or broken. The demo applies a few rules worth copying:
* **Search silently.** Prompt the agent to call `web_search` immediately, without filler like "let me look that up". The first spoken words should be the answer itself.
* **One search per question.** Instruct the agent to trust its first results instead of reformulating the query to double-check. A second search doubles the wait.
* **Cap and truncate results.** Five results with snippets trimmed to about 300 characters give the LLM enough to answer without inflating the prompt.
* **Warm up the search path.** Fire a throwaway Keenable query when the user starts a session, a few seconds before the first real question, so DNS and the search origin are already warm.
* **Handle failures in the agent's voice.** If a search fails, return a tool message telling the agent to say it could not reach the web right now, rather than letting the model guess.
## Authentication
Get an API key from the [Keenable console](https://keenable.ai/console). Keys are prefixed with `keen_` and are passed in the `X-API-Key` header. Set it alongside your Gradium key:
| Variable | Purpose |
| - | - |
| `KEENABLE_API_KEY` | Keenable search key (`keen_...`) |
| `GRADIUM_API_KEY` | Gradium voice (STT and TTS) |
## Why use Keenable with Gradium?
* **Answers inside a conversational pause**: realtime mode is fast enough that the caller hears the answer without dead air or spoken filler.
* **Built for agents**: results come back as clean structured JSON with titles, snippets, and timestamps, ready to summarize and speak.
* **Grounded spoken answers**: the agent answers from live web results instead of stale model memory, and can cite the source naturally.
* **Focused follow-ups**: callers can refine a question conversationally, and filters like `site` and `published_after` keep each new search sharp.
## Demo
The [voice web search demo](https://github.com/gradium-ai/gradbot/tree/main/demos/web_voice_search) shows the full pattern: a FastAPI app that runs a Gradbot session, exposes `web_search` as the agent's only tool, streams source cards to the browser as results arrive, and reports per-turn latency across STT, LLM, search, and TTS. Use it as a reference for building your own instant voice search experience.
# Linkup
Source: https://docs.gradium.ai/integrations/web-search/linkup
Use Linkup web search with Gradium voice AI agents
Linkup is a production-grade web search API for AI. In a Gradium voice agent, Linkup acts as the live web layer: the caller speaks a request, the agent turns it into a focused query, Linkup returns current web results with sources and images, and Gradium speaks the answer back while the sources stream to the screen.
Explore Linkup's search, fetch, and research APIs.
See a Gradbot demo that books hotels using live web search.
## Voice-controlled web search
A voice search agent built on Gradium and Linkup follows this loop:
1. The caller asks a spoken question, such as "Find me a boutique hotel in Lisbon under 150 euros a night."
2. Gradium Speech-to-Text transcribes the request in real time.
3. The agent calls a search tool with a focused query built from the caller's spoken preferences.
4. Linkup returns ranked results with titles, sources, snippets, and images.
5. The agent filters, compares, and summarizes the results.
6. Gradium Text-to-Speech streams the answer back to the caller.
This pattern lets callers control web search without typing. They can refine the search conversationally, ask for tradeoffs, compare sources, or narrow results by criteria like budget, location, style, or dates.
## Core wiring
The hotel demo exposes search as voice-agent tools. Gradium handles the live STT and TTS session; the agent decides when to call `search_hotels` and `get_hotel_details`.
### Search tool definition
```python theme={null}
gradbot.ToolDef(
name="search_hotels",
description=(
"Search the web for available hotels in a destination. Takes some "
"time to search. Keep chatting with the caller about the destination "
"while waiting for results! IMPORTANT: Always include the caller's "
"preferences (budget, style, etc.) in the preferences field so the "
"search is targeted."
),
parameters_json=json.dumps({
"type": "object",
"properties": {
"destination": {
"type": "string",
"description": "The city or destination to search hotels in",
},
"preferences": {
"type": "string",
"description": (
"Caller's requirements: budget range, style, amenities, "
"etc. (e.g. 'under 100 euros per night, boutique style')"
),
},
},
"required": ["destination"],
}),
)
```
### Calling the Linkup Search API
The demo uses the [`linkup-sdk`](https://pypi.org/project/linkup-sdk/) package. Because the client is synchronous, the handler runs each search in a thread pool so it does not block the voice session. `include_images=True` returns image results alongside the text sources.
```python theme={null}
import asyncio
from linkup import LinkupClient
linkup_client = LinkupClient(api_key=os.environ["LINKUP_API_KEY"])
async def do_search(query: str, include_images: bool = True) -> dict:
loop = asyncio.get_running_loop()
result = await loop.run_in_executor(
None,
lambda: linkup_client.search(
query=query,
depth="standard",
output_type="searchResults",
include_images=include_images,
),
)
sources, images = [], []
for item in result.results or []:
if item.type == "image":
images.append({"title": item.name, "url": item.url})
else:
sources.append({
"title": item.name,
"href": item.url,
"body": item.content[:300] if item.content else "",
})
answer = "\n".join(f"- {s['title']}: {s['body']}" for s in sources[:5])
return {"answer": answer, "sources": sources, "images": images}
```
Each text result carries a `name`, `url`, and `content`; image results carry a `name` and `url`. Two parameters shape the search:
* **`depth`** trades latency for thoroughness. `fast` is sub-second keyword search for latency-critical paths like voice, `standard` is single-iteration agentic search at roughly one to three seconds and suits most agent lookups, and `deep` runs up to ten iterations at five to thirty seconds for complex, multi-source investigations. The demo uses `standard`.
* **`output_type`** shapes the return value. `searchResults` returns ranked URLs and snippets for the agent to reason over, `sourcedAnswer` returns a generated answer with citations, and `structured` returns JSON matching a schema you provide. The demo uses `searchResults` so the agent can filter and summarize the sources itself.
### Returning results to the agent
The tool handler streams the sources and images to the browser, swaps in a prompt built from the search results, and sends a compact summary back to the agent along with instructions for how to speak the answer. Keeping the payload small keeps the LLM leg of the turn fast.
```python theme={null}
async def handle_search_hotels(destination, preferences, tool_handle, input_handle, websocket):
query = f"hotels in {destination} {preferences or ''} with star ratings, prices per night, and brief descriptions"
result = await do_search(query)
# Stream source cards and images to the frontend
await websocket.send_json({
"type": "search_results",
"query": f"Hotels in {destination}",
"results": result["sources"],
"images": result.get("images", []),
})
await tool_handle.send(json.dumps({
"success": True,
"destination": destination,
"answer": result["answer"],
"sources": result["sources"],
"message": (
"Hotel search results are ready. Present the top options based on "
"the search results, highlighting star ratings, price ranges, and "
"key features. Ask which one interests the caller."
),
}))
```
### Agent session config
The session config wires the tools into a live Gradium voice session.
```python theme={null}
def make_config(instructions: str) -> gradbot.SessionConfig:
return gradbot.SessionConfig(
voice_id="cLONiZ4hQ8VpQ4Sz",
instructions=instructions,
language=gradbot.Lang.En,
tools=build_tools(),
silence_timeout_s=0.0,
)
await gradbot.websocket.handle_session(
websocket,
on_start=on_start,
on_config=on_config,
on_tool_call=handle_tool_call,
)
```
## Keep the voice loop fast
Latency is what makes voice search feel magical or broken. The demo applies a few rules worth copying:
* **Search off the event loop.** The Linkup client is synchronous, so run it in a thread pool with `run_in_executor` to avoid blocking audio while a query is in flight.
* **Match depth to the turn.** The demo uses `standard` depth and covers the wait with conversation. For lookups that need to land inside a conversational pause, `depth="fast"` returns sub-second keyword results; reserve `deep` for research-style questions where a few extra seconds is acceptable.
* **Cover the wait with conversation.** Prompt the agent to keep chatting about the destination while the search runs, so the caller never hears dead air.
* **Cap and truncate results.** A handful of sources with snippets trimmed to about 300 characters give the LLM enough to answer without inflating the prompt.
* **Stream sources to the screen.** Send source cards and images to the browser as results arrive so the caller sees what the agent is talking about.
* **Handle failures in the agent's voice.** If a search fails, return an empty result and tell the agent to say it could not reach the web right now, rather than letting the model guess.
## Authentication
Get an API key from the [Linkup dashboard](https://app.linkup.so/). Pass it to the `LinkupClient` or set it as an environment variable alongside your Gradium key:
| Variable | Purpose |
| - | - |
| `LINKUP_API_KEY` | Linkup search key |
| `GRADIUM_API_KEY` | Gradium voice (STT and TTS) |
## Why use Linkup with Gradium?
* **Fresh answers in live voice sessions**: give Gradium agents access to current web results without making callers leave the conversation.
* **Search that follows spoken constraints**: turn natural-language preferences like budget, location, style, or dates into focused Linkup queries.
* **Sources and images together**: results come back as clean structured data with text sources and images, ready to summarize, speak, and stream to the screen.
* **Grounded spoken answers**: the agent answers from live web results instead of stale model memory, and can cite the source naturally.
## Demo
The [hotel booking voice agent](https://github.com/gradium-ai/gradbot/tree/main/demos/hotel) shows the full pattern: a FastAPI app that runs a Gradbot session, walks the caller through choosing a destination, hotel, and room, and grounds each step in live Linkup search. Use it as a reference for building your own instant voice search experience.
# Tavily
Source: https://docs.gradium.ai/integrations/web-search/tavily
Use Tavily web search with Gradium voice AI agents
Tavily gives AI applications access to real-time web search, extraction, research, and crawling. In a Gradium voice agent, Tavily acts as the web context layer: the user speaks a request, the agent turns it into a structured search, Tavily returns current web results, and Gradium speaks the answer back.
Explore Tavily's web search and research APIs.
See a Gradbot demo that uses voice AI to search for Paris rentals.
## Voice-controlled web search
A voice search agent usually follows this loop:
1. The user asks a spoken question, such as "Find two-bedroom rentals near Canal Saint-Martin under 2,500 euros."
2. Gradium Speech-to-Text transcribes the request in real time.
3. Your agent decides whether it needs fresh web context and calls Tavily with a focused query.
4. Tavily returns search results and source content for the agent to inspect.
5. The agent filters, compares, and summarizes the results.
6. Gradium Text-to-Speech streams the answer back to the user.
This pattern lets users control web search without typing. They can refine the search conversationally, ask for tradeoffs, compare sources, or narrow results by criteria like budget, neighborhood, date, availability, or ranking.
## Core wiring
The Gradbot demo exposes search as a voice-agent tool. Gradium handles the live STT and TTS session; the agent decides when to call `run_apartment_search`.
### Agent session config
```python theme={null}
def on_start(msg: dict) -> gradbot.SessionConfig:
return gradbot.SessionConfig(
voice_id="ubuXFxVQwVYnZQhy",
instructions=SYSTEM_PROMPT,
language=gradbot.LANGUAGES["en"],
tools=build_tools(),
silence_timeout_s=0.0,
**config.session_kwargs,
)
await gradbot.websocket.handle_session(
websocket,
config=config,
on_start=on_start,
on_tool_call=on_tool_call,
)
```
### Search tool call
```python theme={null}
gradbot.ToolDef(
name="run_apartment_search",
description=(
"Run a fresh apartment search and return top matches. "
"Call only when the user explicitly asks to search."
),
parameters_json=json.dumps({
"type": "object",
"properties": {
"max_results": {"type": "integer"},
"allow_unconfirmed_profile": {"type": "boolean"},
},
"required": [],
}),
)
```
When the model calls the tool, the voice handler routes it into the same search logic used by the REST API and sends a compact result back to the agent before it speaks.
```python theme={null}
async def on_tool_call(handle, input_handle, websocket):
if handle.name == "run_apartment_search":
result = await assistant_tools.run_apartment_search(
db,
user_id,
max_results=int((handle.args or {}).get("max_results") or 10),
allow_unconfirmed_profile=bool(
(handle.args or {}).get("allow_unconfirmed_profile")
),
)
await handle.send_json(_voice_summarize_search(result))
```
The shared search logic eventually calls Tavily with the focused query built from the user's confirmed profile.
```python theme={null}
response = await httpx.AsyncClient().post(
"https://api.tavily.com/search",
json={
"api_key": tavily_api_key,
"query": "location appartement Paris 2 chambres 2500 euros",
"search_depth": "basic",
"max_results": 5,
"include_raw_content": True,
},
timeout=20.0,
)
results = response.json()["results"]
```
## Why use Tavily with Gradium?
* **Fresh answers in live voice sessions**: give Gradium agents access to current web results without making users leave the conversation.
* **Search that follows spoken constraints**: turn natural-language preferences like budget, location, timing, availability, or source type into focused Tavily queries.
* **Hands-free research loops**: let users compare results, refine criteria, and ask follow-up questions while the agent gathers new context.
* **Support and status lookups**: fetch public docs, release notes, pricing pages, or status information so callers get answers from up-to-date sources.
## Demo
The [Paris rental voice agent](https://github.com/gradium-ai/gradbot/tree/main/demos/paris_rental_agent) shows how a Gradbot application can combine a spoken interface with web search. Use it as a reference for the agent loop: listen to the user, call a web search tool, reason over the results, and respond with speech.