61 lines
1.3 KiB
Markdown
61 lines
1.3 KiB
Markdown
# Local ASR Sidecar Protocol
|
|
|
|
Mastermind local-first mode expects speech-to-text to run as an external local
|
|
streaming service. The app connects to the service over WebSocket and sends
|
|
16 kHz mono PCM audio.
|
|
|
|
## Default Endpoint
|
|
|
|
```text
|
|
ws://127.0.0.1:8765/v1/asr/stream
|
|
```
|
|
|
|
The endpoint is configurable in the Local AI settings.
|
|
|
|
## Client Start Message
|
|
|
|
After the WebSocket opens, the app sends a JSON start frame:
|
|
|
|
```json
|
|
{
|
|
"type": "start",
|
|
"sampleRate": 16000,
|
|
"channels": 1,
|
|
"encoding": "pcm_s16le",
|
|
"language": "en-US"
|
|
}
|
|
```
|
|
|
|
After that, the app sends binary frames containing raw little-endian signed
|
|
16-bit PCM audio at 16 kHz.
|
|
|
|
## Sidecar Events
|
|
|
|
The sidecar should send JSON text frames:
|
|
|
|
```json
|
|
{ "type": "ready" }
|
|
```
|
|
|
|
```json
|
|
{ "type": "partial", "text": "intermediate transcript" }
|
|
```
|
|
|
|
```json
|
|
{ "type": "final", "text": "final transcript" }
|
|
```
|
|
|
|
```json
|
|
{ "type": "error", "error": "human-readable error" }
|
|
```
|
|
|
|
Only `final` transcript events are sent to the local LLM. `partial` events are
|
|
shown as status text.
|
|
|
|
## v1 Scope
|
|
|
|
- STT target: English, `en-US`.
|
|
- Sidecar implementation is external to this repository.
|
|
- The app does not require a specific Nemotron, NeMo, Riva, or ONNX runtime as
|
|
long as the WebSocket protocol above is implemented.
|