Local ASR Sidecar Protocol

Mastermind local-first mode expects speech-to-text to run as an external local streaming service. The app connects to the service over WebSocket and sends 16 kHz mono PCM audio.

Default Endpoint

ws://127.0.0.1:8765/v1/asr/stream

The endpoint is configurable in the Local AI settings.

Client Start Message

After the WebSocket opens, the app sends a JSON start frame:

{
  "type": "start",
  "sampleRate": 16000,
  "channels": 1,
  "encoding": "pcm_s16le",
  "language": "en-US"
}

After that, the app sends binary frames containing raw little-endian signed 16-bit PCM audio at 16 kHz.

Sidecar Events

The sidecar should send JSON text frames:

{ "type": "ready" }

{ "type": "partial", "text": "intermediate transcript" }

{ "type": "final", "text": "final transcript" }

{ "type": "error", "error": "human-readable error" }

Only final transcript events enter local semantic reduction. partial events are transient status and must not be persisted.

v1 Scope

Required language modes: English (en-US), Russian (ru-RU), and automatic or mixed Russian-English recognition (auto or an implementation-equivalent mode).
Sidecar implementation is external to this repository.
The app does not require a specific Nemotron, NeMo, Riva, or ONNX runtime as long as the WebSocket protocol above is implemented.
The endpoint must be loopback-only. LAN and internet ASR endpoints are not Local Providers.

Native macOS Client Compatibility

The Swift-native app uses the same WebSocket protocol as the Electron client. Audio frames are raw 16 kHz mono signed 16-bit little-endian PCM.

Microphone and system audio use independent client connections so the sidecar does not mix channel roles. The sidecar does not need to know whether a client is Electron or Swift; Mastermind associates each connection with its Source.

Full transcripts are ephemeral sensitive input. The Swift client consumes a final event, derives structured local context, and then releases transcript content instead of storing it in Assistant History or the Context Graph.

2.1 KiB Raw Blame History