> ## Documentation Index
> Fetch the complete documentation index at: https://developer.kallglot.com/llms.txt
> Use this file to discover all available pages before exploring further.

# WebSocket Streaming

> Real-time bidirectional audio streaming

# WebSocket Streaming

The Kallglot WebSocket API provides real-time bidirectional streaming for audio data, transcripts, and translations. Connect to the WebSocket URL provided when you create a session.

## Connection

Connect to the WebSocket URL from the session creation response:

```
wss://api.kallglot.com/v1/sessions/{session_id}/connect?token={stream_token}
```

### Authentication

Include the session token as a query parameter:

```javascript theme={null}
const ws = new WebSocket(`${session.stream.url}?token=${session.stream.token}`);
```

<Note>
  Stream tokens expire after 5 minutes and are validated only when the WebSocket connects.
</Note>

## Message Format

All messages are JSON-encoded. Each message has a `type` field that indicates the message type.

### Client → Server Messages

#### Audio Data

Send audio data to be processed:

```json theme={null}
{
  "type": "audio.input",
  "sequence": 1729481920,
  "timestamp_ms": 1729481920000,
  "speaker": "customer",
  "audio": {
    "encoding": "mulaw",
    "sample_rate_hz": 8000,
    "payload": "base64-encoded-audio-data"
  }
}
```

| Field                  | Type   | Description                               |
| ---------------------- | ------ | ----------------------------------------- |
| `type`                 | string | Always `audio.input`                      |
| `sequence`             | number | Sequence number to preserve order         |
| `timestamp_ms`         | number | Client timestamp in milliseconds          |
| `speaker`              | string | Identifies the speaker                    |
| `audio.encoding`       | string | Must be `mulaw`                           |
| `audio.sample_rate_hz` | number | Must be `8000`                            |
| `audio.payload`        | string | Base64-encoded mu-law audio (one channel) |

<Note>
  **API v1 currently accepts inbound stream audio only as mu-law (`encoding: mulaw`) at **8 kHz** mono.** Anything else responds with `session.error` and `invalid_audio_format`. Send chunks roughly 100–200 ms apart for predictable latency (typical telphony framing).
</Note>

#### Ping

```json theme={null}
{
  "type": "ping"
}
```

Server responds with `{ "type": "pong", "session_id": "sess_..." }`.

#### End session (WebSocket)

To end the session from the streaming client:

```json theme={null}
{
  "type": "session.end"
}
```

The server persists session end side effects and responds with `{ "type": "session.ended", "session_id": "...", "reason": "explicit_end" }`, then closes the receive loop.

<Note>
  Typical flow: **`audio.input`** to send microphone audio, **`ping`** occasionally, **`session.end`** when finished. Muting or pausing transcripts is **not** available as separate socket commands today.
</Note>

### Server → Client Messages

#### Transcript

Real-time transcription results:

```json theme={null}
{
  "type": "transcript.partial",
  "speaker": "customer",
  "language": "de",
  "text": "Ich habe eine "
}
```

```json theme={null}
{
  "type": "transcript.final",
  "sequence": 1,
  "speaker": "customer",
  "language": "de",
  "text": "Ich habe eine Frage",
  "confidence": 0.94,
  "timestamp": "2026-03-26T11:03:58.902000Z",
  "translation": {
    "language": "en",
    "text": "I have a question"
  }
}
```

| Field         | Type   | Description                                                                  |
| ------------- | ------ | ---------------------------------------------------------------------------- |
| `type`        | string | `transcript.partial` or `transcript.final`                                   |
| `speaker`     | string | Speaker identification (`agent`, `customer`, or channel ID)                  |
| `language`    | string | Detected language code                                                       |
| `text`        | string | Transcribed text                                                             |
| `sequence`    | number | Incrementing segment index (`transcript.final` only)                         |
| `confidence`  | number | Detection confidence when available (`transcript.final` only)                |
| `timestamp`   | string | Segment timestamp (ISO 8601; `transcript.final` only)                        |
| `translation` | object | Present on `transcript.final` when translation ran; `{ "language", "text" }` |

<Info>
  Interim results (`transcript.partial`) stream while audio is processed. Finals (`transcript.final`) finalize a segment for the session transcript once Kallglot has a stable recognition result (and include translation payload when configured).
</Info>

#### Audio Response

Translated speech audio (for bidirectional translation mode):

```json theme={null}
{
  "type": "audio.output",
  "sequence": 12,
  "timestamp_ms": 1729482055000,
  "speaker": "translated_agent",
  "audio": {
    "encoding": "mulaw",
    "sample_rate_hz": 8000,
    "payload": "base64-encoded-mulaw-bytes"
  }
}
```

#### Status Events

Session status updates:

```json theme={null}
{
  "type": "session.ready",
  "session_id": "sess_01HXYZ"
}
```

| Type            | Description                                    |
| --------------- | ---------------------------------------------- |
| `session.ready` | Session is fully active and provider connected |
| `session.ended` | Session has ended                              |

#### Session error events

```json theme={null}
{
  "type": "session.error",
  "session_id": "sess_01HXYZ123456789",
  "error": {
    "code": "invalid_audio_format",
    "message": "Only mulaw audio at 8000Hz is supported in API v1"
  }
}
```

| Field           | Description                                                                                                                                                                        |
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `type`          | Always `session.error`                                                                                                                                                             |
| `session_id`    | Session id                                                                                                                                                                         |
| `error.code`    | Machine-readable code (for example unknown message type uses `invalid_message`; oversize payloads use `message_too_large`; bad audio encoding uses codes from [`errors`](/errors)) |
| `error.message` | Human-readable explanation                                                                                                                                                         |

## Connection Example

```javascript theme={null}
import WebSocket from 'ws';

const session = await createSession();

const ws = new WebSocket(`${session.stream.url}?token=${session.stream.token}`);

ws.on('open', () => {
  console.log('Connected to Kallglot stream');
});

ws.on('message', (data) => {
  const message = JSON.parse(data);

  switch (message.type) {
    case 'transcript.partial':
      console.log(`[Partial] ${message.speaker}: ${message.text}`);
      break;

    case 'transcript.final':
      console.log(`[${message.speaker}] ${message.text}`);
      if (message.translation) {
        console.log(`  → ${message.translation.text}`);
      }
      break;

    case 'audio.output':
      // Play translated audio
      playAudio(Buffer.from(message.audio.payload, 'base64'));
      break;

    case 'session.ready':
      console.log('Session ready');
      break;

    case 'session.error':
      console.error(`Error [${message.error.code}]: ${message.error.message}`);
      break;

    case 'session.ended':
      console.log('Session ended:', message.reason);
      ws.close();
      break;
  }
});

ws.on('close', (code, reason) => {
  console.log(`Connection closed: ${code} - ${reason}`);
});

ws.on('error', (error) => {
  console.error('WebSocket error:', error);
});

// Send audio data
function sendAudio(audioBuffer) {
  ws.send(JSON.stringify({
    type: 'audio.input',
    sequence: Date.now(),
    timestamp_ms: Date.now(),
    speaker: 'customer',
    audio: {
      payload: audioBuffer.toString('base64'),
      encoding: 'mulaw',
      sample_rate_hz: 8000
    }
  }));
}

// End session from the client (mirrors REST POST /v1/sessions/:id/end)
function endSession() {
  ws.send(JSON.stringify({ type: 'session.end' }));
}
```

## Browser Example

```javascript theme={null}
// Create session (call your backend)
const session = await fetch('/api/sessions', {
  method: 'POST',
  body: JSON.stringify({ mode: 'bidirectional_translation' })
}).then(r => r.json());

// Connect to WebSocket
const ws = new WebSocket(`${session.stream.url}?token=${session.stream.token}`);

// Get microphone access
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
const audioContext = new AudioContext({ sampleRate: 16000 });
const source = audioContext.createMediaStreamSource(stream);
const processor = audioContext.createScriptProcessor(4096, 1, 1);

source.connect(processor);
processor.connect(audioContext.destination);

processor.onaudioprocess = (e) => {
  const inputData = e.inputBuffer.getChannelData(0);
  const pcm16 = new Int16Array(inputData.length);

  for (let i = 0; i < inputData.length; i++) {
    pcm16[i] = Math.max(-32768, Math.min(32767, inputData[i] * 32768));
  }

  const mulawBase64 = downsampleEncodeMulawFrames(pcm16);
  if (!mulawBase64 || ws.readyState !== WebSocket.OPEN) {
    return;
  }

  ws.send(JSON.stringify({
    type: 'audio.input',
    sequence: Date.now(),
    timestamp_ms: Date.now(),
    speaker: 'customer',
    audio: {
      payload: mulawBase64,
      encoding: 'mulaw',
      sample_rate_hz: 8000
    }
  }));
};
```

<Note>
  Provide `downsampleEncodeMulawFrames` (or reuse your PSTN codecs) so the browser emits **mono 8 kHz mu-law** Base64 payloads; PCM at 48 kHz is not accepted directly.
</Note>

## Connection Lifecycle

```mermaid theme={null}
sequenceDiagram
    participant Client
    participant Kallglot
    participant Provider

    Client->>Kallglot: POST /v1/sessions
    Kallglot-->>Client: Session + WebSocket URL

    Client->>Kallglot: WebSocket Connect
    Kallglot-->>Client: session.ready

    Kallglot->>Provider: Establish call
    Provider-->>Kallglot: Call connected

    loop Audio Streaming
        Client->>Kallglot: audio.input
        Kallglot-->>Client: transcript.partial
        Kallglot-->>Client: transcript.final (with translation)
        Kallglot-->>Client: audio.output
    end

    Client->>Kallglot: session.end
    Kallglot-->>Client: session.ended
    Kallglot->>Client: WebSocket Close
```

## Best Practices

<AccordionGroup>
  <Accordion title="Use appropriate audio chunk sizes">
    Send audio in 100-200ms chunks for optimal latency and transcription quality. Smaller chunks increase network overhead; larger chunks increase perceived latency.
  </Accordion>

  <Accordion title="Handle interim results correctly">
    Interim payloads use `type: transcript.partial`; treat them as non-final UI hints until `transcript.final` arrives with a `sequence`.
  </Accordion>

  <Accordion title="Implement reconnection logic">
    WebSocket connections may drop due to network issues. Implement exponential backoff reconnection and resume streaming.
  </Accordion>

  <Accordion title="Buffer audio during disconnection">
    If the connection drops briefly, buffer audio data and send it when reconnected to avoid gaps in transcription.
  </Accordion>
</AccordionGroup>

## Socket error payloads

These appear on `session.error` as `error.code` while the HTTP connection stays open:

| Code                   | When                                                           |
| ---------------------- | -------------------------------------------------------------- |
| `invalid_audio_format` | `audio.input` encoding or sample rate is not mu-law @ 8000 Hz  |
| `invalid_message`      | Payload is not valid JSON                                      |
| `message_too_large`    | Incoming WebSocket message exceeds the documented size ceiling |

Invalid or expired stream tokens and ended sessions normally fail **during connect** via WebSocket close codes (see [Error Codes](/errors#websocket-close-codes)).