Transport
This document specifies how video data is transported between components.
Overview
| Use Case | Protocol | Direction |
|---|---|---|
| Live streaming | WebSocket | Server → Proctor (push) |
| Live streaming | WebSocket | Sentinel → Server (push) |
| Historical playback | HTTP | Proctor → Server (pull) |
| Control messages | WebSocket | Bidirectional |
WebSocket: Live Streaming
All live video data flows over WebSocket connections.
Sentinel → Server
The Sentinel pushes fragments to the Server as they are finalized.
See Terminology for the distinction between fragments (live delivery units) and keyframes (join points).
Server → Proctor
The Server pushes fragments to subscribed Proctors as they arrive.
Data Format
The spec does not mandate a specific serialization format for WebSocket messages. Implementations may use:
- JSON with base64-encoded binary data
- Binary WebSocket frames with a header
- Protocol Buffers
- Other formats
The key requirement is that the message contains:
| Field | Type | Description |
|---|---|---|
sentinelId | string | Identifies which Sentinel this fragment belongs to |
sequence | integer | Fragment sequence number |
data | bytes | Raw fMP4 fragment bytes (moof + mdat) |
Additional metadata (framerate, timestamp) may be included or derived from the fragment bytes.
Initialization Segment
The initialization segment is sent separately, typically:
- Once when a Proctor joins a stream
- On request (see Control Messages)
| Field | Type | Description |
|---|---|---|
sentinelId | string | Identifies which Sentinel |
data | bytes | Raw fMP4 initialization segment bytes |
HTTP: Historical Playback
For accessing fragments that have aged out of the memory buffer, Proctors use HTTP.
Required Information
To fetch a historical fragment, the Proctor needs:
| Information | Description |
|---|---|
sentinelId | Which Sentinel’s stream |
sessionId | Which session |
sequence | Which fragment (or range) |
Response
The HTTP response contains the raw fMP4 fragment bytes with appropriate content headers.
| Header | Value |
|---|---|
Content-Type | video/mp4 |
Content-Length | Size in bytes |
Listing Available Fragments
Proctors may need to query which fragments are available for a session. The response should include:
| Field | Type | Description |
|---|---|---|
sentinelId | string | Sentinel identifier |
sessionId | string | Session identifier |
fragments | array | List of available fragment metadata |
Each entry:
| Field | Type | Description |
|---|---|---|
sequence | integer | Fragment number |
startTime | timestamp | When the fragment starts |
duration | integer | Duration in milliseconds |
Protocol Selection
When to Use WebSocket
| Scenario | Use WebSocket |
|---|---|
| Live stream viewing | Yes |
| Joining a stream | Yes (get init + recent fragments) |
| Control messages (keyframe request, FPS change) | Yes |
| Real-time fragment push from Sentinel | Yes |
When to Use HTTP
| Scenario | Use HTTP |
|---|---|
| Fetching fragments older than buffer window | Yes |
| Querying available sessions/fragments | Yes |
| Downloading for export/archival | Yes |
Connection Lifecycle
Sentinel Connection
- Sentinel establishes WebSocket connection to Server
- Sentinel sends registration/identification
- Sentinel begins pushing fragments
- Connection remains open for session duration
- On disconnect, session ends
Proctor Connection
- Proctor establishes WebSocket connection to Server
- Proctor sends registration/identification
- Proctor subscribes to one or more Sentinel streams
- Server pushes fragments for subscribed streams
- Proctor may switch subscriptions during session
- Connection remains open while Proctor is active
Bandwidth Considerations
Since Sentinel and Server are expected to be on the same LAN:
- Bandwidth is not a primary constraint
- Single-frame messages are acceptable
- No need for aggressive batching or compression beyond H.264
For Proctor connections (potentially over WAN):
- The memory buffer (15-20 seconds) provides catch-up capacity
- Proctors with slow connections may fall behind
- If behind by more than buffer window, Proctor must skip forward or use HTTP for historical catch-up