Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/CeeblueTV/wrts-client/llms.txt

Use this file to discover all available pages before exploring further.

The RTS (Real-Time Streaming) container format is a custom binary format designed to deliver encoded media frames with the minimum possible overhead. Standard container formats such as ISOBMFF (MP4) carry significant per-segment metadata to support random access, seeking, and compatibility with archival playback. RTS discards everything except what is strictly necessary to reconstruct and time-align frames on the receiver side, making it well-suited to both the WebSocket transport path (WSSource) and the HTTP sequential-segment path (HTTPAdaptiveSource). The format is defined in src/media/reader/RTSReader.ts.

​
Design Goals

  • Minimal container overhead — each packet carries only a header and the raw encoded frame payload; there is no box nesting or codec-initialization metadata in the media stream itself
  • LEB128 variable-length encoding — all header fields except the payload itself are encoded in LEB128 (also called 7-bit encoding), which uses a single byte for values 0–127 and grows only for larger values; this keeps headers small for the short timestamps and durations that dominate in real-time media
  • Extensible packet types — four packet types cover all current needs (media frames, timed metadata, stream configuration, and control messages) while reserving space for future extension
  • Stateful timestamp inference — after the first frame of a track, subsequent timestamps are inferred from the previous frame’s duration rather than transmitted explicitly, saving bytes on every packet

​
Packet Types

Carries a single encoded audio or video frame. The type field in the header distinguishes audio (1) from video (2). Contains a timestamp (only on the first packet per track after an Init Tracks signal), duration, key-frame flag, optional composition offset for B-frames, and the raw encoded payload bytes.
Carries a timed non-media payload, typically a JSON object (for example, timed metadata such as subtitles or chapter markers). Uses trackId = 0 and type = 0 in the header. Includes a time delta (added to the stored next-time for the track), a duration, and then the payload.
Carries a JSON object with stream-level metadata. This is always the first packet received on a new stream and contains the protocol version, current live time, the tracks array (with IDs, codec, resolution, and bandwidth), and content protection information. Uses trackId = 0 and type = 0 — distinguished from a Data Packet by position (it always arrives first).
A control message that signals the (re)initialization of the audio and video track IDs. On receipt, the client clears all stored next-timestamps and expects an explicit time field in the next media packet for each track. Uses trackId = 0 and type = 3.

​
Header Format

Every RTS packet begins with a LEB128-encoded header field that packs both the track identity and packet type into a single compact number: 
header = (trackId + 1) << 2 | type

 MSB                                          LSB
 ┌──────────────────────────────┬──────────────┐
 │   trackId + 1  (LEB128 bits) │  type  (2b)  │
 └──────────────────────────────┴──────────────┘

 type field values:
   0 = Data or Metadata    (when trackId == 0)
   1 = Audio
   2 = Video
   3 = Init Tracks         (when trackId == 0)
The trackId + 1 encoding ensures that a raw header value of zero is never used for a media track — zero is reserved for command messages (Data, Metadata, and Init Tracks packets), which are identified by trackId == 0 after decoding.

​
Media Packet Structure

Media packets carry the core audio and video frames: 
[header] [time?] [duration_and_flags] [compositionOffset?] [payload...]
FieldTypeNotes
headerLEB128(trackId+1) << 2 | type; type = 1 for audio, 2 for video
timeLEB128Present only for the first packet of a track after an Init Tracks signal; subsequent times are inferred as previousTime + previousDuration
duration_and_flagsLEB128duration << 2 | hasCompositionOffset << 1 | isKeyFrame
compositionOffsetLEB128 (signed 16-bit)Present only when hasCompositionOffset flag is set; used for B-frames where decode order differs from presentation order
payloadbytesRaw encoded frame bytes (e.g., H.264 NAL units or AAC audio frame)
The duration is extracted as value >> 2 in milliseconds. The isKeyFrame flag is value & 1. The hasCompositionOffset flag is value & 2.

​
Data Packet Structure

Data packets carry generic timed payloads such as JSON metadata events: 
[header] [time_delta] [duration] [payload...]
FieldTypeNotes
headerLEB128(trackId+1) << 2 | 0; type = 0, trackId identifies the data track
time_deltaLEB128Value added to the stored next-time for this track to produce the absolute timestamp (time = time_delta + nextTime)
durationLEB128Duration of the data event in milliseconds
payloadbytesJSON-encoded object

​
Metadata Packet Structure

The Metadata Packet is always the first packet in a stream. It establishes the stream configuration before any media packets are processed: 
[header] [payload...]
FieldTypeNotes
headerLEB1280 << 2 | 0 → encoded value 0; trackId = 0, type = 0
payloadbytesJSON object containing version, currentTime, tracks array, sequence, and optionally contentProtection
The Metadata Packet and Data Packet share the same trackId = 0, type = 0 header encoding. The client distinguishes between them by context: the Metadata Packet is always the first message received; subsequent trackId = 0, type = 0 packets are treated as Data Packets.

​
Init Tracks Packet Structure

The Init Tracks Packet resets the timestamp state and declares which track IDs are currently active for audio and video: 
[header] [audioTrackId+1] [videoTrackId+1]
FieldTypeNotes
headerLEB1280 << 2 | 3 → encoded value 3; trackId = 0, type = 3
audioTrackId+1LEB128Active audio track ID incremented by 1; a value of 0 means no audio track
videoTrackId+1LEB128Active video track ID incremented by 1; a value of 0 means no video track
On receipt, the client clears all stored per-track next-timestamps (_nextTimes). The next Media Packet for each track must include an explicit time field rather than inferring it from the previous duration.

​
LEB128 Encoding

All header fields (except the raw frame payload) use LEB128 variable-length encoding, also referred to as 7-bit encoding:
  • Each byte contributes 7 bits of value
  • The most-significant bit of each byte is a continuation flag: 1 means another byte follows, 0 means this is the last byte
  • Values 0–127 encode in a single byte; values 128–16383 encode in two bytes; and so on
In media streams, timestamps and durations are typically small numbers (milliseconds, usually under 10,000). LEB128 keeps these fields at one or two bytes each, while retaining the ability to represent arbitrarily large values without a fixed-width field.
LEB128 examples
Value 42   → 0x2A             (1 byte,  fits in 7 bits)
Value 300  → 0xAC 0x02        (2 bytes, 300 = 0b10_0101100)
Value 2000 → 0xD0 0x0F        (2 bytes, 2000 = 0b1111_1010000)

​
Relation to CMAF / MP4

The HTTPAdaptiveSource supports two container formats, selected by the mediaExt of the endpoint URL:
ExtensionReaderUse case
.rtsRTSReaderWebRTS native format; minimal overhead; used by default
.mp4 / .cmafCMAFReaderCMAF segments from ISOBMFF-compatible origin; passthrough to MSE
Both RTSReader and CMAFReader extend the abstract Reader base class and emit the same onSample, onMetadata, and onInitTracks events, making them interchangeable from the Source’s perspective.
Backward compatibility: If the stream’s protocolVersion.major is <= 1, the client automatically selects RTSReaderOld instead of RTSReader. The older format uses a fixed uint8 header-size prefix rather than the inline LEB128 packet framing of the current format. RTSReaderOld is selected transparently — no application-level changes are required.
// From Source.ts — newReader()
if (protocolVersion && protocolVersion.major <= 1) {
    reader = new RTSReaderOld({ withSize: params.isStream });
} else {
    reader = new RTSReader({ withSize: params.isStream });
}
For the full API reference for RTSReader, RTSReaderOld, and CMAFReader, see the Media Types API reference.

Build docs developers (and LLMs) love

Get started for freeTalk to us