Media sources

Media sources provide APIs for adding media data to an output file. Mediabunny offers multiple source types at different abstraction levels, allowing you to choose the right balance between convenience and control for your use case.

Overview of media sources

Media sources can be organized into three abstraction levels:

High-level sources

CanvasSource, AudioBufferSource, MediaStreamVideoTrackSource, MediaStreamAudioTrackSourceEasy to use, handles encoding automatically.

Mid-level sources

VideoSampleSource, AudioSampleSourceWork with raw samples, handles encoding internally.

Low-level sources

EncodedVideoPacketSource, EncodedAudioPacketSourceDirect packet control, you handle encoding.

When to use each source

CanvasSource

Best for: Browser-based rendering, animations, games, data visualizations CanvasSource is ideal when you’re rendering to a canvas element and want to capture that rendering as video. This is the most common use case for browser-based video creation.

src/media-source.ts

const canvasSource = new CanvasSource(canvas, {
  codec: 'avc',
  bitrate: QUALITY_HIGH,
});

await canvasSource.add(0.0, 0.1); // Timestamp, duration
await canvasSource.add(0.1, 0.1);

Advantages:

Simplest API for canvas-based workflows
Automatically creates VideoFrames from canvas
Handles frame timing internally

Source code: src/media-source.ts:979-1028

VideoSampleSource

Best for: Direct VideoFrame manipulation, custom rendering pipelines, WebCodecs integration Use VideoSampleSource when you need fine-grained control over individual video frames or are working with VideoFrames from other sources.

src/media-source.ts

const sampleSource = new VideoSampleSource({
  codec: 'hevc',
  bitrate: 5e6,
});

const sample = new VideoSample(videoFrame, { timestamp: 0.0 });
await sampleSource.add(sample);
sample.close();

Advantages:

Direct access to VideoFrame API
Fine control over frame properties
Can accept frames from any source

Source code: src/media-source.ts:938-971

MediaStreamVideoTrackSource

Best for: Real-time capture (webcams, screen recording), live streaming to file MediaStreamVideoTrackSource automatically captures from a MediaStreamTrack in real-time, making it perfect for recording user media.

src/media-source.ts

const stream = await navigator.mediaDevices.getDisplayMedia({ video: true });
const videoTrack = stream.getVideoTracks()[0];

const source = new MediaStreamVideoTrackSource(videoTrack, {
  codec: 'vp9',
  bitrate: 1e7,
});

// Automatically starts capturing when output.start() is called
source.errorPromise.catch(error => console.error(error));

Advantages:

Automatic real-time capture
Built-in pause/resume support
Handles timestamp synchronization across multiple tracks

Important: Always handle errorPromise to catch asynchronous errors. Source code: src/media-source.ts:1039-1267

EncodedVideoPacketSource

Best for: Custom encoding pipelines, remuxing without re-encoding, WebCodecs manual control Use this source when you need complete control over the encoding process or want to bypass encoding entirely.

src/media-source.ts

const packetSource = new EncodedVideoPacketSource('av1');

// You handle encoding yourself
await packetSource.add(encodedPacket, {
  decoderConfig: {
    codec: 'av01.0.04M.08',
    codedWidth: 1920,
    codedHeight: 1080,
  },
});

Advantages:

Complete control over encoding
Can bypass encoding for remuxing
Direct access to packet stream

Requirements:

Must provide decoder config metadata
Must handle B-frames correctly (decode order vs presentation order)
Packets must be added in decode order

Source code: src/media-source.ts:172-202

Audio sources

AudioBufferSource

Best for: Web Audio API integration, audio processing workflows

src/media-source.ts

const bufferSource = new AudioBufferSource({
  codec: 'opus',
  bitrate: QUALITY_MEDIUM,
});

await bufferSource.add(audioBuffer1);
await bufferSource.add(audioBuffer2);

Advantages:

Direct AudioBuffer support
Automatic timestamp management
Perfect for Web Audio API workflows

Source code: src/media-source.ts:1833-1875

AudioSampleSource

Best for: Raw audio data, AudioData manipulation, custom audio processing

src/media-source.ts

const sampleSource = new AudioSampleSource({
  codec: 'aac',
  bitrate: 128e3,
});

await sampleSource.add(audioSample);

Advantages:

Fine-grained control over audio samples
Works with AudioData directly
Precise timestamp control

Source code: src/media-source.ts:1792-1825

MediaStreamAudioTrackSource

Best for: Microphone capture, live audio recording

src/media-source.ts

const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
const audioTrack = stream.getAudioTracks()[0];

const source = new MediaStreamAudioTrackSource(audioTrack, {
  codec: 'opus',
  bitrate: 128e3,
});

source.errorPromise.catch(error => console.error(error));

Advantages:

Automatic real-time capture
Synchronized with other MediaStream sources
Pause/resume support

Source code: src/media-source.ts:1886-2035

EncodedAudioPacketSource

Best for: Custom audio encoding, remuxing, direct packet control

src/media-source.ts

const packetSource = new EncodedAudioPacketSource('aac');

await packetSource.add(encodedPacket, {
  decoderConfig: {
    codec: 'mp4a.40.2',
    numberOfChannels: 2,
    sampleRate: 48000,
  },
});

Advantages:

Complete encoding control
Bypass encoding for remuxing
Direct packet access

Source code: src/media-source.ts:1297-1326

Advanced patterns

Handling backpressure

All media source add() methods return promises. Always await these to respect encoder and writer backpressure:

// Wrong - ignores backpressure
for (let i = 0; i < frames.length; i++) {
  canvasSource.add(i * frameDuration, frameDuration);
}

// Correct - respects backpressure
for (let i = 0; i < frames.length; i++) {
  await canvasSource.add(i * frameDuration, frameDuration);
}

Closing sources early

Close sources as soon as you’re done adding data to improve performance:

await videoSource.add(lastFrame);
videoSource.close(); // Signals no more data coming

// Output can now optimize buffering for other tracks

Managing video sample size changes

Control what happens when video frame dimensions change:

const source = new VideoSampleSource({
  codec: 'avc',
  bitrate: 1e6,
  sizeChangeBehavior: 'contain', // Options: 'deny', 'passThrough', 'fill', 'contain', 'cover'
});

Encoding alpha channels

Preserve transparency when encoding:

const source = new CanvasSource(canvas, {
  codec: 'vp9', // Use VP9 or other alpha-supporting codec
  bitrate: QUALITY_HIGH,
  alpha: 'keep', // Preserve alpha channel
});

Only certain codecs and containers support alpha channels. VP9 in WebM is the most common combination.

Custom key frame intervals

Control how frequently key frames are inserted:

const source = new VideoSampleSource({
  codec: 'hevc',
  bitrate: 5e6,
  keyFrameInterval: 2, // Key frame every 2 seconds (default is 5)
});

Shorter key frame intervals improve seeking but increase file size. When using multiple video tracks, use the same interval for all tracks.

Pausing MediaStream sources

Temporarily pause capture without stopping the underlying stream:

const videoSource = new MediaStreamVideoTrackSource(videoTrack, config);

// Later, pause capture
videoSource.pause();

// Resume capture (timestamps adjusted to maintain continuous playback)
videoSource.resume();

Best practices

Choose the right abstraction level

Start with high-level sources (CanvasSource, AudioBufferSource) unless you need the control of lower-level sources.

Always await add() calls

Respect backpressure by awaiting all add() method calls to prevent memory issues.

Close sources promptly

Call close() on sources as soon as you’re done adding data to improve performance.

Handle errorPromise for MediaStream sources

Always attach error handlers to errorPromise when using MediaStream sources.

Match codecs to containers

Ensure your chosen codec is supported by your output format (see supported formats).

Get Started

Core Concepts

Guides

Advanced

Extensions

Overview of media sources

High-level sources

Mid-level sources

Low-level sources

When to use each source

CanvasSource

VideoSampleSource

MediaStreamVideoTrackSource

EncodedVideoPacketSource

Audio sources

AudioBufferSource

AudioSampleSource

MediaStreamAudioTrackSource

EncodedAudioPacketSource

Advanced patterns

Handling backpressure

Closing sources early

Managing video sample size changes

Encoding alpha channels

Custom key frame intervals

Pausing MediaStream sources

Best practices

See also

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

Advanced

Extensions

Documentation Index

​Overview of media sources

High-level sources

Mid-level sources

Low-level sources

​When to use each source

​CanvasSource

​VideoSampleSource

​MediaStreamVideoTrackSource

​EncodedVideoPacketSource

​Audio sources

​AudioBufferSource

​AudioSampleSource

​MediaStreamAudioTrackSource

​EncodedAudioPacketSource

​Advanced patterns

​Handling backpressure

​Closing sources early

​Managing video sample size changes

​Encoding alpha channels

​Custom key frame intervals

​Pausing MediaStream sources

​Best practices

​See also

Build docs developers (and LLMs) love

Overview of media sources

When to use each source

CanvasSource

VideoSampleSource

MediaStreamVideoTrackSource

EncodedVideoPacketSource

Audio sources

AudioBufferSource

AudioSampleSource

MediaStreamAudioTrackSource

EncodedAudioPacketSource

Advanced patterns

Handling backpressure

Closing sources early

Managing video sample size changes

Encoding alpha channels

Custom key frame intervals

Pausing MediaStream sources

Best practices

See also