MediaEngine

Overview

MediaEngine defines the codecs supported by a PeerConnection, and the configuration of those codecs. It manages codec registration, negotiation, and header extension configuration.

Type Definition

type MediaEngine struct {
    negotiatedVideo      bool
    negotiatedAudio      bool
    negotiateMultiCodecs bool
    videoCodecs          []RTPCodecParameters
    audioCodecs          []RTPCodecParameters
    negotiatedVideoCodecs []RTPCodecParameters
    negotiatedAudioCodecs []RTPCodecParameters
    headerExtensions      []mediaEngineHeaderExtension
    negotiatedHeaderExtensions map[int]mediaEngineHeaderExtension
    mu sync.RWMutex
}

Methods

RegisterDefaultCodecs

Registers the default codecs supported by Pion WebRTC.

func (m *MediaEngine) RegisterDefaultCodecs() error

error

Returns error if codec registration fails

Show Default Audio Codecs

Opus (48kHz, 2 channels) - PayloadType 111
G.722 (8kHz) - PayloadType 9
PCMU (8kHz) - PayloadType 0
PCMA (8kHz) - PayloadType 8

Show Default Video Codecs

VP8 - PayloadType 96 (with RTX at 97)
VP9 Profile 0 - PayloadType 98 (with RTX at 99)
VP9 Profile 2 - PayloadType 100 (with RTX at 101)
H.264 Multiple profiles:
- Baseline (profile-level-id=42001f) - PayloadTypes 102, 104
- Constrained Baseline (profile-level-id=42e01f) - PayloadTypes 106, 108
- Main (profile-level-id=4d001f) - PayloadTypes 127, 39
- High (profile-level-id=64001f) - PayloadType 112
H.265 - PayloadType 116 (with RTX at 117)
AV1 - PayloadType 45 (with RTX at 46)

All video codecs include RTCP feedback: goog-remb, ccm fir, nack, nack pli

m := &webrtc.MediaEngine{}
if err := m.RegisterDefaultCodecs(); err != nil {
    panic(err)
}

api := webrtc.NewAPI(webrtc.WithMediaEngine(m))
peerConnection, err := api.NewPeerConnection(webrtc.Configuration{})

RegisterCodec

Adds a codec to the MediaEngine. These are the list of codecs supported by this PeerConnection.

func (m *MediaEngine) RegisterCodec(codec RTPCodecParameters, typ RTPCodecType) error

codec

RTPCodecParameters

required

The codec parameters to register

typ

RTPCodecType

required

The codec type (RTPCodecTypeAudio or RTPCodecTypeVideo)

error

Returns error if codec with same payload type already registered or invalid type

m := &webrtc.MediaEngine{}

// Register only Opus for audio
err := m.RegisterCodec(webrtc.RTPCodecParameters{
    RTPCodecCapability: webrtc.RTPCodecCapability{
        MimeType:     webrtc.MimeTypeOpus,
        ClockRate:    48000,
        Channels:     2,
        SDPFmtpLine:  "minptime=10;useinbandfec=1",
    },
    PayloadType: 111,
}, webrtc.RTPCodecTypeAudio)
if err != nil {
    panic(err)
}

// Register only VP8 for video
err = m.RegisterCodec(webrtc.RTPCodecParameters{
    RTPCodecCapability: webrtc.RTPCodecCapability{
        MimeType:  webrtc.MimeTypeVP8,
        ClockRate: 90000,
        RTCPFeedback: []webrtc.RTCPFeedback{
            {Type: "nack"},
            {Type: "nack", Parameter: "pli"},
        },
    },
    PayloadType: 96,
}, webrtc.RTPCodecTypeVideo)

RegisterHeaderExtension

Adds a header extension to the MediaEngine. To determine the negotiated value use GetHeaderExtensionID after signaling is complete.

func (m *MediaEngine) RegisterHeaderExtension(
    extension RTPHeaderExtensionCapability,
    typ RTPCodecType,
    allowedDirections ...RTPTransceiverDirection,
) error

extension

RTPHeaderExtensionCapability

required

The header extension capability to register

typ

RTPCodecType

required

The codec type (audio or video)

allowedDirections

...RTPTransceiverDirection

Optional list of allowed transceiver directions. Defaults to sendonly and recvonly.

error

Returns error if direction is invalid (must be sendonly or recvonly)

m := &webrtc.MediaEngine{}
m.RegisterDefaultCodecs()

// Register audio level extension for audio
err := m.RegisterHeaderExtension(
    webrtc.RTPHeaderExtensionCapability{URI: webrtc.AudioLevelExtensionURI},
    webrtc.RTPCodecTypeAudio,
)

// Register transport-cc for video (both directions)
err = m.RegisterHeaderExtension(
    webrtc.RTPHeaderExtensionCapability{URI: webrtc.TransportCCExtensionURI},
    webrtc.RTPCodecTypeVideo,
    webrtc.RTPTransceiverDirectionSendonly,
    webrtc.RTPTransceiverDirectionRecvonly,
)

RegisterFeedback

Adds feedback mechanism to already registered codecs.

func (m *MediaEngine) RegisterFeedback(feedback RTCPFeedback, typ RTPCodecType)

feedback

RTCPFeedback

required

The RTCP feedback to add

typ

RTPCodecType

required

The codec type to add feedback to

m := &webrtc.MediaEngine{}
m.RegisterDefaultCodecs()

// Add REMB feedback to all video codecs
m.RegisterFeedback(
    webrtc.RTCPFeedback{Type: "goog-remb"},
    webrtc.RTPCodecTypeVideo,
)

// Add transport-cc feedback
m.RegisterFeedback(
    webrtc.RTCPFeedback{Type: "transport-cc"},
    webrtc.RTPCodecTypeVideo,
)

Usage Examples

Custom Codec Configuration

// Create MediaEngine with only specific codecs
m := &webrtc.MediaEngine{}

// Audio: Only Opus
m.RegisterCodec(webrtc.RTPCodecParameters{
    RTPCodecCapability: webrtc.RTPCodecCapability{
        MimeType:     webrtc.MimeTypeOpus,
        ClockRate:    48000,
        Channels:     2,
        SDPFmtpLine:  "minptime=10;useinbandfec=1",
    },
    PayloadType: 111,
}, webrtc.RTPCodecTypeAudio)

// Video: Only H.264 Baseline
m.RegisterCodec(webrtc.RTPCodecParameters{
    RTPCodecCapability: webrtc.RTPCodecCapability{
        MimeType:     webrtc.MimeTypeH264,
        ClockRate:    90000,
        SDPFmtpLine:  "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f",
        RTCPFeedback: []webrtc.RTCPFeedback{
            {Type: "nack"},
            {Type: "nack", Parameter: "pli"},
            {Type: "ccm", Parameter: "fir"},
        },
    },
    PayloadType: 102,
}, webrtc.RTPCodecTypeVideo)

// Create API with custom MediaEngine
api := webrtc.NewAPI(webrtc.WithMediaEngine(m))
peerConnection, _ := api.NewPeerConnection(webrtc.Configuration{})

VP8 Only Configuration

m := &webrtc.MediaEngine{}

// Register VP8 only
if err := m.RegisterCodec(webrtc.RTPCodecParameters{
    RTPCodecCapability: webrtc.RTPCodecCapability{
        MimeType:  webrtc.MimeTypeVP8,
        ClockRate: 90000,
        RTCPFeedback: []webrtc.RTCPFeedback{
            {Type: "goog-remb"},
            {Type: "ccm", Parameter: "fir"},
            {Type: "nack"},
            {Type: "nack", Parameter: "pli"},
        },
    },
    PayloadType: 96,
}, webrtc.RTPCodecTypeVideo); err != nil {
    panic(err)
}

// Register RTX for VP8
if err := m.RegisterCodec(webrtc.RTPCodecParameters{
    RTPCodecCapability: webrtc.RTPCodecCapability{
        MimeType:    webrtc.MimeTypeRTX,
        ClockRate:   90000,
        SDPFmtpLine: "apt=96",
    },
    PayloadType: 97,
}, webrtc.RTPCodecTypeVideo); err != nil {
    panic(err)
}

// Register Opus for audio
if err := m.RegisterCodec(webrtc.RTPCodecParameters{
    RTPCodecCapability: webrtc.RTPCodecCapability{
        MimeType:  webrtc.MimeTypeOpus,
        ClockRate: 48000,
        Channels:  2,
    },
    PayloadType: 111,
}, webrtc.RTPCodecTypeAudio); err != nil {
    panic(err)
}

api := webrtc.NewAPI(webrtc.WithMediaEngine(m))

Registering Header Extensions

m := &webrtc.MediaEngine{}
m.RegisterDefaultCodecs()

// Common header extensions
extensions := []struct {
    uri  string
    typ  webrtc.RTPCodecType
}{
    {webrtc.SDESMidURI, webrtc.RTPCodecTypeVideo},
    {webrtc.SDESRTPStreamIDURI, webrtc.RTPCodecTypeVideo},
    {webrtc.TransportCCExtensionURI, webrtc.RTPCodecTypeVideo},
    {webrtc.AudioLevelExtensionURI, webrtc.RTPCodecTypeAudio},
}

for _, ext := range extensions {
    if err := m.RegisterHeaderExtension(
        webrtc.RTPHeaderExtensionCapability{URI: ext.uri},
        ext.typ,
    ); err != nil {
        panic(err)
    }
}

api := webrtc.NewAPI(webrtc.WithMediaEngine(m))

Multiple Codec Negotiation

m := &webrtc.MediaEngine{}

// Enable multi-codec negotiation (allows codec switching during session)
m.setMultiCodecNegotiation(true)

// Register multiple video codecs
videoCodecs := []struct {
    mime        string
    payloadType webrtc.PayloadType
    fmtp        string
}{
    {webrtc.MimeTypeVP8, 96, ""},
    {webrtc.MimeTypeVP9, 98, "profile-id=0"},
    {webrtc.MimeTypeH264, 102, "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f"},
}

for _, codec := range videoCodecs {
    m.RegisterCodec(webrtc.RTPCodecParameters{
        RTPCodecCapability: webrtc.RTPCodecCapability{
            MimeType:    codec.mime,
            ClockRate:   90000,
            SDPFmtpLine: codec.fmtp,
        },
        PayloadType: codec.payloadType,
    }, webrtc.RTPCodecTypeVideo)
}

Audio-Only Configuration

m := &webrtc.MediaEngine{}

// Register only audio codecs
audioCodecs := []struct {
    mime        string
    clockRate   uint32
    channels    uint16
    payloadType webrtc.PayloadType
}{
    {webrtc.MimeTypeOpus, 48000, 2, 111},
    {webrtc.MimeTypePCMU, 8000, 0, 0},
    {webrtc.MimeTypePCMA, 8000, 0, 8},
}

for _, codec := range audioCodecs {
    m.RegisterCodec(webrtc.RTPCodecParameters{
        RTPCodecCapability: webrtc.RTPCodecCapability{
            MimeType:  codec.mime,
            ClockRate: codec.clockRate,
            Channels:  codec.channels,
        },
        PayloadType: codec.payloadType,
    }, webrtc.RTPCodecTypeAudio)
}

api := webrtc.NewAPI(webrtc.WithMediaEngine(m))
peerConnection, _ := api.NewPeerConnection(webrtc.Configuration{})

Common Header Extension URIs

Show Video Extensions

urn:ietf:params:rtp-hdrext:sdes:mid - Media ID
urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id - RTP Stream ID
urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id - Repaired Stream ID
http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time - Absolute Send Time
http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 - Transport-wide CC
http://www.webrtc.org/experiments/rtp-hdrext/playout-delay - Playout Delay
urn:3gpp:video-orientation - Video Orientation

Show Audio Extensions

urn:ietf:params:rtp-hdrext:ssrc-audio-level - Audio Level
urn:ietf:params:rtp-hdrext:sdes:mid - Media ID
http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 - Transport-wide CC

Core API

Media

Data Channels

Transport Layer

Statistics

Overview

Type Definition

Methods

RegisterDefaultCodecs

RegisterCodec

RegisterHeaderExtension

RegisterFeedback

Usage Examples

Custom Codec Configuration

VP8 Only Configuration

Registering Header Extensions

Multiple Codec Negotiation

Audio-Only Configuration

Common Header Extension URIs

See Also

Build docs developers (and LLMs) love

Core API

Media

Data Channels

Transport Layer

Statistics

​Overview

​Type Definition

​Methods

​RegisterDefaultCodecs

​RegisterCodec

​RegisterHeaderExtension

​RegisterFeedback

​Usage Examples

​Custom Codec Configuration

​VP8 Only Configuration

​Registering Header Extensions

​Multiple Codec Negotiation

​Audio-Only Configuration

​Common Header Extension URIs

​See Also

Build docs developers (and LLMs) love

Overview

Type Definition

Methods

RegisterDefaultCodecs

RegisterCodec

RegisterHeaderExtension

RegisterFeedback

Usage Examples

Custom Codec Configuration

VP8 Only Configuration

Registering Header Extensions

Multiple Codec Negotiation

Audio-Only Configuration

Common Header Extension URIs

See Also