Get Ready for WebRTC Simulcast

[getroot]

WebRTC Simulcast Now Available

We are thrilled to announce the release of WebRTC Simulcast! This feature allows Adaptive Bitrate Streaming (ABR) without consuming additional Server resources, making it a highly efficient solution.

I encourage you to test this new feature and provide us with your feedback to help us refine and enhance it further.

How to Test Simulcast

You can test this functionality now using the OvenMediaEngine Master Branch and a WHIP Encoder. Since we plan to focus long-term on WHIP and gradually phase out our custom signaling protocol, Simulcast is only supported via WHIP.

A recommended WHIP encoder for testing is OBS. While the official OBS Studio version is expected to support WHIP+Simulcast soon, it hasn’t been merged into the Main branch yet. In the meantime, you can test it using the following pull request: obsproject/obs-studio#10885.

(OvenLiveKit updates will follow soon.)

---

Simulcast Playlists

When multiple input video Tracks exist, it means that several Tracks with the same Variant Name are present. For example, consider the following basic OutputProfile and assume there are three input video Tracks. In this case, three Tracks with the Variant Name video_bypass will be created:

<OutputProfile>
	<Name>stream</Name>
	<OutputStreamName>${OriginStreamName}</OutputStreamName>
	<Encodes>
		<Video>
			<Name>video_bypass</Name>
			<Bypass>true</Bypass>
		</Video>
	</Encodes>
</OutputProfile>

Structuring Playlists

How can we structure Playlists with multiple Tracks? A simple method introduces an Index concept in Playlists:

<Playlist>
	<Name>simulcast</Name>
	<FileName>template</FileName>
	<Options>
		<WebRtcAutoAbr>true</WebRtcAutoAbr>
		<HLSChunklistPathDepth>0</HLSChunklistPathDepth>
		<EnableTsPackaging>true</EnableTsPackaging>
	</Options>
	<Rendition>
		<Name>first</Name>
		<Video>video_bypass</Video>
		<VideoIndexHint>0</VideoIndexHint> <!-- Optional, default : 0 -->
		<Audio>aac_audio</Audio>
	</Rendition>
	<Rendition>
		<Name>second</Name>
		<Video>video_bypass</Video>
		<VideoIndexHint>1</VideoIndexHint> <!-- Optional, default : 0 -->
		<Audio>aac_audio</Audio>
		<AudioIndexHint>0</AudioIndexHint> <!-- Optional, default : 0 -->
	</Rendition>
</Playlist>

VideoIndexHint and AudioIndexHint specify the Index of input video and audio Tracks, respectively.

However, when using the above configuration, if the encoder broadcasts 3 video tracks with Simulcast, it is inconvenient to change the configuration and restart the server to provide HLS/WebRTC streaming with 3 ABR layers. So I implemented a dynamic Rendition tool called RenditionTemplate.

---

RenditionTemplate

The RenditionTemplate feature automatically generates Renditions based on specified conditions, eliminating the need to define each one manually. Here’s an example:

<Playlist>
	<Name>template</Name>
	<FileName>template</FileName>
	<Options>
		<WebRtcAutoAbr>true</WebRtcAutoAbr>
		<HLSChunklistPathDepth>0</HLSChunklistPathDepth>
		<EnableTsPackaging>true</EnableTsPackaging>
	</Options>
	<RenditionTemplate>
		<Name>hls_${Height}p</Name>
		<VideoTemplate>
			<EncodingType>bypassed</EncodingType>
		</VideoTemplate>
		<AudioTemplate>
			<VariantName>aac_audio</VariantName>
		</AudioTemplate>
	</RenditionTemplate>
</Playlist>

This configuration creates Renditions for all bypassed videos and uses audio Tracks with the aac_audio Variant Name.
The following macros can be used in the Name of a RenditionTemplate:
${Width} | ${Height} | ${Bitrate} | ${Framerate} | ${Samplerate} | ${Channel}

Additional Conditions

You can specify conditions to control Rendition creation. For example, to include only videos with a minimum resolution of 280p and bitrate above 500kbps, or to exclude videos exceeding 1080p or 2Mbps:

<VideoTemplate>
	<EncodingType>bypassed</EncodingType> <!-- all, bypassed, encoded -->
	<VariantName>bypass_video</VariantName>
	<VideoIndexHint>0</VideoIndexHint>
	<MaxWidth>1080</MaxWidth>
	<MinWidth>240</MinWidth>
	<MaxHeight>720</MaxHeight>
	<MinHeight>240</MinHeight>
	<MaxFPS>30</MaxFPS>
	<MinFPS>30</MinFPS>
	<MaxBitrate>2000000</MaxBitrate>
	<MinBitrate>500000</MinBitrate>
</VideoTemplate>
<AudioTemplate>
         <EncodingType>encoded</EncodingType> <!-- all, bypassed, encoded -->
	<VariantName>aac_audio</VariantName>
	<MaxBitrate>128000</MaxBitrate>
	<MinBitrate>128000</MinBitrate>
	<MaxSamplerate>48000</MaxSamplerate>
	<MinSamplerate>48000</MinSamplerate>
	<MaxChannel>2</MaxChannel>
	<MinChannel>2</MinChannel>
	<AudioIndexHint>0</AudioIndexHint>
</AudioTemplate>

This feature, while designed for Simulcast, can be applied to various use cases to optimize resource usage and streaming quality.

[getroot]

The following OutputProfile example is the configuration I used for testing. With this, WHIP Simulcast can be received and ABR can be provided to viewers over WebRTC, LLHLS, and HLSv3.

<OutputProfiles>
	<OutputProfile>
		<Name>stream</Name>
		<OutputStreamName>${OriginStreamName}</OutputStreamName>
		<Playlist>
			<Name>template</Name>
			<FileName>template</FileName>
			<Options>
				<WebRtcAutoAbr>true</WebRtcAutoAbr>
				<HLSChunklistPathDepth>0</HLSChunklistPathDepth>
				<EnableTsPackaging>true</EnableTsPackaging>
			</Options>
			<RenditionTemplate>
				<Name>hls_${Height}p</Name>
				<VideoTemplate>
					<EncodingType>bypassed</EncodingType>
				</VideoTemplate>
				<AudioTemplate>
					<VariantName>aac_audio</VariantName>
				</AudioTemplate>
			</RenditionTemplate>
			<RenditionTemplate>
				<Name>webrtc_${Height}p</Name>
				<VideoTemplate>
					<EncodingType>bypassed</EncodingType>
				</VideoTemplate>
				<AudioTemplate>
					<VariantName>bypass_audio</VariantName>
				</AudioTemplate>
			</RenditionTemplate>
		</Playlist>
		<Encodes>
			<Audio>
				<Name>aac_audio</Name>
				<Codec>aac</Codec>
				<Bitrate>128000</Bitrate>
				<Samplerate>48000</Samplerate>
				<Channel>2</Channel>
			</Audio>
			<Audio>
				<Name>bypass_audio</Name>
				<Bypass>true</Bypass>
			</Audio>
			<Video>
				<Name>video_bypass</Name>
				<Bypass>true</Bypass>
			</Video> 
		</Encodes>
	</OutputProfile>
</OutputProfiles>