Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switching WebRTC streams #89

Closed
IanMitchell77 opened this issue Mar 21, 2020 · 21 comments
Closed

Switching WebRTC streams #89

IanMitchell77 opened this issue Mar 21, 2020 · 21 comments

Comments

@IanMitchell77
Copy link

Hi,

How do you have different streams at different bitrates then have the player switch between them seamlessly?

I have configured my server to encode two streams at different bitrates, in the frontend I use player.setCurrentSource to switch the streams, autoStart is true(when I create the player) and its taking several seconds to start playing av.

Best Regards,

Ian

@getroot
Copy link
Member

getroot commented Mar 24, 2020

I think it's a player issue.
Does it take several seconds even when you first try to play the stream?

@IanMitchell77
Copy link
Author

Yes, it does take several seconds the first time.

What is the recommended server configuration and player configuration for this?

@getroot
Copy link
Member

getroot commented Mar 24, 2020

I need details to analyze the problem.

Do you use OME at dev branch?
What is your configuration(server.xml)?
What do you use browser to play WebRTC?
Give me your ovenmediaengine.log file when you got the problems.
What is your encoder? If it is obs, please give me the detailed configuration capture like this.
image

@IanMitchell77
Copy link
Author

IanMitchell77 commented Mar 24, 2020

Server.xml

OvenMediaEngine default * 9000 1935 81 81 *:10000-10005/udp 3333
		<Applications>
			<Application>
				<Name>app</Name>
				<Type>live</Type>
				<Encodes>
					<!--<Encode>
						<Name>VP8</Name>
						<Audio>
							<Codec>opus</Codec>
							<Bitrate>128000</Bitrate>
							<Samplerate>48000</Samplerate>
							<Channel>2</Channel>
						</Audio>
						<Video>
							<Codec>vp8</Codec>
							<Width>640</Width>
							<Height>360</Height>
							<Bitrate>500000</Bitrate>
							<Framerate>30.0</Framerate>
						</Video>
					</Encode>
					<Encode>
						<Name>H264</Name>
						<Audio>
							<Codec>aac</Codec>
							<Bitrate>128000</Bitrate>
							<Samplerate>48000</Samplerate>
							<Channel>2</Channel>
						</Audio>
						<Video>
							<Codec>h264</Codec>
							<Width>640</Width>
							<Height>360</Height>
							<Bitrate>500000</Bitrate>
							<Framerate>30.0</Framerate>
						</Video>
					</Encode>-->
					<Encode>
                                                    <Name>VP8_low</Name>
                                                    <Audio>
                                                            <Codec>opus</Codec>
                                                            <Bitrate>48000</Bitrate>
                                                            <Samplerate>48000</Samplerate>
                                                            <Channel>2</Channel>
                                                    </Audio>
                                                    <Video>
                                                            <Codec>vp8</Codec>
                                                            <Width>640</Width>
                                                            <Height>360</Height>
                                                            <Bitrate>350000</Bitrate>
                                                            <Framerate>30.0</Framerate>
                                                    </Video>
                                            </Encode>
                                            <Encode>
                                                    <Name>H264_low</Name>
                                                    <Audio>
                                                            <Codec>aac</Codec>
                                                            <Bitrate>48000</Bitrate>
                                                            <Samplerate>48000</Samplerate>
                                                            <Channel>2</Channel>
                                                    </Audio>
                                                    <Video>
                                                            <Codec>h264</Codec>
                                                            <Width>640</Width>
                                                            <Height>360</Height>
                                                            <Bitrate>350000</Bitrate>
                                                            <Framerate>30.0</Framerate>
                                                    </Video>
                                            </Encode>
					 <Encode>
                                                    <Name>VP8_medium</Name>
                                                    <Audio>
                                                            <Codec>opus</Codec>
                                                            <Bitrate>48000</Bitrate>
                                                            <Samplerate>48000</Samplerate>
                                                            <Channel>2</Channel>
                                                    </Audio>
                                                    <Video>
                                                            <Codec>vp8</Codec>
                                                            <Width>640</Width>
                                                            <Height>360</Height>
                                                            <Bitrate>700000</Bitrate>
                                                            <Framerate>30.0</Framerate>
                                                    </Video>
                                            </Encode>
                                            <Encode>
                                                    <Name>H264_medium</Name>
                                                    <Audio>
                                                            <Codec>aac</Codec>
                                                            <Bitrate>48000</Bitrate>
                                                            <Samplerate>48000</Samplerate>
                                                            <Channel>2</Channel>
                                                    </Audio>
                                                    <Video>
                                                            <Codec>h264</Codec>
                                                            <Width>640</Width>
                                                            <Height>360</Height>
                                                            <Bitrate>700000</Bitrate>
                                                            <Framerate>30.0</Framerate>
                                                    </Video>
                                            </Encode>
					 <Encode>
                                                    <Name>VP8_high</Name>
                                                    <Audio>
                                                            <Codec>opus</Codec>
                                                            <Bitrate>64000</Bitrate>
                                                            <Samplerate>48000</Samplerate>
                                                            <Channel>2</Channel>
                                                    </Audio>
                                                    <Video>
                                                            <Codec>vp8</Codec>
                                                            <Width>640</Width>
                                                            <Height>360</Height>
                                                            <Bitrate>1500000</Bitrate>
                                                            <Framerate>30.0</Framerate>
                                                    </Video>
                                            </Encode>
                                            <Encode>
                                                    <Name>H264_high</Name>
                                                    <Audio>
                                                            <Codec>aac</Codec>
                                                            <Bitrate>64000</Bitrate>
                                                            <Samplerate>48000</Samplerate>
                                                            <Channel>2</Channel>
                                                    </Audio>
                                                    <Video>
                                                            <Codec>h264</Codec>
                                                            <Width>640</Width>
                                                            <Height>360</Height>
                                                            <Bitrate>1500000</Bitrate>
                                                            <Framerate>30.0</Framerate>
                                                    </Video>
                                            </Encode>
				</Encodes>
				<Streams>
					<!--<Stream>
						<Name>${OriginStreamName}_o</Name>
						<Profiles>
							<Profile>VP8</Profile>
							<Profile>H264</Profile>
						</Profiles>
					</Stream>-->
					<!--<Stream>
                                                    <Name>${OriginStreamName}_o</Name>
                                                    <Profiles>
                                                            <Profile>VP8_low</Profile>
                                                            <Profile>H264_low</Profile>
                                                            <Profile>VP8_medium</Profile>
                                                            <Profile>H264_medium</Profile>
                                                            <Profile>VP8_high</Profile>
                                                            <Profile>H264_high</Profile>
                                                    </Profiles>
                                            </Stream>-->
					<Stream>
                                                    <Name>${OriginStreamName}_low_o</Name>
                                                    <Profiles>
                                                            <Profile>VP8_low</Profile>
                                                            <Profile>H264_low</Profile>
                                                    </Profiles>
                                            </Stream>
                                            <Stream>
                                                    <Name>${OriginStreamName}_medium_o</Name>
                                                    <Profiles>
                                                            <Profile>VP8_medium</Profile>
                                                            <Profile>H264_medium</Profile>
                                                    </Profiles>
                                            </Stream>
                                            <Stream>
                                                    <Name>${OriginStreamName}_high_o</Name>
                                                    <Profiles>
                                                            <Profile>VP8_high</Profile>
                                                            <Profile>H264_high</Profile>
                                                    </Profiles>
                                            </Stream>
				</Streams>
				<Providers>
					<RTMP />
				</Providers>
				<Publishers>
					<ThreadCount>32</ThreadCount>
					<HLS>
						<SegmentDuration>5</SegmentDuration>
						<SegmentCount>3</SegmentCount>
						<CrossDomain>
							<Url>*</Url>
						</CrossDomain>
					</HLS>
					<DASH>
						<SegmentDuration>5</SegmentDuration>
						<SegmentCount>3</SegmentCount>
						<CrossDomain>
							<Url>*</Url>
						</CrossDomain>
					</DASH>
					<WebRTC>
						<Timeout>30000</Timeout>
					</WebRTC>
				</Publishers>
			</Application>
		</Applications>
	</Host>
</Hosts>

In the webpage.

<script>
    var player = OvenPlayer.create("player", {
        debug : true,
        autoStart : true,
    sources: [

{
"file": "ws://my_ip:3333/app/stream_low_o",
"type": "webrtc",
"label": "Low Quality WebRTC"
},
{
"file": "ws://my_ip:3333/app/stream_medium_o",
"type": "webrtc",
"label": "Medium Quality WebRTC"
},
{
"file": "ws://my_ip:3333/app/stream_high_o",
"type": "webrtc",
"label": "High Quality WebRTC"
},
{
"file": "http://my_ip:81/app/stream_low_o/manifest.mpd",
"type": "dash",
"label": "Low Quality MPEG-DASH"
},
{
"file": "http://my_ip:81/app/stream_medium_o/manifest.mpd",
"type": "dash",
"label": "Medium Quality MPEG-DASH"
},
{
"file": "http://my_ip:81/app/stream_high_o/manifest.mpd",
"type": "dash",
"label": "High Quality MPEG-DASH"
}
]
});

player.setAutoQuality(true);
</script>

@getroot
Copy link
Member

getroot commented Mar 25, 2020

I am not sure of this problem, but I think there is probably a performance problem with the server. Live transcoding is very expensive. I recommend you to monitor the server's status.

@IanMitchell77
Copy link
Author

Its a 32 core machine(hence 32 threads) with a GPU.

Its using a tiny fraction of the cycles available and i dont see any come maxing out.

Is there some configuration that I should use?

@getroot
Copy link
Member

getroot commented Mar 26, 2020

If the server is not using the maximum CPU, it is probably not the cause. Could you lower Publisher's ThreadCount to 8 and try it again? Too many threads may be the problem.

8

If this doesn't work, give me more information.

Do you use OME at dev branch?
What do you use browser to play WebRTC?
Give me your ovenmediaengine.log file when you got the problems.
What is your encoder? If it is obs, please give me the detailed configuration capture.

and how about DASH playback?

@IanMitchell77
Copy link
Author

Hi,

Reducing the threads didnt help.

I am using version fd40dd5 of the player.

I am using Chrome on F27.

Here are my OBS settings
image

This is from the log file:
[2020-03-26 21:18:02.903] I 6230 Signalling | rtc_signalling_server.cpp:93 | New client is connected: <ClientSocket: 0x7f6500000b90, #27, state: 4, TCP, [v4] 176.255.126.144:47680>
[2020-03-26 21:18:03.005] E 6230 Socket | socket_address.cpp:276 | An error occurred while resolve DNS for host [93cb9fae-8a72-45bd-a0a1-05a165f7a33c.local]
[2020-03-26 21:18:03.005] E 6230 Socket | socket_address.cpp:83 | An error occured: 93cb9fae-8a72-45bd-a0a1-05a165f7a33c.local:49768
[2020-03-26 21:18:03.006] E 6230 Socket | socket_address.cpp:276 | An error occurred while resolve DNS for host [86618dcf-b87a-4bc9-8a50-2eba6a5dfcd4.local]
[2020-03-26 21:18:03.006] E 6230 Socket | socket_address.cpp:83 | An error occured: 86618dcf-b87a-4bc9-8a50-2eba6a5dfcd4.local:55050
[2020-03-26 21:18:08.676] I 6230 Signalling | rtc_signalling_server.cpp:240 | Client is disconnected: <WebSocketClient: 0x7f65000d0030, <ClientSocket: 0x7f6500000b90, #27, state: 0, TCP, [v4] 176.255.126.144:47680>> (app / stream_low_o, ufrag: local: BT01hl, remote: j/DN)
[2020-03-26 21:18:08.693] I 6230 HttpServer | http_server.cpp:109 | Client(<ClientSocket: 0x7f6500000b90, #27, state: 4, TCP, [v4] 176.255.126.144:47682>) is connected on [v4] 0.0.0.0:3333
[2020-03-26 21:18:08.694] I 6230 Signalling | rtc_signalling_server.cpp:93 | New client is connected: <ClientSocket: 0x7f6500000b90, #27, state: 4, TCP, [v4] 176.255.126.144:47682>
[2020-03-26 21:18:08.792] E 6230 Socket | socket_address.cpp:276 | An error occurred while resolve DNS for host [93cb9fae-8a72-45bd-a0a1-05a165f7a33c.local]
[2020-03-26 21:18:08.792] E 6230 Socket | socket_address.cpp:83 | An error occured: 93cb9fae-8a72-45bd-a0a1-05a165f7a33c.local:52407
[2020-03-26 21:18:08.793] E 6230 Socket | socket_address.cpp:276 | An error occurred while resolve DNS for host [86618dcf-b87a-4bc9-8a50-2eba6a5dfcd4.local]
[2020-03-26 21:18:08.793] E 6230 Socket | socket_address.cpp:83 | An error occured: 86618dcf-b87a-4bc9-8a50-2eba6a5dfcd4.local:45496
[2020-03-26 21:18:13.955] I 6230 Signalling | rtc_signalling_server.cpp:240 | Client is disconnected: <WebSocketClient: 0x7f650012d380, <ClientSocket: 0x7f6500000b90, #27, state: 0, TCP, [v4] 176.255.126.144:47682>> (app / stream_medium_o, ufrag: local: moWiFv, remote: wf+J)
[2020-03-26 21:18:13.989] I 6230 HttpServer | http_server.cpp:109 | Client(<ClientSocket: 0x7f6500000b90, #27, state: 4, TCP, [v4] 176.255.126.144:47684>) is connected on [v4] 0.0.0.0:3333
[2020-03-26 21:18:13.990] I 6230 Signalling | rtc_signalling_server.cpp:93 | New client is connected: <ClientSocket: 0x7f6500000b90, #27, state: 4, TCP, [v4] 176.255.126.144:47684>
[2020-03-26 21:18:14.111] E 6230 Socket | socket_address.cpp:276 | An error occurred while resolve DNS for host [93cb9fae-8a72-45bd-a0a1-05a165f7a33c.local]
[2020-03-26 21:18:14.111] E 6230 Socket | socket_address.cpp:83 | An error occured: 93cb9fae-8a72-45bd-a0a1-05a165f7a33c.local:49338
[2020-03-26 21:18:14.114] E 6230 Socket | socket_address.cpp:276 | An error occurred while resolve DNS for host [86618dcf-b87a-4bc9-8a50-2eba6a5dfcd4.local]
[2020-03-26 21:18:14.114] E 6230 Socket | socket_address.cpp:83 | An error occured: 86618dcf-b87a-4bc9-8a50-2eba6a5dfcd4.local:38372

@IanMitchell77
Copy link
Author

Using the MPEG-DASH streams it seems to switch much faster but the latency is very long.

Is it possible to switch fast and have ultra low latency?

@getroot
Copy link
Member

getroot commented Mar 27, 2020

Ah!
Are you currently using OME from the Master branch?
I recommend using the DEV branch. There are numerous upgrades and performance improvements.
Thank you for your contribution.

@getroot
Copy link
Member

getroot commented Mar 27, 2020

We are merging the Dev branch to the Master branch next week and this is the new version of the manual. https://airensoft.gitbook.io/ovenmediaengine/v/0.9.5/

@IanMitchell77
Copy link
Author

Ill switch to the dev branch of OME and let you know - I was using release 0.9.1

@IanMitchell77
Copy link
Author

I am now on the dev branch of OME and I am on master of the player.

I dont see changes.

To be able to switch quickly should I have several encoder profiles and have different streams using them or should I have just one stream with all the encode variants(as commented out above)?

@getroot
Copy link
Member

getroot commented Mar 27, 2020

It is very difficult to analyze this problem. Because it doesn't happen to me. Isn't the packet loss caused by your network too slow? If you are using chrome, you can also enter chrome: // webrtc-internals in the address bar and view detailed statistics.

I tested it with the exact same settings as you. (Latest DEV) And I saw the stream switching complete within 1 second and start playing and I used http://demo.ovenplayer.com as player to play this.

My configuration of server are:

<?xml version="1.0" encoding="UTF-8" ?>

<Server version="5">
	<Name>OvenMediaEngine</Name>
	<!-- Host type (origin/edge) -->
	<Type>origin</Type>
	<!-- Specify IP address to bind (* means all IPs) -->
	<IP>*</IP>

	<!-- Settings for the ports to bind -->
	<Bind>
		<Providers>
			<RTMP><Port>${env:OME_RTMP_PROV_PORT:1935}</Port></RTMP>
		</Providers>

		<Publishers>
            <!-- The OVT is protocol for ORIGIN-EDGE -->
			<OVT><Port>${env:OME_ORIGIN_PORT:9000}</Port></OVT>
			<HLS><Port>${env:OME_HLS_STREAM_PORT:8080}</Port></HLS>
			<DASH><Port>${env:OME_DASH_STREAM_PORT:8080}</Port></DASH>
			<WebRTC>
				<Signalling><Port>${env:OME_SIGNALLING_PORT:3333}</Port></Signalling>
				<IceCandidates>
					<IceCandidate>${env:OME_ICE_CANDIDATES:*:10000-10005/udp}</IceCandidate>
				</IceCandidates>
			</WebRTC>
		</Publishers>
	</Bind>

	<VirtualHosts>
		<!--
			You can include multiple XML files by doing the following:
			<VirtualHost include="sites-enabled/*.xml" />
		-->
		<VirtualHost include="VHost*.xml" />
		<VirtualHost>
			<Name>default</Name>

			<!-- Settings for multi domain and TLS -->
			<Domain>
                <Names>
				<!-- Domain names
					<Name>stream1.airensoft.com</Name>
					<Name>stream2.airensoft.com</Name>
					<Name>*.sub.airensoft.com</Name>
				-->
                    <Name>*</Name>
                </Names>
				<!--
				<TLS>
					<CertPath>path/to/file.crt</CertPath>
					<KeyPath>path/to/file.key</KeyPath>
					<ChainCertPath>path/to/file.crt</ChainCertPath>
				</TLS>
				-->
			</Domain>

		<Applications>
			<Application>
				<Name>app</Name>
				<Type>live</Type>
				<Encodes>
					<Encode>
							<Name>VP8_low</Name>
							<Audio>
									<Codec>opus</Codec>
									<Bitrate>48000</Bitrate>
									<Samplerate>48000</Samplerate>
									<Channel>2</Channel>
							</Audio>
							<Video>
									<Codec>vp8</Codec>
									<Width>640</Width>
									<Height>360</Height>
									<Bitrate>350000</Bitrate>
									<Framerate>30.0</Framerate>
							</Video>
					</Encode>
					<Encode>
							<Name>H264_low</Name>
							<Audio>
									<Codec>aac</Codec>
									<Bitrate>48000</Bitrate>
									<Samplerate>48000</Samplerate>
									<Channel>2</Channel>
							</Audio>
							<Video>
									<Codec>h264</Codec>
									<Width>640</Width>
									<Height>360</Height>
									<Bitrate>350000</Bitrate>
									<Framerate>30.0</Framerate>
							</Video>
					</Encode>
					<Encode>
							<Name>VP8_medium</Name>
							<Audio>
									<Codec>opus</Codec>
									<Bitrate>48000</Bitrate>
									<Samplerate>48000</Samplerate>
									<Channel>2</Channel>
							</Audio>
							<Video>
									<Codec>vp8</Codec>
									<Width>640</Width>
									<Height>360</Height>
									<Bitrate>700000</Bitrate>
									<Framerate>30.0</Framerate>
							</Video>
					</Encode>
					<Encode>
							<Name>H264_medium</Name>
							<Audio>
									<Codec>aac</Codec>
									<Bitrate>48000</Bitrate>
									<Samplerate>48000</Samplerate>
									<Channel>2</Channel>
							</Audio>
							<Video>
									<Codec>h264</Codec>
									<Width>640</Width>
									<Height>360</Height>
									<Bitrate>700000</Bitrate>
									<Framerate>30.0</Framerate>
							</Video>
					</Encode>
					 <Encode>
							<Name>VP8_high</Name>
							<Audio>
									<Codec>opus</Codec>
									<Bitrate>64000</Bitrate>
									<Samplerate>48000</Samplerate>
									<Channel>2</Channel>
							</Audio>
							<Video>
									<Codec>vp8</Codec>
									<Width>640</Width>
									<Height>360</Height>
									<Bitrate>1500000</Bitrate>
									<Framerate>30.0</Framerate>
							</Video>
					</Encode>
					<Encode>
							<Name>H264_high</Name>
							<Audio>
									<Codec>aac</Codec>
									<Bitrate>64000</Bitrate>
									<Samplerate>48000</Samplerate>
									<Channel>2</Channel>
							</Audio>
							<Video>
									<Codec>h264</Codec>
									<Width>640</Width>
									<Height>360</Height>
									<Bitrate>1500000</Bitrate>
									<Framerate>30.0</Framerate>
							</Video>
					</Encode>
				</Encodes>
				<Streams>
					
					<Stream>
							<Name>${OriginStreamName}_low_o</Name>
							<Profiles>
									<Profile>VP8_low</Profile>
									<Profile>H264_low</Profile>
							</Profiles>
					</Stream>
					
					
					<Stream>
							<Name>${OriginStreamName}_medium_o</Name>
							<Profiles>
									<Profile>VP8_medium</Profile>
									<Profile>H264_medium</Profile>
							</Profiles>
					</Stream>

					
					<Stream>
							<Name>${OriginStreamName}_high_o</Name>
							<Profiles>
									<Profile>VP8_high</Profile>
									<Profile>H264_high</Profile>
							</Profiles>
					</Stream>
					
				</Streams>

				<Providers>
					<RTMP />
				</Providers>

				<Publishers>
					<ThreadCount>8</ThreadCount>
					<HLS>
						<SegmentDuration>5</SegmentDuration>
						<SegmentCount>3</SegmentCount>
						<CrossDomain>
							<Url>*</Url>
						</CrossDomain>
					</HLS>
					<DASH>
						<SegmentDuration>5</SegmentDuration>
						<SegmentCount>3</SegmentCount>
						<CrossDomain>
							<Url>*</Url>
						</CrossDomain>
					</DASH>
					<WebRTC>
						<Timeout>30000</Timeout>
					</WebRTC>
				</Publishers>
			</Application>
		</Applications>
		</VirtualHost>
	</VirtualHosts>
</Server>

@getroot
Copy link
Member

getroot commented Mar 27, 2020

Could you upload ovenmediaengine.log file generated by the dev branch version?

@IanMitchell77
Copy link
Author

So, I think we have different expectations and there is no problem.

From an email from you I understand that I can switch streams in 1 frame - how do we achieve this?

@getroot
Copy link
Member

getroot commented Mar 30, 2020

Because you said "several seconds", I misunderstood the exact meaning of what you said.

The ability to switch streams by 1Frame difference must be fully supported by the player.

HLS and DASH support ABR in Player, so we will add this feature to OME soon. We will add ABR to Playlist (Manifest, M3U8) by putting multiple <Profile> in <Stream> of OME configuration. This is not difficult.

However, we have a lot of discussion about WebRTC's ABR. This is, as far as I know, not yet an exact standard. (Please tell me if you know.) There is a Simulcast, but this is not for seamless swtiching. Another method we are considering is changing the bitrate at the server's RTP level and sending it. But this requires a lot of proof to be used commercially.

If there are any changes to ABR, we will notify you through this site.

@IanMitchell77
Copy link
Author

To be clear, you cant switch stream in 1 frame with OvenMediaEngine and OvenPlayer but you plan to add support for ABR HLS and DASH streams in the OvenMediaEngine - right?

How soon do you think that will happen?

@getroot
Copy link
Member

getroot commented Mar 30, 2020

I am not sure how long that feature will be added, but we will try to add ABR feature in 2 months. (Because we are usually busy with commercial projects.)

@getroot
Copy link
Member

getroot commented May 8, 2020

This issue has been closed since it has been inactive for quite some time. If you want to continue discussing this issue, please feel free to reopen it.

@getroot getroot closed this as completed May 8, 2020
@GregOriol
Copy link
Contributor

GregOriol commented Mar 21, 2021

@getroot I was looking at this issue to figure out how to configure OME to stream various quality over webrtc, and your example of a server configuration seems helpful. However, I think it is now outdated as from what I could figure, <Encodes> and <Streams> have been replaced by <OutputProfiles>, am I right? Could it be possible for you to post an up-to-date Server.xml example? or maybe add one to the docs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants