Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IMP] add simulcast #5

Merged
merged 1 commit into from
Jan 9, 2024
Merged

Conversation

ThanhDodeurOdoo
Copy link
Collaborator

@ThanhDodeurOdoo ThanhDodeurOdoo commented Dec 14, 2023

This commit adds the support for simulcast:

Simulcast is a way to let producers sent multiple version of the streams
as different layers of quality. This allows the server to select
the optimal layer based on the available bandwidth for each client.

Two env variables are used to control this feature:

VIDEO_MAX_BITRATE: Specifies the bitrate for the highest encoding
layer per stream. It sets the maxBitrate property of the
highest encoding of the RTCRtpEncodingParameters and is used
to compute the bitrate attributed to the other layers.

MAX_BITRATE_OUT: The maximum outgoing bitrate (=from the server) per
session (meaning that this cap is shared between all the incoming
streams for any given user). The appropriate simulcast layers used by
each consumer will be selected to honor this limit. If the bitrate is
still too high, packets will be dropped. Without this limit, the
connections will use as much bandwidth as possible, which means that
the simulcast layer will be chosen based on the client (or server) max
available bandwidth.

This commits also bumps the minor version as the bundle and the server
need to be updated to benefit from the feature (although both the
client and the server are backwards compatible).

@ThanhDodeurOdoo ThanhDodeurOdoo changed the title [MERGE] [MERGE] add simulcast and fix traceback on signal Dec 14, 2023
@ThanhDodeurOdoo ThanhDodeurOdoo changed the title [MERGE] add simulcast and fix traceback on signal [MERGE] add simulcast and fix traceback on signal 29 Dec 14, 2023
@ThanhDodeurOdoo ThanhDodeurOdoo force-pushed the master-discuss-call-simulcast-tso branch from 59a00a5 to 9f804f9 Compare December 14, 2023 10:59
@ThanhDodeurOdoo ThanhDodeurOdoo changed the title [MERGE] add simulcast and fix traceback on signal 29 [IMP] add simulcast Dec 14, 2023
@ThanhDodeurOdoo ThanhDodeurOdoo force-pushed the master-discuss-call-simulcast-tso branch 4 times, most recently from 7fecffe to 7c1f32c Compare December 15, 2023 07:46
@ThanhDodeurOdoo
Copy link
Collaborator Author

I tested a bit, bitrate limits appear to be respected

more tests could be made in the PR that updates the sfu bundle in the odoo codebase

@ThanhDodeurOdoo ThanhDodeurOdoo force-pushed the master-discuss-call-simulcast-tso branch 2 times, most recently from 9a6c24b to ecf16e5 Compare December 15, 2023 09:19
Comment on lines +250 to +247
encodings: [
{ scaleResolutionDownBy: 4, maxBitrate: Math.floor(MAX_VIDEO_BITRATE / 4) },
{ scaleResolutionDownBy: 2, maxBitrate: Math.floor(MAX_VIDEO_BITRATE / 2) },
{ scaleResolutionDownBy: 1, maxBitrate: MAX_VIDEO_BITRATE },
],
Copy link
Collaborator

@alexkuhn alexkuhn Dec 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm wrong, but a scale down of 4 implies a reduction of bitrate by 16, not 4, because it's a fourth of x and a fourth of y, resulting to 16 less pixels to render for an equivalent framerate, thus roughtly a bitrate divided by 16.

Of course, this math works with the bitrate scaling with pixels, which is not necessarily entirely true with the compression algorithms at hand. Probably the scale down has diminishing return of reduced bitrate if you take the compression algorithm into account, but probably not to the point it's close to linear with 1080p for example.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the resolution should scale down by the square of the value, but as you mentioned it may not scale exactly like that and there is likely some constant overhead on top of it, if we are too strict we may lose packets at the lower resolutions. Since it's a cap it shouldn't be a problem to have a generous margin since the lower resolutions will only use the bitrate they need.

@ThanhDodeurOdoo
Copy link
Collaborator Author

@seb-odoo

Copy link
Collaborator

@seb-odoo seb-odoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No objection from me if it works and others already approved.

Some nitpicks in comments.

README.md Outdated Show resolved Hide resolved
src/config.js Show resolved Hide resolved
src/config.js Outdated Show resolved Hide resolved
src/config.js Outdated Show resolved Hide resolved
@ThanhDodeurOdoo ThanhDodeurOdoo force-pushed the master-discuss-call-simulcast-tso branch from d47bb25 to 418c3f6 Compare January 9, 2024 12:42
This commit adds the support for simulcast:

Simulcast is a way to let producers sent multiple version of the streams
as different layers of quality. This allows the server to select
the optimal layer based on the available bandwidth for each client.

Two env variables are used to control this feature:

`VIDEO_MAX_BITRATE`: Specifies the bitrate for the highest encoding
layer per stream. It sets the `maxBitrate` property of the
highest encoding of the `RTCRtpEncodingParameters` and is used
to compute the bitrate attributed to the other layers.

`MAX_BITRATE_OUT`: The maximum outgoing bitrate (=from the server) per
session (meaning that this cap is shared between all the incoming
streams for any given user). The appropriate simulcast layers used by
each consumer will be selected to honor this limit. If the bitrate is
still too high, packets will be dropped. Without this limit, the
connections will use as much bandwidth as possible, which means that
the simulcast layer will be chosen based on the client (or server) max
available bandwidth.

This commits also bumps the minor version as the bundle and the server
need to be updated to benefit from the feature (although both the
client and the server are backwards compatible).
@ThanhDodeurOdoo ThanhDodeurOdoo force-pushed the master-discuss-call-simulcast-tso branch from 418c3f6 to 6502d2c Compare January 9, 2024 13:33
@seb-odoo seb-odoo merged commit beafcc2 into master Jan 9, 2024
1 check passed
@seb-odoo seb-odoo deleted the master-discuss-call-simulcast-tso branch January 9, 2024 16:26
@seb-odoo
Copy link
Collaborator

seb-odoo commented Jan 9, 2024

Possible to add tests for simulcast in the future?

@ThanhDodeurOdoo
Copy link
Collaborator Author

@seb-odoo It is difficult to test because it would require a stream mock that takes bandwidth and actual video resolution into account.
Note that with these parameters we are almost directly at the webRTC API layer ( https://w3c.github.io/webrtc-pc/#rtcrtpencodingparameters ).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants