This repository has been archived by the owner on Mar 22, 2022. It is now read-only.
Add standalone media track objects #152
Assignees
Labels
breaking change
This change breaks the current API and requires migration
enhancement
New feature or request
multitrack
Issues related to multi-track support
Milestone
Comments
This was referenced Jan 2, 2020
djee-ms
added a commit
that referenced
this issue
Jan 9, 2020
Fix the documentation and README.md to clarify that in 1.0 the PeerConnection supports a single audio/video track per instance, and multiple instances need to be used for sending/receiving more. The support of multi-track per instance is tracked by issue #152.
djee-ms
added a commit
to djee-ms/MixedReality-WebRTC
that referenced
this issue
Feb 26, 2020
This is part 1/N of the change, see global description below ---- Add standalone media track objects in both the C++ and C# library. This enables supporting multiple audio and video tracks per peer connection. For local tracks, the `PeerConnection` interface is updated such that the functionality of `AddLocalAudioTrackAsync()` and `AddLocalVideoTrackAsync()` is split in 2 steps: - the creation step, whose interface is on the local track object itself, creates a standalone track object not associated with any peer connection and not currently used by anything. - the association step, which takes the track and attach it to a peer connection via a transceiver. For remote tracks, the `TrackAdded` and `TrackRemoved` events now generate a new remote track object, which is passed to the event handler as argument. The C# `PeerConnection` object also now keeps a list of all the tracks attached to it, which can therefore be enumerated. Because tracks are standalone objects, methods like `PeerConnection.SetLocalAudioTrackEnabled()` are replaced with a property `LocalAudioTrack.Enabled`, providing a more familiar pattern to C# developers. One important change required for remote tracks to be paired by name is removing the legacy options `RTCOfferAnswerOptions::offer_to_receive_video = true` and `RTCOfferAnswerOptions::offer_to_receive_audio = true` from `CreateOffer()` and `CreateAnswer()`. This change (see microsoft#75) has larger implications that it looks when using Unified Plan (default), as previously those options were implicitly making an offer to receive some media, whereas this step is now explicit and the user needs to add some transceiver in receive-only or send-receive mode before making an offer/answer to allow the remote peer to send. A new Unity demo `StandaloneDemo` shows how to use multiple tracks between two peers. The demo is standalone in the sense signaling is hard-coded and does not require an external node-dss server or any other signaling solution. Bug: microsoft#152
djee-ms
added a commit
that referenced
this issue
Mar 4, 2020
This is part 1/N of the change, see global description below ---- Add standalone media track objects in both the C++ and C# library. This enables supporting multiple audio and video tracks per peer connection. For local tracks, the `PeerConnection` interface is updated such that the functionality of `AddLocalAudioTrackAsync()` and `AddLocalVideoTrackAsync()` is split in 2 steps: - the creation step, whose interface is on the local track object itself, creates a standalone track object not associated with any peer connection and not currently used by anything. - the association step, which takes the track and attach it to a peer connection via a transceiver. For remote tracks, the `TrackAdded` and `TrackRemoved` events now generate a new remote track object, which is passed to the event handler as argument. The C# `PeerConnection` object also now keeps a list of all the tracks attached to it, which can therefore be enumerated. Because tracks are standalone objects, methods like `PeerConnection.SetLocalAudioTrackEnabled()` are replaced with a property `LocalAudioTrack.Enabled`, providing a more familiar pattern to C# developers. One important change required for remote tracks to be paired by name is removing the legacy options `RTCOfferAnswerOptions::offer_to_receive_video = true` and `RTCOfferAnswerOptions::offer_to_receive_audio = true` from `CreateOffer()` and `CreateAnswer()`. This change (see #75) has larger implications that it looks when using Unified Plan (default), as previously those options were implicitly making an offer to receive some media, whereas this step is now explicit and the user needs to add some transceiver in receive-only or send-receive mode before making an offer/answer to allow the remote peer to send. A new Unity demo `StandaloneDemo` shows how to use multiple tracks between two peers. The demo is standalone in the sense signaling is hard-coded and does not require an external node-dss server or any other signaling solution. Bug: #152
This was referenced Mar 12, 2020
djee-ms
added a commit
that referenced
this issue
Mar 30, 2020
Update the Unity tutorial in the documentation to reflect the latest API changes introduced by the multi-track work (#152). This change does not update the C++ and C# library documentations. This will be done in a future change. In addition, rename the CreateOffer() and SetRemoteDescriptionAsync() methods of the PeerConnection Unity component to StartConnection() and HandleConnectionMessageAsync(), respectively. This disambiguates them from the methods of the PeerConnection class in the C# library, in an effort to highlight that the former ones perform more work than the latter ones.
djee-ms
added
multitrack
Merged
djee-ms
added a commit
that referenced
this issue
Apr 21, 2020
Add support for multiple audio and video tracks per peer connection. This change introduces a new transceiver API closely related to the same-named concept in the WebRTC 1.0 standard. The transceiver API allows the user to add audio and video transceivers to a peer connection, then optionally associate some local track to send some media through those transceivers and/or receive some media from the remote peer via a remote track associated with the transceiver. The new API is more flexible and more powerful, offering more control to the user and opening up many use cases not previously available, at the expense of some moderately increased complexity. In particular it allows: - multiple audio and video tracks per peer connection, limited only by the bandwidth - hot-swapping a track for another track without a renegotiation, dramatically reducing the delay of replacing a track with another one (e.g. switching to "on hold" music instead of voice chat) - "track warm up" (getting a track transport ready even before the track media is available) and more aggressive media transport establishing (callee peer can start sending media to caller peer earlier than before) This change constitutes a major API breaking change, which moves tracks management away from the PeerConnection object to the new Transceiver object. Users are encouraged to read the migration guide associated with this change to migrate existing v1.0-style applications. In particular, understanding the functioning of transceivers and the asymmetrical role of the caller peer (the peer initiating a session with a new offer) and the callee peer (the peer responding to a remote offer) are critical to be able to use this new API correctly. Bug: #144, #152
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Currently in 1.0 the
PeerConnectionobject supports a single local/remote audio/video track (so 4 in total). This corresponds to the simple video chat use case of a single A/V feed sent and received. Inmastera standaloneLocalVideoTrackobject has been added, and with it the ability to open multiple local video tracks per peer connection (see #35). However because a single remote video track is supported, there is currently no way to receive multiple video tracks on the remote peer. And for audio multi-tracks is simply not supported by a single peer connection (multiple connections can be used as a workaround).This feature introduces some standalone media tracks for all 4 categories of local/remote audio/video tracks, to effectively enable multi-track support per peer connection.
The text was updated successfully, but these errors were encountered: