Skip to content
This repository has been archived by the owner on Mar 22, 2022. It is now read-only.

Commit

Permalink
Add transceiver API to C++
Browse files Browse the repository at this point in the history
This is part 1/N of the change, see global description below

----

Add standalone media track objects in both the C++ and C# library.
This enables supporting multiple audio and video tracks per peer
connection.

For local tracks, the `PeerConnection` interface is updated such that
the functionality of `AddLocalAudioTrackAsync()` and
`AddLocalVideoTrackAsync()` is split in 2 steps:

- the creation step, whose interface is on the local track object
itself, creates a standalone track object not associated with any peer
connection and not currently used by anything.

- the association step, which takes the track and attach it to a peer
connection via a transceiver.

For remote tracks, the `TrackAdded` and `TrackRemoved` events now
generate a new remote track object, which is passed to the event handler
as argument. The C# `PeerConnection` object also now keeps a list of all
the tracks attached to it, which can therefore be enumerated.

Because tracks are standalone objects, methods like
`PeerConnection.SetLocalAudioTrackEnabled()` are replaced with a
property `LocalAudioTrack.Enabled`, providing a more familiar pattern to
C# developers.

One important change required for remote tracks to be
paired by name is removing the legacy options
`RTCOfferAnswerOptions::offer_to_receive_video = true` and
`RTCOfferAnswerOptions::offer_to_receive_audio = true` from
`CreateOffer()` and `CreateAnswer()`. This change (see #75) has larger
implications that it looks when using Unified Plan (default), as
previously those options were implicitly making an offer to receive some
media, whereas this step is now explicit and the user needs to add some
transceiver in receive-only or send-receive mode before making an
offer/answer to allow the remote peer to send.

A new Unity demo `StandaloneDemo` shows how to use multiple tracks
between two peers. The demo is standalone in the sense signaling is
hard-coded and does not require an external node-dss server or any other
signaling solution.

Bug: #152
  • Loading branch information
djee-ms committed Mar 4, 2020
1 parent ad10956 commit ba75e79
Show file tree
Hide file tree
Showing 53 changed files with 5,971 additions and 794 deletions.
469 changes: 337 additions & 132 deletions libs/Microsoft.MixedReality.WebRTC.Native/include/interop_api.h

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,21 @@ mrsLocalVideoTrackAddRef(LocalVideoTrackHandle handle) noexcept;
MRS_API void MRS_CALL
mrsLocalVideoTrackRemoveRef(LocalVideoTrackHandle handle) noexcept;

/// Create a new local video track by opening a local video capture device
/// (webcam).
/// [UWP] This must be invoked from another thread than the main UI thread.
MRS_API mrsResult MRS_CALL mrsLocalVideoTrackCreateFromDevice(
const LocalVideoTrackInitConfig* config,
const char* track_name,
LocalVideoTrackHandle* track_handle_out) noexcept;

/// Create a new local video track by using an existing external video source.
MRS_API mrsResult MRS_CALL mrsLocalVideoTrackCreateFromExternalSource(
ExternalVideoTrackSourceHandle source_handle,
const LocalVideoTrackFromExternalSourceInitConfig* config,
const char* track_name,
LocalVideoTrackHandle* track_handle_out) noexcept;

/// Register a custom callback to be called when the local video track captured
/// a frame. The captured frames is passed to the registered callback in I420
/// encoding.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,18 @@ MRS_API void MRS_CALL mrsPeerConnectionRegisterIceGatheringStateChangedCallback(
mrsPeerConnectionIceGatheringStateChangedCallback callback,
void* user_data) noexcept;

/// Create a new audio transceiver attached to the given peer connection.
/// The audio transceiver is initially inactive.
MRS_API mrsResult MRS_CALL
mrsPeerConnectionAddAudioTransceiver(PeerConnectionHandle peer_handle,
const AudioTransceiverInitConfig* config,
AudioTransceiverHandle* handle) noexcept;

/// Create a new video transceiver attached to the given peer connection.
/// The audio transceiver is initially inactive.
MRS_API mrsResult MRS_CALL
mrsPeerConnectionAddVideoTransceiver(PeerConnectionHandle peer_handle,
const VideoTransceiverInitConfig* config,
VideoTransceiverHandle* handle) noexcept;

} // extern "C"
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

// This is a precompiled header, it must be on its own, followed by a blank
// line, to prevent clang-format from reordering it with other headers.
#include "pch.h"

#include "interop/audio_transceiver_interop.h"
#include "media/audio_transceiver.h"

using namespace Microsoft::MixedReality::WebRTC;

void MRS_CALL
mrsAudioTransceiverAddRef(AudioTransceiverHandle handle) noexcept {
if (auto transceiver = static_cast<AudioTransceiver*>(handle)) {
transceiver->AddRef();
} else {
RTC_LOG(LS_WARNING)
<< "Trying to add reference to NULL AudioTransceiver object.";
}
}

void MRS_CALL
mrsAudioTransceiverRemoveRef(AudioTransceiverHandle handle) noexcept {
if (auto transceiver = static_cast<AudioTransceiver*>(handle)) {
transceiver->RemoveRef();
} else {
RTC_LOG(LS_WARNING)
<< "Trying to remove reference from NULL AudioTransceiver object.";
}
}

void MRS_CALL mrsAudioTransceiverRegisterStateUpdatedCallback(
AudioTransceiverHandle handle,
mrsAudioTransceiverStateUpdatedCallback callback,
void* user_data) noexcept {
if (auto transceiver = static_cast<AudioTransceiver*>(handle)) {
transceiver->RegisterStateUpdatedCallback(
Transceiver::StateUpdatedCallback{callback, user_data});
}
}

mrsResult MRS_CALL mrsAudioTransceiverSetDirection(
AudioTransceiverHandle transceiver_handle,
mrsTransceiverDirection new_direction) noexcept {
if (auto transceiver = static_cast<AudioTransceiver*>(transceiver_handle)) {
return transceiver->SetDirection(new_direction);
}
return Result::kInvalidNativeHandle;
}

mrsResult MRS_CALL
mrsAudioTransceiverSetLocalTrack(AudioTransceiverHandle transceiver_handle,
LocalAudioTrackHandle track_handle) noexcept {
if (!transceiver_handle) {
return Result::kInvalidNativeHandle;
}
auto transceiver = static_cast<AudioTransceiver*>(transceiver_handle);
auto track = static_cast<LocalAudioTrack*>(track_handle);
return transceiver->SetLocalTrack(track);
}

mrsResult MRS_CALL mrsAudioTransceiverGetLocalTrack(
AudioTransceiverHandle transceiver_handle,
LocalAudioTrackHandle* track_handle_out) noexcept {
if (!track_handle_out) {
return Result::kInvalidParameter;
}
if (auto transceiver = static_cast<AudioTransceiver*>(transceiver_handle)) {
*track_handle_out = transceiver->GetLocalTrack().release();
return Result::kSuccess;
}
return Result::kInvalidNativeHandle;
}

mrsResult MRS_CALL mrsAudioTransceiverGetRemoteTrack(
AudioTransceiverHandle transceiver_handle,
RemoteAudioTrackHandle* track_handle_out) noexcept {
if (!track_handle_out) {
return Result::kInvalidParameter;
}
if (auto transceiver = static_cast<AudioTransceiver*>(transceiver_handle)) {
*track_handle_out = transceiver->GetRemoteTrack().release();
return Result::kSuccess;
}
return Result::kInvalidNativeHandle;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

#pragma once

#include "export.h"
#include "interop_api.h"

extern "C" {

using mrsAudioTransceiverStateUpdatedCallback =
void(MRS_CALL*)(void* user_data,
mrsTransceiverStateUpdatedReason reason,
mrsTransceiverOptDirection negotiated_direction,
mrsTransceiverDirection desired_direction);

/// Add a reference to the native object associated with the given handle.
MRS_API void MRS_CALL
mrsAudioTransceiverAddRef(AudioTransceiverHandle handle) noexcept;

/// Remove a reference from the native object associated with the given handle.
MRS_API void MRS_CALL
mrsAudioTransceiverRemoveRef(AudioTransceiverHandle handle) noexcept;

MRS_API void MRS_CALL mrsAudioTransceiverRegisterStateUpdatedCallback(
AudioTransceiverHandle handle,
mrsAudioTransceiverStateUpdatedCallback callback,
void* user_data) noexcept;

/// Set the new desired transceiver direction.
MRS_API mrsResult MRS_CALL
mrsAudioTransceiverSetDirection(AudioTransceiverHandle transceiver_handle,
mrsTransceiverDirection new_direction) noexcept;

/// Set the local audio track associated with this transceiver. This new track
/// replaces the existing one, if any. This doesn't require any SDP
/// renegotiation.
MRS_API mrsResult MRS_CALL
mrsAudioTransceiverSetLocalTrack(AudioTransceiverHandle transceiver_handle,
LocalAudioTrackHandle track_handle) noexcept;

/// Get the local audio track associated with this transceiver, if any.
/// The returned handle holds a reference to the video transceiver, which must
/// be released with |mrsLocalAudioTrackRemoveRef()| once not needed anymore.
MRS_API mrsResult MRS_CALL mrsAudioTransceiverGetLocalTrack(
AudioTransceiverHandle transceiver_handle,
LocalAudioTrackHandle* track_handle_out) noexcept;

/// Get the remote audio track associated with this transceiver, if any.
/// The returned handle holds a reference to the video transceiver, which must
/// be released with |mrsRemoteAudioTrackRemoveRef()| once not needed anymore.
MRS_API mrsResult MRS_CALL mrsAudioTransceiverGetRemoteTrack(
AudioTransceiverHandle transceiver_handle,
RemoteAudioTrackHandle* track_handle_out) noexcept;

} // extern "C"
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,20 @@ using namespace Microsoft::MixedReality::WebRTC;
/// always valid, never deallocated.
std::string_view ObjectTypeToString(ObjectType type) {
static_assert((int)ObjectType::kPeerConnection == 0, "");
static_assert((int)ObjectType::kLocalVideoTrack == 1, "");
static_assert((int)ObjectType::kExternalVideoTrackSource == 2, "");
static_assert((int)ObjectType::kLocalAudioTrack == 1, "");
static_assert((int)ObjectType::kLocalVideoTrack == 2, "");
static_assert((int)ObjectType::kExternalVideoTrackSource == 3, "");
static_assert((int)ObjectType::kRemoteAudioTrack == 4, "");
static_assert((int)ObjectType::kRemoteVideoTrack == 5, "");
static_assert((int)ObjectType::kDataChannel == 6, "");
static_assert((int)ObjectType::kAudioTransceiver == 7, "");
static_assert((int)ObjectType::kVideoTransceiver == 8, "");
constexpr const std::string_view s_types[] = {
"PeerConnection", "LocalVideoTrack", "ExternalVideoTrackSource"};
"PeerConnection", "LocalAudioTrack",
"LocalVideoTrack", "ExternalVideoTrackSource",
"RemoteAudioTrack", "RemoteVideoTrack",
"DataChannel", "AudioTransceiver",
"VideoTransceiver"};
return s_types[(int)type];
}

Expand Down
Loading

0 comments on commit ba75e79

Please sign in to comment.