This repository has been archived by the owner on Aug 30, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
b3b5216
commit 818af52
Showing
127 changed files
with
31,579 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: 54bd462083b6f3007e078b8646fab0df | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
.. _coresdk_api: | ||
|
||
CoreSDK API | ||
___________ | ||
The main part of the Dolby.io Communications C++ SDK is the Core SDK that is responsible for communicating with the Dolby.io backend and providing conferencing functionalities, such as | ||
creating sessions, joining conferences, receiving Websocket events, sending/receiving messages, receiving WebRTC media streams, managing audio devices and leaving conferences. | ||
The SDK also provides the ability to configure the default Audio/Video recorder to handle and store the incoming media streams in desired formats. | ||
The SDK provides the ability to create a default Audio/Video injector to which the application can provide raw media frames that will | ||
then be injected into the conference. The SDK also provides the ability to set spatial locations of participants' audio and participate in a shared spatial scene within a conference. | ||
The :ref:`api_sdk`, :ref:`api_session`, :ref:`api_conference`, :ref:`api_device_management`, :ref:`api_recorder`, :ref:`default_injector` sections are the core of the user interface to the CoreSDK. | ||
|
||
.. toctree:: | ||
:maxdepth: 2 | ||
:titlesonly: | ||
:glob: | ||
|
||
sdk/sdk.rst | ||
sdk/session.rst | ||
sdk/conference.rst | ||
sdk/device_management.rst | ||
sdk/recorder.rst | ||
sdk/injector.rst | ||
sdk/common/async_ops.rst | ||
sdk/common/log_level.rst | ||
sdk/common/exception.rst | ||
sdk/common/event_handling.rst | ||
|
||
.. note:: | ||
The Core SDK API provides methods that are asynchronous in nature and return :ref:`async_result <api_async_result>`. This means that if | ||
the result or the completion of operation is necessary for subsequent calls, you have to be careful. For more details, read the notes from the :cpp:class:`sdk <dolbyio::comms::sdk>`, | ||
:cpp:class:`conference <dolbyio::comms::services::conference>`, :cpp:class:`session <dolbyio::comms::services::device_management>` or :cpp:class:`session <dolbyio::comms::services::session>` interfaces and refer | ||
to the examples of each documented asynchronous method. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
.. _api_inject_audio: | ||
|
||
Audio Source Interface | ||
======================== | ||
#include <comms/media/media_engine.h> | ||
|
||
The interface through which raw audio frames are provided to the CoreSDK. This audio source must provide signed 16 bit PCM data in 10ms chunks at 10ms intervals. Application writers who want to | ||
implement this source must override the two virtual functions of :cpp:func:`registering <dolbyio::comms::audio_source::register_audio_frame_rtc_source>` and | ||
cpp:func:`deregistering <dolbyio::comms::audio_source::deregister_audio_frame_rtc_source>` the RTC Audio Source on the Injector. Attaching the RTC Audio source to the injector establishes the audio | ||
pipeline so that frames can be passed to the CoreSDK. | ||
|
||
.. doxygenclass:: dolbyio::comms::audio_source | ||
:project: C++ SDK | ||
|
||
The :cpp:class:`RTC Audio Source <dolbyio::comms::rtc_audio_source>` is NOT to be implemented by the application. This is the interface through which the injector can view its own Audio Sink. After | ||
receiving audio frames from some media source, the injector provides the raw audio frames to this RTC Audio Source. The provided audio is expected to be in 10ms chunks provided every 10ms. | ||
The RTC Audio Source then pushes the audio data further down the audio pipeline until it is injected into the conference. The RTC Audio Source expects :cpp:class:`audio frames <dolbyio::comms::audio_frame>`. | ||
|
||
.. doxygenclass:: dolbyio::comms::rtc_audio_source | ||
:project: C++ SDK | ||
|
||
.. doxygenclass:: dolbyio::comms::audio_frame | ||
:project: C++ SDK | ||
|
||
See :ref:`injection_example` for an example of child injection class for all possible media. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
.. _injection_example: | ||
|
||
Example Injector Implementation | ||
=============================== | ||
Below are the header and cpp files for a Media Injector Implementation. In order for this injector to be used by the C++ SDK, it must be passed to the conference service | ||
using the :cpp:func:`dolbyio::comms::services::conference::set_media_source` method. Now when a conference is joined and audio/video is started, this injector is part of the media injection | ||
pipeline. | ||
|
||
The interfaces inherited by this custom_injector example must be implemented by any custom injector intended to inject video and audio frames to a conference. | ||
|
||
If you want to use a custom injector module instead of the default injector module, you can replace the CMakeLists.txt in the :ref:`getting_started_server` section with the sample below. | ||
The paths in this CMakeLists.txt are assuming that the file is located in the same place as in the Writing Application portion of the Getting Started. | ||
|
||
custom_injector.h | ||
_________________ | ||
.. literalinclude:: ../../../src/public/comms/sample/custom_injector/custom_injector.h | ||
:language: c++ | ||
|
||
custom_injector.cc | ||
__________________ | ||
.. literalinclude:: ../../../src/public/comms/sample/custom_injector/custom_injector.cc | ||
:language: c++ | ||
|
||
main.cc | ||
_______ | ||
.. literalinclude:: ../../../src/public/comms/sample/custom_injector/main.cc | ||
:language: c++ | ||
|
||
Injector CMakeLists.txt | ||
_______________________ | ||
.. literalinclude:: ../../../src/public/comms/sample/custom_injector/CMakeLists.txt | ||
:language: console |
11 changes: 11 additions & 0 deletions
11
docs/_sources/api/injection/media_source_interface.rst.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
.. _api_media_source_interface: | ||
|
||
Media Source Interface | ||
========================= | ||
#include <comms/media/media_engine.h> | ||
|
||
Top level interface that exposes the underlying source interfaces for injecting raw video and raw audio. This abstract class exposes | ||
the two possible source interfaces: :cpp:class:`video <dolbyio::comms::video_source>`, :cpp:class:`audio <dolbyio::comms::audio_source>`. These two source interfaces essentially serve as input for the RTC Audio and Video sources created by the Media Engine. | ||
|
||
.. doxygenclass:: dolbyio::comms::media_source_interface | ||
:project: C++ SDK |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
.. _api_inject_video: | ||
|
||
Video Source Interface | ||
======================== | ||
#include <comms/media/media_engine.h> | ||
|
||
The interface through which raw video frames are provided to the CoreSDK. Application writers who want to implement this source must override the two virtual functions of :cpp:func:`registering <dolbyio::comms::video_source::register_video_frame_rtc_source>` and cpp:func:`deregistering <dolbyio::comms::video_source::deregister_video_frame_rtc_source>` the RTC Video Source on the Injector. When a | ||
VideoTrack is successfully added to an active PeerConnection, these virtual functions are invoked to provide/remove the reference to the RTC Video Source. Attaching the RTC Video source to the | ||
injector establishes the video pipeline so that frames can be passed to Webrtc. | ||
|
||
.. doxygenclass:: dolbyio::comms::video_source | ||
:project: C++ SDK | ||
|
||
|
||
The RTC Video Source is NOT to be implemented by the application. This is the interface through which the injector can view its own Video Sink. After receiving video frames from some media source, the | ||
injector provides the raw video frames to this RTC Video Source. The RTC Video Source then pushes the frame buffers along the video pipeline until they are injected into the conference. The | ||
raw video frame is deleted once it has been encoded. The RTC Video Source expects :cpp:class:`video frames <dolbyio::comms::video_frame>`. | ||
|
||
.. doxygenclass:: dolbyio::comms::rtc_video_source | ||
:project: C++ SDK | ||
|
||
See :ref:`injection_example` for an example of child injection class for all possible media. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
.. _api_media_injection: | ||
|
||
Media Injection API | ||
___________________ | ||
|
||
.. note:: | ||
The Media Injection feature of the C++SDK is intended for Server applications. | ||
|
||
|
||
The Media Injection API provides interfaces for writing modules that inject audio/video frames into the CoreSDK. The API provides the ability to create a module that accepts raw video/audio | ||
frames from any media source and passes them to respective sinks: :cpp:class:`RTCAudio <dolbyio::comms::rtc_audio_source>` and :cpp:class:`RTCVideo <dolbyio::comms::rtc_video_source>`. A media sink from | ||
the perspective of the injector is a media source from the perspective of the CoreSDK. The media is then encoded and the streams are sent into the conference. The C++ SDK provides a | ||
default injector module, explained in the :ref:`Default Injector <default_injector>` section, which the application can instantiate and provide raw frames to. This Default Injector module then takes | ||
care of getting the frames to the CoreSDK. The API for injecting media streams uses a pattern of setting producer objects, which are C++ classes with pure virtual functions. The virtual functions for | ||
connecting :cpp:class:`RTCAudioSource <dolbyio::comms::rtc_audio_source>` and :cpp:class:`RTCVideoSource <dolbyio::comms::rtc_video_source>` to an injector are exposed through the | ||
:cpp:class:`audio <dolbyio::comms::audio_source>` and :cpp:class:`video <dolbyio::comms::video_source>` source interface. The injector provides media frames to these RTC sources. The custom injector | ||
must also implement the :cpp:class:`dolbyio::comms::media_source_interface`, which is the interface through which the injector is provided to the Conference Service. | ||
|
||
.. toctree:: | ||
:maxdepth: 2 | ||
:titlesonly: | ||
:glob: | ||
|
||
injection/media_source_interface.rst | ||
injection/audio.rst | ||
injection/video.rst | ||
injection/example.rst |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
.. _api_media_recording: | ||
|
||
Media Recording API | ||
___________________ | ||
|
||
.. note:: | ||
The Media Recording feature of the C++ SDK is intended for Server applications. | ||
|
||
The Media Recording API provides interfaces for writing modules that allow recording audio and video. These modules are responsible for passing frames from incoming WebRTC Media Streams and handling, processing, and storing the media data. The C++ SDK provides a default recording module. However, using the Media Recording API, users can create and connect their own audio/video recording module to handle, process, and store streams from conferences. The API for recording media streams uses a pattern of setting listener objects, which are C++ classes with pure virtual functions. The virtual functions are exposed through sink interfaces for each of the type of media, which is provided by the Media Engine. The video data is received as :ref:`Raw <api_record_raw>` YUV frames or in the :ref:`Encoded <api_record_encoded>` format. The :ref:`audio <api_record_audio>` data is received in the decoded format. Listeners set by the application are called whenever data is available; these methods are called on the dedicated WebRTC threads that produced the data. | ||
|
||
.. toctree:: | ||
:maxdepth: 2 | ||
:titlesonly: | ||
:glob: | ||
|
||
recording/media_sink_interface.rst | ||
recording/audio.rst | ||
recording/encoded_video.rst | ||
recording/raw_video.rst | ||
recording/events.rst | ||
recording/example.rst | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
.. _api_record_audio: | ||
|
||
Audio Sink Interface | ||
======================== | ||
#include <comms/media/media_engine.h> | ||
|
||
The interface for handling incoming audio frames. Applications must set listener which override the virtual function. The | ||
:cpp:func:`handle_audio <dolbyio::comms::audio_sink::handle_audio>` function is called with the decoded PCM audio data. When inside a conference the | ||
C++ SDK will receive a single audio stream which contains the mixed audio of all unmuted participants within that conference. | ||
|
||
.. doxygenclass:: dolbyio::comms::audio_sink | ||
:project: C++ SDK | ||
|
||
See :ref:`recording_example` for a template for child recording class for all possible media. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
.. _api_record_encoded: | ||
|
||
Encoded Video Sink Interface | ||
============================ | ||
#include <comms/media/media_engine.h> | ||
|
||
The interface for handling encoded video frames. Application must set listeners which override both of the virtual functions. The | ||
:cpp:func:`handle_frame_encoded <dolbyio::comms::video_sink_encoded::handle_frame_encoded>` function is called after the RTP data is depacktized and an encoded frame | ||
is constructed. This function will be executed on the Webrtc DecodingQueue thread created for the track to which the encoded frames belong. | ||
|
||
.. doxygenclass:: dolbyio::comms::video_sink_encoded | ||
:project: C++ SDK | ||
|
||
See :ref:`recording_example` for a template for child recording class for all possible media. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
.. _recording_events: | ||
|
||
Media Recording Events | ||
====================== | ||
The events which are of interest to record media can all be accessed and subscribed through the :ref:`api_conference`. These :ref:`conf_events` contain information about participants joining and leaving the conference as well as audio and video track events emitted by Webrtc necessary to configure the recording pipelines. These events | ||
will indicate when video or audio tracks have been added or removed from the peer connection and from which peers streams are coming via the same. | ||
The Media Recording Module should set handlers for all of these events via the :cpp:func:`dolbyio::comms::services::conference::add_event_handler` methods. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
.. _recording_example: | ||
|
||
Example Recorder Implementation | ||
=============================== | ||
Below are the header and cpp files for a Media Recorder Implementation, in order for this recorder to be used by the C++ SDK it must be passed to the conference service as | ||
using the :cpp:func:`dolbyio::comms::services::conference::set_media_sink` method. Now when a conference is joined all incoming media will be passed to the recorder implementation. | ||
|
||
The recorder should be configured appropriately to be used for handling encoded or raw video and PCM/AAC audio. The application should set this configuration before passing the recorder implementation | ||
to the conference service. | ||
|
||
The interfaces inheritted from by this custom_recorder example must be implemented by any custom recorder which wishes to receive video and audio frames. | ||
|
||
At the bottom of the page there is sample CMakeLists.txt which can replace the one shown in the :ref:`getting_started_server` section if you want to use a custom recording module instead of our default one. | ||
This paths in this CMakeLists.txt are assuming that the file is located in the same place as in the Writing Application portion of Getting Started. | ||
|
||
custom_recorder.h | ||
_________________ | ||
.. literalinclude:: ../../../src/public/comms/sample/custom_recorder/custom_recorder.h | ||
:language: c++ | ||
|
||
custom_recorder.cc | ||
__________________ | ||
.. literalinclude:: ../../../src/public/comms/sample/custom_recorder/custom_recorder.cc | ||
:language: c++ | ||
|
||
Recorder CMakeLists.txt | ||
_______________________ | ||
.. literalinclude:: ../../../src/public/comms/sample/custom_recorder/CMakeLists.txt | ||
:language: console |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
.. _api_media_capturer_interface: | ||
|
||
Media Sink Interface | ||
========================= | ||
#include <comms/media/media_engine.h> | ||
|
||
Top level interface which exposes the underlying sink interfaces for receiving and handling audio, raw video and encoded video data. This abstract class just exposes | ||
the three possible sink interfaces: :cpp:class:`raw video <dolbyio::comms::video_sink_yuv>`, :cpp:class:`encoded video <dolbyio::comms::video_sink_encoded>` | ||
and :cpp:class:`audio <dolbyio::comms::audio_sink>`. | ||
|
||
.. doxygenclass:: dolbyio::comms::media_sink_interface | ||
:project: C++ SDK |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
.. _api_record_raw: | ||
|
||
Raw Video Sink Interface | ||
======================== | ||
#include <comms/media/media_engine.h> | ||
|
||
The interface for handling decoded video frames. Application must set listener which overrides the virtual function. The | ||
:cpp:func:`handle_frame <dolbyio::comms::video_sink_yuv::handle_frame>` function is called after the Webrtc Decoder has decoded the frame and will provide the | ||
decoded data wrapped in the :cpp:class:`frame <dolbyio::comms::frame>` object. The frame class wraps the decoded video frame and essentailly just provides interface | ||
to extract the underlying YUV data. | ||
|
||
.. doxygenclass:: dolbyio::comms::video_sink_yuv | ||
:project: C++ SDK | ||
|
||
.. doxygenclass:: dolbyio::comms::video_frame | ||
:project: C++ SDK | ||
|
||
See :ref:`recording_example` for a template for child recording class for all possible media. |
Oops, something went wrong.