From 89bd50e82d116cc2280463de94b47bb13a363ae4 Mon Sep 17 00:00:00 2001 From: Matt Wolenetz Date: Wed, 30 Mar 2022 12:55:47 -0700 Subject: [PATCH] Switch from src to srcObject for worker MSE attachment (#305) Adds text to require implementations to fail attachment using a worker MediaSource's object URL. Adds a MediaSourceHandle object and describes how to use it to attach a worker MediaSource to a media element. --- media-source-respec.html | 104 +++++++++++++++++++++++++++++++++------ 1 file changed, 88 insertions(+), 16 deletions(-) diff --git a/media-source-respec.html b/media-source-respec.html index 8d340a2..645689b 100644 --- a/media-source-respec.html +++ b/media-source-respec.html @@ -323,12 +323,13 @@

MediaSource Object

attribute EventHandler onsourceended; attribute EventHandler onsourceclose; static readonly attribute boolean canConstructInDedicatedWorker; - SourceBuffer addSourceBuffer (DOMString type); - undefined removeSourceBuffer (SourceBuffer sourceBuffer); - undefined endOfStream (optional EndOfStreamError error); - undefined setLiveSeekableRange (double start, double end); - undefined clearLiveSeekableRange (); - static boolean isTypeSupported (DOMString type); + SourceBuffer addSourceBuffer (DOMString type); + undefined removeSourceBuffer (SourceBuffer sourceBuffer); + undefined endOfStream (optional EndOfStreamError error); + undefined setLiveSeekableRange (double start, double end); + undefined clearLiveSeekableRange (); + MediaSourceHandle getHandle (); + static boolean isTypeSupported (DOMString type); };

Attributes

@@ -636,7 +637,25 @@

Attributes

  • If {{MediaSource/[[live seekable range]]}} contains a range, then set {{MediaSource/[[live seekable range]]}} to be a new empty {{TimeRanges}} object.
    • -
    No parameters.
    Return type: {{undefined}}
    isTypeSupported, static
    +
    No parameters.
    Return type: {{undefined}}
    + +
    getHandle
    +

    Creates a handle useful for attachment to an {{HTMLMediaElement}} via {{HTMLMediaElement/srcObject}}.

    +
      +
    1. If the implementation does not support creating a handle for this {{MediaSource}}, then throw a + {{NotSupportedError}} exception and abort these steps. +

      Implementations MAY choose to only allow handle creation for {{MediaSource}} objects in a + {{DedicatedWorkerGlobalScope}}, as a minimum requirement for enabling attachment of such a {{MediaSource}} + object to an {{HTMLMediaElement}}.

      +

      Consider requiring conforming implementations to let such handle be created on a {{Window}} + {{MediaSource}} object, too?

      +
      2. +
    2. Create a new {{MediaSourceHandle}} object and associated resources.
      3. +
    3. Return the new object.
      4. +
    +
    No parameters.
    Return type: {{MediaSourceHandle}}
    + +
    isTypeSupported, static

    Check to see whether the MediaSource is capable of creating SourceBuffer objects for the specified MIME type.

      @@ -695,9 +714,9 @@

      Cross-context communication model

      {{HTMLMediaElement}} is exposed only to {{Window}} contexts, but {{MediaSource}} and related objects defined in this specification are exposed in {{Window}} and {{DedicatedWorkerGlobalScope}} contexts. This lets applications construct a {{MediaSource}} object in either of those types of context and attach it to an - {{HTMLMediaElement}} object in a {{Window}} context using a [=MediaSource object URL=] as described in the - [=attaching to a media element=] algorithm. A {{MediaSource}} object is not {{Transferable}}; it is only visible - in the context where it was created.

      + {{HTMLMediaElement}} object in a {{Window}} context using a [=MediaSource object URL=] or a + {{MediaSourceHandle}} as described in the [=attaching to a media element=] algorithm. A {{MediaSource}} object + is not {{Transferable}}; it is only visible in the context where it was created.

      The rest of this section describes a model for bounding information latency for attachments of a {{Window}} media element to a {{DedicatedWorkerGlobalScope}} {{MediaSource}}. While the model describes communication using @@ -727,16 +746,48 @@

      Algorithms

      Attaching to a media element

      -

      A MediaSource object can be attached to a media element by assigning a [=MediaSource object URL=] to the media element {{HTMLMediaElement/src}} attribute or the src attribute of a <source> inside a media element. A [=MediaSource object URL=] is created by passing a MediaSource object to {{URL/createObjectURL()}}.

      +

      There are distinct mechanisms for attaching a {{MediaSource}} to a media element depending on where the {{MediaSource}} object was constructed, in a {{Window}} versus in a {{DedicatedWorkerGlobalScope}}:

      +
        +
      • +

        Attaching a {{MediaSource}} that was constructed in a {{Window}} can be done by assigning a + [=MediaSource object URL=] for that {{MediaSource}} to the media element {{HTMLMediaElement/src}} + attribute or the src attribute of a <source> inside a media element. A [=MediaSource object URL=] + is created by passing a MediaSource object to {{URL/createObjectURL()}}.

        +

        Though implementations MAY allow [=MediaSource object URL=] creation in a + {{DedicatedWorkerGlobalScope}} for a {{MediaSource}} constructed in that worker, attempting to use that + [=MediaSource object URL=] to attach to a media element using either the {{HTMLMediaElement/src}} + attribute or the src attribute of a <source> inside a media element MUST fail in the media + element's resource selection aglorithm, as extended below.

        +

        Extending the object URL attachment mechanism to worker MediaSource object URLs would + further propagate this idiom that is less preferred versus using srcObject, and would unnecessarily + increase user agent interoperability risk and implementation complexity.

        +
        • +
      • Attaching a {{MediaSource}} that was constructed in a {{DedicatedWorkerGlobalScope}} can be done by + obtaining a handle from it using {{MediaSource/getHandle}}, transferring that {{MediaSourceHandle}} to the + {{Window}} context and assigning it to the media element {{HTMLMediaElement/srcObject}} attribute. For the + purposes of aligning this specification with {{HTMLMediaElement}} resource loading and fetching + algorithms, the underlying {{DedicatedWorkerGlobalScope}} {{MediaSource}} is the MediaSource + object mentioned there, and the {{MediaSourceHandle}} object is the media provider object. +

        MediaSourceHandle needs to be added to HTMLMediaElement's MediaProvider IDL typedef and + related text involving media provider objects.

        +
        • +
      -

      If the was invoked with - a media provider object that is a MediaSource object or a URL record whose object is a MediaSource object, - then let mode be local, skip the first step in the - (which may otherwise set mode to remote) and add the steps and clarifications below to the section of the . +

      If the was invoked with a media provider object that is a + {{MediaSource}} object, a {{MediaSourceHandle}} object or a URL record whose object is a {{MediaSource}} + object, then let mode be local, skip the first step in the + (which may otherwise set mode to remote) and add the steps and clarifications below to the section of the .

      The 's first step is expected to eventually align with selecting local mode for URL records whose objects are media provider objects. The intent is that if the HTMLMediaElement's src attribute or selected child <source>'s src attribute is a blob: URL matching a [=MediaSource object URL=] when the respective src attribute was last changed, then that MediaSource object is used as the media provider object and current media resource in the local mode logic in the . This also means that the remote mode logic that includes observance of any preload attribute is skipped when a MediaSource object is attached. Even with that eventual change to [[HTML]], the execution of the following steps at the beginning of the local mode logic is still required when the current media resource is a MediaSource object.

      Relative to the action which triggered the media element's resource selection algorithm, these steps are asynchronous. The resource fetch algorithm is run after the task that invoked the resource selection algorithm is allowed to continue and a stable state is reached. Implementations may delay the steps in the "Otherwise" clause, below, until the MediaSource object is ready for use.

      -
      If {{MediaSource/readyState}} is NOT set to {{ReadyState/""closed""}}
      +
      If the media provider object is + a URL record whose object is a {{MediaSource}} that was constructed in a {{DedicatedWorkerGlobalScope}}, + such as would occur if attempting to use a [=MediaSource object URL=] from a + {{DedicatedWorkerGlobalScope}} {{MediaSource}} +
      +
      Run the steps of the 's .
      +
      If {{MediaSource/readyState}} is NOT set to {{ReadyState/""closed""}}
      Run the steps of the 's .
      Otherwise
      @@ -1100,6 +1151,27 @@

      Mirror if necessary

    +
    +

    MediaSourceHandle Object

    +

    The MediaSourceHandle object represents a proxy for a {{MediaSource}} object that is useful for attaching a + {{DedicatedWorkerGlobalScope}} {{MediaSource}} to a {{Window}} {{HTMLMediaElement}} using + {{HTMLMediaElement/srcObject}} as described in the [=attaching to a media element=] algorithm.

    + +

    This distinct object is necessary to attach a cross-context {{MediaSource}} to a media element + because {{MediaSource}} objects themselves are not transferable since they are event targets.

    + + 
    +[Exposed=(Window,DedicatedWorker)]
+interface MediaSourceHandle {};
    + +

    Transfer

    + The {{MediaSourceHandle}} [=transfer steps=] and [=transfer-receiving steps=] require the implementation to + maintain an implicit internal slot referencing the underlying {{MediaSource}} to enable [=attaching to a media + element=] using {{HTMLMediaElement/srcObject}} and consequent setup of an attachment's [=cross-context + communication model=]. +
    +
    +

    SourceBuffer Object