Switch from src to srcObject for worker MSE attachment #305

wolenetz · 2022-03-09T22:48:27Z

Adds text to require implementations to fail attachment using a worker
MediaSource's object URL.

Adds a MediaSourceHandle object and describes how to use it to attach a
worker MediaSource to a media element.

wolenetz · 2022-03-09T22:49:26Z

Note that this PR is currently based upon PR #304. Once #304 is reviewed, approved and merged, I'll rebase this.

wolenetz · 2022-03-09T22:50:58Z

@mwatson2 - please review this and #304 (#304 is what this PR is based upon currently. See f9df82e for the portion of this PR that is intended to be specific to this PR once #304 is landed and this PR is rebased onto main.

karlt · 2022-03-10T02:07:38Z

media-source-respec.html

+            <p class="issue">Consider requiring conforming implementations to let such handle be created on a {{Window}}
+              {{MediaSource}} object, too?</p>
+          </li>
+          <li>Create a new {{MediaSourceHandle}} object and associated resources.</li>


In my worldview, the algorithm would throw before this point if a MediaSourceHandle has already been returned by a previous getHandle() call on this MediaSource.

That's because I haven't thought of any better way to prevent "Attaching to a media element" happening for two media elements concurrently.

That clarification need not block this PR, if you'd like to deal with this separately.

Hmm. In the objectURL main-thread current way, you can obtain multiple object URLs for the same MediaSource. If you then attempt synchronous start of attachment to multiple media elements using either one of these multiple times, or the variety of these urls, at most one may succeed.

IOW, the current mechanism allows for multiple urls and simultaneous attempts to attach to the underlying mediasource, but doesn't let more than one of these succeed per underlying mediasource.

With auto-url-revocation upon successful attachment start of MSE object urls logic (as was added in Chrome), a single MSE objectURL cannot be successfully used in multiple attachments.

w.r.t. the new handle for attachment, I'd like to understand your concern better:

Would a single handle for a single MediaSource be allowed to attach more than once (assuming the underlying MediaSource returns to "closed" readyState between the attachments as per the detach logic)? Or should a single handle for a single MediaSource be allowed at most one successful attachment?

Would multiple distinct handles be retrievable for the same underlying MediaSource, and be usable for attachment so long as, at the attachment attempt time, the underlying MediaSource is "closed"?

(These questions are orthogonal to the further restriction that implementations are not yet required to support main thread MediaSource handle's attachment success ever, though they MAY in the current draft.)

IOW, the current mechanism allows for multiple urls and simultaneous attempts to attach to the underlying mediasource, but doesn't let more than one of these succeed per underlying mediasource.

I'm assuming the goal is not to support the concurrent connection of multiple media elements to a single MediaSource. (Perhaps this could be achieved, given the largely one-directional flow of information from MediaSource to HTMLMediaElement, but the MediaSource#readyState model at least would need some adjustment.)

You draw an accurate distinction between multiple urls and my expectation of only a single handle. The main-thread has the distinction of no concurrent operations and the ease of synchronously checking for an existing attachment.

With auto-url-revocation upon successful attachment start of MSE object urls logic (as was added in Chrome), a single MSE objectURL cannot be successfully used in multiple attachments.

I guess this model would depend on a global mutex, even shared across processes.
I'd prefer to aim for something more deterministic that could be implemented with a message-passing design, if that would meet the needs.

Would a single handle for a single MediaSource be allowed to attach more than once (assuming the underlying MediaSource returns to "closed" readyState between the attachments as per the detach logic)? Or should a single handle for a single MediaSource be allowed at most one successful attachment?

I'm anticipating that a single MediaSourceHandle would be allowed to be attached to at most one media element at any one time.

I haven't put too much thought into permitting detaching from one media element and reattaching to another, but I guess this could be supported, if desired.

Would multiple distinct handles be retrievable for the same underlying MediaSource, and be usable for attachment so long as, at the attachment attempt time, the underlying MediaSource is "closed"?

I'm expecting not. That's because of the difficultly in deterministically preventing "Attaching to a media element" when another attachment might be concurrently in progress. Each such handle could be transferred to a different realm, having a different event loop, and so attachment attempts could occur concurrently.

Even if two handles were transferred to the same realm, the asynchronicity of 'Set the readyState to "open"' means that srcObject might be set on two media elements before readyState changes. I dare say that attachment failure could be reported asynchronously, but this would not be deterministic in the case of realms having different event loops, and API that prevented such failure attempts or caught them synchronously (or even as a promise rejection, if necessary) would be preferable to async error events.

Simultaneous, concurrent usage of the same underlying MediaSource with different media elements is certainly not the intent.

Given we have this opportunity to reduce developer confusion about this intent (versus the more general object URL creation/use idiom), I tend to agree that simplifying to only allow one handle per MediaSource would be a good idea.

However, I'm less clear about how to specify what happens once such handle is transferred from worker context to main context: should the mediaSource.getHandle() method fail (in worker) with some kind of exception?

Note, that the handle itself could know whether or not it is currently involved in an attachment to a media element, such that the MSE-specific spec text for attaching and detaching such a handle can precisely ensure that at most one element can currently have the handle as the "assigned media object" (or otherwise do the appropriate attachment failure steps on the attempts to attach an already attached handle).

This would mirror the expected behavior for eventual specification of support for main thread-owned MediaSource attachments to a media element via srcObject (not in scope of this PR work): such a main thread MediaSource could only ever be attached successfully if it isn't already a media provider object (or already being used in an attachment via the dereferenced MediaSource if attached by a main-thread-originated MediaSource objectURL).

@karlt If you agree with this approach, I'll land this PR and create a follow-up that employs this restriction (since my co-editor has already approved this PR). (Heads-up, @mwatson2 ). Also, thank you both for your assistance on this.

Given we have this opportunity to reduce developer confusion about this intent (versus the more general object URL creation/use idiom),

That's an excellent explanation.

However, I'm less clear about how to specify what happens once such handle is transferred from worker context to main context: should the mediaSource.getHandle() method fail (in worker) with some kind of exception?

Yes, I would expect mediaSource.getHandle() to throw when called a second time, and continue to throw after the handle from the first call has been transferred.

When transferred from Worker to Window, the handle in worker context would become neutered, if that is the right term, so that it could not be transferred to multiple recipients.

Note, that the handle itself could know whether or not it is currently involved in an attachment to a media element, such that the MSE-specific spec text for attaching and detaching such a handle can precisely ensure that at most one element can currently have the handle as the "assigned media object" (or otherwise do the appropriate attachment failure steps on the attempts to attach an already attached handle).

Yes, that should be very workable, and would be a simple and clear concept for developers to follow.

@karlt If you agree with this approach, I'll land this PR and create a follow-up that employs this restriction (since my co-editor has already approved this PR).

Yes, please go ahead and land this. Thank you for writing it up.

@karlt thank you for your comments. I'll land this now.

But first I'll rebase over the already merged prerequisite.

Adds text to require implementations to fail attachment using a worker MediaSource's object URL. Adds a MediaSourceHandle object and describes how to use it to attach a worker MediaSource to a media element.

@karlt

Adds text intending to help prevent using one or more `MediaSourceHandle` for the same underlying `MediaSource` for simultaneous, concurrent attachments to media elements: * Updates `MediaSource.getHandle()` to succeed at most once per `MediaSource` instance. * Updates `MediaSourceHandle` transfer section to more clearly indicate that transferring a handle out of a context makes that handle unavailable for transferring again out of the same context (unless it is first transferred back to that context). * Updates the "attaching to a media element" text to indicate that a `MediaSourceHandle` can be attached successfully to at most one media element at a time, and with the previous 2 clarifications, ensures that the underlying worker context `MediaSource` can only be attached successfully to at most one media element at a time. Note that legacy MSE object URL attachments similarly already achieve the same intent in existing implementations. The switch to more clear `srcObject` usage for worker MSE attachment affords the spec an opportunity to be more clear about this intent. This refinement originates from discussion on the previous PR with @karlt: w3c#305

@karlt

Adds text intending to help prevent using one or more `MediaSourceHandle` for the same underlying `MediaSource` for simultaneous, concurrent attachments to media elements: * Updates `MediaSource.getHandle()` to succeed at most once per `MediaSource` instance. * Updates `MediaSourceHandle` transfer section to more clearly indicate that transferring a handle out of a context makes that handle unavailable for transferring again out of the same context (unless it is first transferred back to that context). * Updates the "attaching to a media element" text to indicate that a `MediaSourceHandle` can be attached successfully to at most one media element at a time, and with the previous 2 clarifications, ensures that the underlying worker context `MediaSource` can only be attached successfully to at most one media element at a time. Note that legacy MSE object URL attachments similarly already achieve the same intent in existing implementations. The switch to more clear `srcObject` usage for worker MSE attachment affords the spec an opportunity to be more clear about this intent. This refinement originates from discussion on the previous PR with @karlt: w3c#305

@karlt

Adds text intending to help prevent using one or more `MediaSourceHandle` for the same underlying `MediaSource` for simultaneous, concurrent attachments to media elements: * Updates `MediaSource.getHandle()` to succeed at most once per `MediaSource` instance. * Updates `MediaSourceHandle` transfer section to more clearly indicate that transferring a handle out of a context makes that handle unavailable for transferring again out of the same context (unless it is first transferred back to that context). * Updates the "attaching to a media element" text to indicate that a `MediaSourceHandle` can be attached successfully to at most one media element at a time, and with the previous 2 clarifications, ensures that the underlying worker context `MediaSource` can only be attached successfully to at most one media element at a time. Note that legacy MSE object URL attachments similarly already achieve the same intent in existing implementations. The switch to more clear `srcObject` usage for worker MSE attachment affords the spec an opportunity to be more clear about this intent. This refinement originates from discussion on the previous PR with @karlt: w3c#305

@karlt

Adds text intending to help prevent using one or more `MediaSourceHandle` for the same underlying `MediaSource` for simultaneous, concurrent attachments to media elements: * Updates `MediaSource.getHandle()` to succeed at most once per `MediaSource` instance. * Updates `MediaSourceHandle` transfer section to more clearly indicate that transferring a handle out of a context makes that handle unavailable for transferring again out of the same context (unless it is first transferred back to that context). * Updates the "attaching to a media element" text to indicate that a `MediaSourceHandle` can be attached successfully to at most one media element at a time, and with the previous 2 clarifications, ensures that the underlying worker context `MediaSource` can only be attached successfully to at most one media element at a time. Note that legacy MSE object URL attachments similarly already achieve the same intent in existing implementations. The switch to more clear `srcObject` usage for worker MSE attachment affords the spec an opportunity to be more clear about this intent. This refinement originates from discussion on the previous PR with @karlt: w3c#305

@karlt

Adds text intending to help prevent using one or more `MediaSourceHandle` for the same underlying `MediaSource` for simultaneous, concurrent attachments to media elements: * Updates `MediaSource.getHandle()` to succeed at most once per `MediaSource` instance. * Updates `MediaSourceHandle` transfer section to more clearly indicate that transferring a handle out of a context makes that handle unavailable for transferring again out of the same context (unless it is first transferred back to that context). * Updates the "attaching to a media element" text to indicate that a `MediaSourceHandle` can be attached successfully to at most one media element at a time, and with the previous 2 clarifications, ensures that the underlying worker context `MediaSource` can only be attached successfully to at most one media element at a time. Note that legacy MSE object URL attachments similarly already achieve the same intent in existing implementations. This change includes a clarification for those similar to the third bullet, but for a `MediaSource` in the "attaching to a media element" text, since it was never the intent even for main-thread `MediaSource` attachments for them to be successfully attached to more than one `HTMLMediaElement` at a time. The switch to more clear `srcObject` usage for worker MSE attachment affords the spec an opportunity to be more clear about this intent. This refinement originates from discussion on the previous PR with @karlt: w3c#305 This work is associated with the MSE-in-Worker spec issue: w3c#175

Adds core MediaSourceHandle interface and modules implementation. Later changes will enable apps to get a handle from a dedicated worker MediaSource, post it to the main thread and attach it to an HTMLMediaElement via the srcObject attribute; they will also include test updates. References: Full prototype CL: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) BUG=878133 Change-Id: I9a3b47ea02d2755a9860999e245eafc26f864977 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3687146 Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Kentaro Hara <haraken@chromium.org> Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012115}

Adds ability for apps to get a MediaSourceHandle from a dedicated worker MediaSource instance, gated by RuntimeEnabledFeature "MediaSourceInWorkersUsingHandle". Updates the PassKey required for creation of a concrete MediaSourceAttachmentSupplement to allow that only from either URLMediaSource::createObjectURL or the new MediaSource::getHandle method. This specificity is enabled by a new AttachmentCreationPassKeyProvider type. Later changes will enable apps to post the handle to the main thread and attach it to an HTMLMediaElement via the srcObject attribute; they will also include test updates. References: Full prototype CL: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) crbug.com/506273 is somewhat related, but not fixed by this change (it refers to directly setting MediaSource on Window context to the media element's srcObject attribute.) This change sequence is specific to enabling MSE-in-Worker attachment via srcObject using a handle object. BUG=878133,506273 Change-Id: Ic61f4cc4193080bdbc39234b98897d9a789778d6 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3688613 Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Will Cassella <cassew@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012129}

Adds ability to serialize a MediaSourceHandleImpl, conditionally failing with DataCloneError if the handle is_serialized() already or if the handle is_used() (both of those are initialized to be false in the new instance created during deserialization.) Later changes will enable apps to attach the handle to a main thread HTMLMediaElement via the srcObject attribute; they will also include test updates. References: Full prototype CL: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) BUG=878133 Change-Id: I19cc8a450423964aef78e8e468776e8cd912503a Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3688599 Reviewed-by: Will Cassella <cassew@chromium.org> Reviewed-by: Kentaro Hara <haraken@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012132}

Adds ability to attach a dedicated-worker-owned MediaSource to an HTMLMediaElement using a MediaSourceHandle for that MediaSource instance that the app has posted over to the main thread and assigned to the srcObject attribute on the media element. Previously, only a MediaStream could be set as a srcObject in Chrome. This change adds the relevant MediaProvider IDL union type and makes it the type of that srcObject attribute. For historical reasons, the srcObject attribute of an HTMLMediaElement is implemented in modules as a partial interface; this change updates that partial interface as well as the media element implementation to enable the scenario without regressing existing ability to use MediaStream as media element's srcObject. The attachment of the worker MediaSource using the handle does not involve lookup into a URL registry; rather, an internal blob URL is built into the handle when it was retrieved from the MediaSource and that URL is used to satisfy existing media element load safety checks as well as provide the underlying WebMediaPlayer a URL that it can use when logging to devtools. Later changes will add further test updates and remove the previous ability to attach a worker MediaSource using a registered object URL. References: Full prototype CL: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) crbug.com/506273 is somewhat related, but not fixed by this change (it refers to directly setting MediaSource on Window context to the media element's srcObject attribute.) This change sequence is specific to enabling MSE-in-Worker attachment via srcObject using a handle object. BUG=878133,506273 Change-Id: I86cddd87bafae1c7cbae9e94ea4614418067012f Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3688740 Reviewed-by: Kentaro Hara <haraken@chromium.org> Reviewed-by: Elad Alon <eladalon@chromium.org> Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012202}

If the MediaSourceInWorkersUsingHandle feature is enabled, this change prevents successful ability of obtaining an objectURL that would succeed in loading a worker-owned MediaSource. It changes the wpt tests to use handle for attachment and verifies expected new behavior of getHandle and that worker objectURL attachment fails (these tests run on experimental builds of Chromium with currently-experimental MediaSourceInWorkersUsingHandle feature enabled, just like the currently-experimental MediaSourceInWorkers feature.) References: Full prototype CL for the parts 1-4 that have already landed: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) BUG=878133 Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3

If the MediaSourceInWorkersUsingHandle feature is enabled, this change prevents successful ability of obtaining an objectURL that would succeed in loading a worker-owned MediaSource. It changes the wpt tests to use handle for attachment and verifies expected new behavior of getHandle and that worker objectURL attachment fails (these tests run on experimental builds of Chromium with currently-experimental MediaSourceInWorkersUsingHandle feature enabled, just like the currently-experimental MediaSourceInWorkers feature.) References: Full prototype CL for the parts 1-4 that have already landed: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) BUG=878133 Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Will Cassella <cassew@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012712}

…bjectURL" This reverts commit 6315549. Reason for revert: This is causing failures on the WebKit Linux Leak builder i.e. https://ci.chromium.org/ui/p/chromium/builders/ci/WebKit%20Linux%20Leak/39394/overview Original change's description: > MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL > > If the MediaSourceInWorkersUsingHandle feature is enabled, this change > prevents successful ability of obtaining an objectURL that would succeed > in loading a worker-owned MediaSource. > > It changes the wpt tests to use handle for attachment and verifies > expected new behavior of getHandle and that worker objectURL attachment > fails (these tests run on experimental builds of Chromium with > currently-experimental MediaSourceInWorkersUsingHandle feature enabled, > just like the currently-experimental MediaSourceInWorkers feature.) > > References: > Full prototype CL for the parts 1-4 that have already landed: > https://chromium-review.googlesource.com/c/chromium/src/+/3515334 > MSE spec issue: > w3c/media-source#175 > MSE spec feature updates switching from worker MSE attachment via > object URL to attachment via srcObject MediaSourceHandle: > * w3c/media-source#305 > * further clarifications in discussion at > w3c/media-source#306 (comment) > > BUG=878133 > > Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> > Reviewed-by: Will Cassella <cassew@chromium.org> > Cr-Commit-Position: refs/heads/main@{#1012712} Bug: 878133 Change-Id: I1e405ae1de146d1f3183592b00a43bd3c38d849d No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3695890 Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Commit-Queue: Nidhi Jaju <nidhijaju@chromium.org> Owners-Override: Nidhi Jaju <nidhijaju@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012823}

…bjectURL" This reverts commit 6315549b8c2ece3dbbf3062c1a87347589a5e115. Reason for revert: This is causing failures on the WebKit Linux Leak builder i.e. https://ci.chromium.org/ui/p/chromium/builders/ci/WebKit%20Linux%20Leak/39394/overview Original change's description: > MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL > > If the MediaSourceInWorkersUsingHandle feature is enabled, this change > prevents successful ability of obtaining an objectURL that would succeed > in loading a worker-owned MediaSource. > > It changes the wpt tests to use handle for attachment and verifies > expected new behavior of getHandle and that worker objectURL attachment > fails (these tests run on experimental builds of Chromium with > currently-experimental MediaSourceInWorkersUsingHandle feature enabled, > just like the currently-experimental MediaSourceInWorkers feature.) > > References: > Full prototype CL for the parts 1-4 that have already landed: > https://chromium-review.googlesource.com/c/chromium/src/+/3515334 > MSE spec issue: > w3c/media-source#175 > MSE spec feature updates switching from worker MSE attachment via > object URL to attachment via srcObject MediaSourceHandle: > * w3c/media-source#305 > * further clarifications in discussion at > w3c/media-source#306 (comment) > > BUG=878133 > > Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> > Reviewed-by: Will Cassella <cassew@chromium.org> > Cr-Commit-Position: refs/heads/main@{#1012712} Bug: 878133 Change-Id: I1e405ae1de146d1f3183592b00a43bd3c38d849d No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3695890 Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Commit-Queue: Nidhi Jaju <nidhijaju@chromium.org> Owners-Override: Nidhi Jaju <nidhijaju@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012823}

…ionally fail worker objectURL, a=testonly Automatic update from web-platform-tests MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL If the MediaSourceInWorkersUsingHandle feature is enabled, this change prevents successful ability of obtaining an objectURL that would succeed in loading a worker-owned MediaSource. It changes the wpt tests to use handle for attachment and verifies expected new behavior of getHandle and that worker objectURL attachment fails (these tests run on experimental builds of Chromium with currently-experimental MediaSourceInWorkersUsingHandle feature enabled, just like the currently-experimental MediaSourceInWorkers feature.) References: Full prototype CL for the parts 1-4 that have already landed: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) BUG=878133 Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Will Cassella <cassew@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012712} -- wpt-commits: 3fec1e386568b78c390acee2307b5a07b75e3d30 wpt-pr: 34364

…: Conditionally fail worker objectURL", a=testonly Automatic update from web-platform-tests Revert "MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL" This reverts commit 6315549b8c2ece3dbbf3062c1a87347589a5e115. Reason for revert: This is causing failures on the WebKit Linux Leak builder i.e. https://ci.chromium.org/ui/p/chromium/builders/ci/WebKit%20Linux%20Leak/39394/overview Original change's description: > MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL > > If the MediaSourceInWorkersUsingHandle feature is enabled, this change > prevents successful ability of obtaining an objectURL that would succeed > in loading a worker-owned MediaSource. > > It changes the wpt tests to use handle for attachment and verifies > expected new behavior of getHandle and that worker objectURL attachment > fails (these tests run on experimental builds of Chromium with > currently-experimental MediaSourceInWorkersUsingHandle feature enabled, > just like the currently-experimental MediaSourceInWorkers feature.) > > References: > Full prototype CL for the parts 1-4 that have already landed: > https://chromium-review.googlesource.com/c/chromium/src/+/3515334 > MSE spec issue: > w3c/media-source#175 > MSE spec feature updates switching from worker MSE attachment via > object URL to attachment via srcObject MediaSourceHandle: > * w3c/media-source#305 > * further clarifications in discussion at > w3c/media-source#306 (comment) > > BUG=878133 > > Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> > Reviewed-by: Will Cassella <cassew@chromium.org> > Cr-Commit-Position: refs/heads/main@{#1012712} Bug: 878133 Change-Id: I1e405ae1de146d1f3183592b00a43bd3c38d849d No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3695890 Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Commit-Queue: Nidhi Jaju <nidhijaju@chromium.org> Owners-Override: Nidhi Jaju <nidhijaju@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012823} -- wpt-commits: 9589b7d7210eb65f30af294f454912a5e90a7d53 wpt-pr: 34371

…ionally fail worker objectURL, a=testonly Automatic update from web-platform-tests MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL If the MediaSourceInWorkersUsingHandle feature is enabled, this change prevents successful ability of obtaining an objectURL that would succeed in loading a worker-owned MediaSource. It changes the wpt tests to use handle for attachment and verifies expected new behavior of getHandle and that worker objectURL attachment fails (these tests run on experimental builds of Chromium with currently-experimental MediaSourceInWorkersUsingHandle feature enabled, just like the currently-experimental MediaSourceInWorkers feature.) References: Full prototype CL for the parts 1-4 that have already landed: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) BUG=878133 Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Will Cassella <cassew@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012712} -- wpt-commits: 3fec1e386568b78c390acee2307b5a07b75e3d30 wpt-pr: 34364

…: Conditionally fail worker objectURL", a=testonly Automatic update from web-platform-tests Revert "MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL" This reverts commit 6315549b8c2ece3dbbf3062c1a87347589a5e115. Reason for revert: This is causing failures on the WebKit Linux Leak builder i.e. https://ci.chromium.org/ui/p/chromium/builders/ci/WebKit%20Linux%20Leak/39394/overview Original change's description: > MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL > > If the MediaSourceInWorkersUsingHandle feature is enabled, this change > prevents successful ability of obtaining an objectURL that would succeed > in loading a worker-owned MediaSource. > > It changes the wpt tests to use handle for attachment and verifies > expected new behavior of getHandle and that worker objectURL attachment > fails (these tests run on experimental builds of Chromium with > currently-experimental MediaSourceInWorkersUsingHandle feature enabled, > just like the currently-experimental MediaSourceInWorkers feature.) > > References: > Full prototype CL for the parts 1-4 that have already landed: > https://chromium-review.googlesource.com/c/chromium/src/+/3515334 > MSE spec issue: > w3c/media-source#175 > MSE spec feature updates switching from worker MSE attachment via > object URL to attachment via srcObject MediaSourceHandle: > * w3c/media-source#305 > * further clarifications in discussion at > w3c/media-source#306 (comment) > > BUG=878133 > > Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> > Reviewed-by: Will Cassella <cassew@chromium.org> > Cr-Commit-Position: refs/heads/main@{#1012712} Bug: 878133 Change-Id: I1e405ae1de146d1f3183592b00a43bd3c38d849d No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3695890 Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Commit-Queue: Nidhi Jaju <nidhijaju@chromium.org> Owners-Override: Nidhi Jaju <nidhijaju@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012823} -- wpt-commits: 9589b7d7210eb65f30af294f454912a5e90a7d53 wpt-pr: 34371

…bjectURL With the underlying leak fixed by https://chromium-review.googlesource.com/c/chromium/src/+/3704191, this change can be relanded now. Previous revert: > Revert "MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL" > > This reverts commit 6315549b8c2ece3dbbf3062c1a87347589a5e115. > > Reason for revert: This is causing failures on the WebKit Linux Leak builder > i.e. https://ci.chromium.org/ui/p/chromium/builders/ci/WebKit%20Linux%20Leak/39394/overview > > Original change's description: > > MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL > > > > If the MediaSourceInWorkersUsingHandle feature is enabled, this change > > prevents successful ability of obtaining an objectURL that would succeed > > in loading a worker-owned MediaSource. > > > > It changes the wpt tests to use handle for attachment and verifies > > expected new behavior of getHandle and that worker objectURL attachment > > fails (these tests run on experimental builds of Chromium with > > currently-experimental MediaSourceInWorkersUsingHandle feature enabled, > > just like the currently-experimental MediaSourceInWorkers feature.) > > > > References: > > Full prototype CL for the parts 1-4 that have already landed: > > https://chromium-review.googlesource.com/c/chromium/src/+/3515334 > > MSE spec issue: > > w3c/media-source#175 > > MSE spec feature updates switching from worker MSE attachment via > > object URL to attachment via srcObject MediaSourceHandle: > > * w3c/media-source#305 > > * further clarifications in discussion at > > w3c/media-source#306 (comment) > > > > BUG=878133 > > > > Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 > > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > > Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> > > Reviewed-by: Will Cassella <cassew@chromium.org> > > Cr-Commit-Position: refs/heads/main@{#1012712} > > Bug: 878133 > Change-Id: I1e405ae1de146d1f3183592b00a43bd3c38d849d > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3695890 > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> > Commit-Queue: Nidhi Jaju <nidhijaju@chromium.org> > Owners-Override: Nidhi Jaju <nidhijaju@chromium.org> > Cr-Commit-Position: refs/heads/main@{#1012823} Bug: 878133 Change-Id: I56e4ecd4d8b58d9d58ed3c575b0fb52f596b6fae

…bjectURL With the underlying leak fixed by https://chromium-review.googlesource.com/c/chromium/src/+/3704191, this change can be relanded now. Previous revert: > Revert "MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL" > > This reverts commit 6315549. > > Reason for revert: This is causing failures on the WebKit Linux Leak builder > i.e. https://ci.chromium.org/ui/p/chromium/builders/ci/WebKit%20Linux%20Leak/39394/overview > > Original change's description: > > MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL > > > > If the MediaSourceInWorkersUsingHandle feature is enabled, this change > > prevents successful ability of obtaining an objectURL that would succeed > > in loading a worker-owned MediaSource. > > > > It changes the wpt tests to use handle for attachment and verifies > > expected new behavior of getHandle and that worker objectURL attachment > > fails (these tests run on experimental builds of Chromium with > > currently-experimental MediaSourceInWorkersUsingHandle feature enabled, > > just like the currently-experimental MediaSourceInWorkers feature.) > > > > References: > > Full prototype CL for the parts 1-4 that have already landed: > > https://chromium-review.googlesource.com/c/chromium/src/+/3515334 > > MSE spec issue: > > w3c/media-source#175 > > MSE spec feature updates switching from worker MSE attachment via > > object URL to attachment via srcObject MediaSourceHandle: > > * w3c/media-source#305 > > * further clarifications in discussion at > > w3c/media-source#306 (comment) > > > > BUG=878133 > > > > Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 > > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > > Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> > > Reviewed-by: Will Cassella <cassew@chromium.org> > > Cr-Commit-Position: refs/heads/main@{#1012712} > > Bug: 878133 > Change-Id: I1e405ae1de146d1f3183592b00a43bd3c38d849d > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3695890 > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> > Commit-Queue: Nidhi Jaju <nidhijaju@chromium.org> > Owners-Override: Nidhi Jaju <nidhijaju@chromium.org> > Cr-Commit-Position: refs/heads/main@{#1012823} Bug: 878133 Change-Id: I56e4ecd4d8b58d9d58ed3c575b0fb52f596b6fae Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3708465 Reviewed-by: Will Cassella <cassew@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/main@{#1014755}

…bjectURL With the underlying leak fixed by https://chromium-review.googlesource.com/c/chromium/src/+/3704191, this change can be relanded now. Previous revert: > Revert "MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL" > > This reverts commit 6315549b8c2ece3dbbf3062c1a87347589a5e115. > > Reason for revert: This is causing failures on the WebKit Linux Leak builder > i.e. https://ci.chromium.org/ui/p/chromium/builders/ci/WebKit%20Linux%20Leak/39394/overview > > Original change's description: > > MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL > > > > If the MediaSourceInWorkersUsingHandle feature is enabled, this change > > prevents successful ability of obtaining an objectURL that would succeed > > in loading a worker-owned MediaSource. > > > > It changes the wpt tests to use handle for attachment and verifies > > expected new behavior of getHandle and that worker objectURL attachment > > fails (these tests run on experimental builds of Chromium with > > currently-experimental MediaSourceInWorkersUsingHandle feature enabled, > > just like the currently-experimental MediaSourceInWorkers feature.) > > > > References: > > Full prototype CL for the parts 1-4 that have already landed: > > https://chromium-review.googlesource.com/c/chromium/src/+/3515334 > > MSE spec issue: > > w3c/media-source#175 > > MSE spec feature updates switching from worker MSE attachment via > > object URL to attachment via srcObject MediaSourceHandle: > > * w3c/media-source#305 > > * further clarifications in discussion at > > w3c/media-source#306 (comment) > > > > BUG=878133 > > > > Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 > > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > > Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> > > Reviewed-by: Will Cassella <cassew@chromium.org> > > Cr-Commit-Position: refs/heads/main@{#1012712} > > Bug: 878133 > Change-Id: I1e405ae1de146d1f3183592b00a43bd3c38d849d > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3695890 > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> > Commit-Queue: Nidhi Jaju <nidhijaju@chromium.org> > Owners-Override: Nidhi Jaju <nidhijaju@chromium.org> > Cr-Commit-Position: refs/heads/main@{#1012823} Bug: 878133 Change-Id: I56e4ecd4d8b58d9d58ed3c575b0fb52f596b6fae Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3708465 Reviewed-by: Will Cassella <cassew@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/main@{#1014755}

…: Conditionally fail worker objectURL, a=testonly Automatic update from web-platform-tests Reland: MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL With the underlying leak fixed by https://chromium-review.googlesource.com/c/chromium/src/+/3704191, this change can be relanded now. Previous revert: > Revert "MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL" > > This reverts commit 6315549b8c2ece3dbbf3062c1a87347589a5e115. > > Reason for revert: This is causing failures on the WebKit Linux Leak builder > i.e. https://ci.chromium.org/ui/p/chromium/builders/ci/WebKit%20Linux%20Leak/39394/overview > > Original change's description: > > MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL > > > > If the MediaSourceInWorkersUsingHandle feature is enabled, this change > > prevents successful ability of obtaining an objectURL that would succeed > > in loading a worker-owned MediaSource. > > > > It changes the wpt tests to use handle for attachment and verifies > > expected new behavior of getHandle and that worker objectURL attachment > > fails (these tests run on experimental builds of Chromium with > > currently-experimental MediaSourceInWorkersUsingHandle feature enabled, > > just like the currently-experimental MediaSourceInWorkers feature.) > > > > References: > > Full prototype CL for the parts 1-4 that have already landed: > > https://chromium-review.googlesource.com/c/chromium/src/+/3515334 > > MSE spec issue: > > w3c/media-source#175 > > MSE spec feature updates switching from worker MSE attachment via > > object URL to attachment via srcObject MediaSourceHandle: > > * w3c/media-source#305 > > * further clarifications in discussion at > > w3c/media-source#306 (comment) > > > > BUG=878133 > > > > Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 > > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > > Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> > > Reviewed-by: Will Cassella <cassew@chromium.org> > > Cr-Commit-Position: refs/heads/main@{#1012712} > > Bug: 878133 > Change-Id: I1e405ae1de146d1f3183592b00a43bd3c38d849d > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3695890 > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> > Commit-Queue: Nidhi Jaju <nidhijaju@chromium.org> > Owners-Override: Nidhi Jaju <nidhijaju@chromium.org> > Cr-Commit-Position: refs/heads/main@{#1012823} Bug: 878133 Change-Id: I56e4ecd4d8b58d9d58ed3c575b0fb52f596b6fae Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3708465 Reviewed-by: Will Cassella <cassew@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/main@{#1014755} -- wpt-commits: 8e5b88e97b620159b1275a548f090ccc46c3529f wpt-pr: 34451

@karlt

Adds text intending to help prevent using one or more `MediaSourceHandle` for the same underlying `MediaSource` for simultaneous, concurrent attachments to media elements: * Updates `MediaSource.getHandle()` to succeed at most once per `MediaSource` instance. * Updates `MediaSourceHandle` transfer section to more clearly indicate that transferring a handle out of a context makes that handle unavailable for transferring again out of the same context (unless it is first transferred back to that context). * Updates the "attaching to a media element" text to indicate that a `MediaSourceHandle` can be attached successfully to at most one media element at a time, and with the previous 2 clarifications, ensures that the underlying worker context `MediaSource` can only be attached successfully to at most one media element at a time. Note that legacy MSE object URL attachments similarly already achieve the same intent in existing implementations. This change includes a clarification for those similar to the third bullet, but for a `MediaSource` in the "attaching to a media element" text, since it was never the intent even for main-thread `MediaSource` attachments for them to be successfully attached to more than one `HTMLMediaElement` at a time. The switch to more clear `srcObject` usage for worker MSE attachment affords the spec an opportunity to be more clear about this intent. This refinement originates from discussion on the previous PR with @karlt: w3c#305 This work is associated with the MSE-in-Worker spec issue: w3c#175

@karlt

Work-in-progress update, refactoring previous PR commit to align with results of discussion on that PR. Adds text intending to help prevent using one or more `MediaSourceHandle` for the same underlying `MediaSource` for simultaneous, concurrent attachments to media elements: * Changes `MediaSource.getHandle()` to be a `[SameObject] readonly attribute` named `handle`. TODO: see w3c#306 (comment) for steps involved in this refactoring, some of which are not yet completed. * Updates `MediaSourceHandle` transfer section to more clearly indicate that transferring a handle out of a context makes that handle unavailable for transferring again out of the same context (unless it is first transferred back to that context). * Updates the "attaching to a media element" text to indicate that a `MediaSourceHandle` can be attached successfully to at most one media element at a time, and with the previous 2 clarifications, ensures that the underlying worker context `MediaSource` can only be attached successfully to at most one media element at a time. Note that legacy MSE object URL attachments similarly already achieve the same intent in existing implementations. This change includes a clarification for those similar to the third bullet, but for a `MediaSource` in the "attaching to a media element" text, since it was never the intent even for main-thread `MediaSource` attachments for them to be successfully attached to more than one `HTMLMediaElement` at a time. The switch to more clear `srcObject` usage for worker MSE attachment affords the spec an opportunity to be more clear about this intent. This refinement originates from discussion on the previous PR with @karlt: w3c#305 This work is associated with the MSE-in-Worker spec issue: w3c#175

@karlt

Work-in-progress update, refactoring previous PR commit to align with results of discussion on that PR. Adds text intending to help prevent using one or more `MediaSourceHandle` for the same underlying `MediaSource` for simultaneous, concurrent attachments to media elements: * Changes `MediaSource.getHandle()` to be a `[SameObject] readonly attribute` named `handle`. * in-progress: updating transfer and attachment steps/text. TODO: see w3c#306 (comment) for steps involved in this refactoring, some of which are not yet completed. * Updates `MediaSourceHandle` transfer section to more clearly indicate that transferring a handle out of a context makes that handle unavailable for transferring again out of the same context (unless it is first transferred back to that context). * Updates the "attaching to a media element" text to indicate that a `MediaSourceHandle` can be attached successfully to at most one media element at a time, and with the previous 2 clarifications, ensures that the underlying worker context `MediaSource` can only be attached successfully to at most one media element at a time. Note that legacy MSE object URL attachments similarly already achieve the same intent in existing implementations. This change includes a clarification for those similar to the third bullet, but for a `MediaSource` in the "attaching to a media element" text, since it was never the intent even for main-thread `MediaSource` attachments for them to be successfully attached to more than one `HTMLMediaElement` at a time. The switch to more clear `srcObject` usage for worker MSE attachment affords the spec an opportunity to be more clear about this intent. This refinement originates from discussion on the previous PR with @karlt: w3c#305 This work is associated with the MSE-in-Worker spec issue: w3c#175

@karlt

Work-in-progress update, refactoring previous PR commit to align with results of discussion on that PR. Overall intent for a `MediaSourceHandle` is that it should be [Transferable], and be restricted to "at most one" successful attachment to the underlying `MediaSource` by a handle. Further, if a `Window` `MediaSource` was previously attached using a legacy MSE object URL, subsequent attempts to use that `MediaSource` instance's `MediaSourceHandle` for attachment must also fail. * Changes `MediaSource.getHandle()` to be a `[SameObject] readonly attribute` named `handle`. * Adds HTMLMediaElement.srcObject extension subsection. * Updates MediaSourceHandle transfer and related attachment text to more clearly indicate that transferring a handle prevents re-transfer of that instance due to the [[Detached]] internal slot's logic, and that transfer is also synchronously prevented if the handle instance's [[has ever been assigned as srcobject]] internal slot is true. Also adds multiple notes to describe intent and to clarify. * Updates the "attaching to a media element" text to indicate that a `MediaSourceHandle` can be attached successfully to at most one media element ever. Note that legacy MSE object URL attachments similarly already achieve the same intent in existing implementations. This change includes a clarification for MSE object URL attachments, since it was never the intent even for main-thread `MediaSource` attachments for them to be successfully attached to more than one `HTMLMediaElement` at a time. The switch to more clear `srcObject` usage for worker MSE attachment affords the spec an opportunity to be more clear about this intent. This refinement originates from discussion on the previous PR with @karlt (w3c#305) and from lengthy discussion on this change itself at w3c#306. This work is associated with the MSE-in-Worker spec issue: w3c#175

@karlt

Overall intent for a `MediaSourceHandle` is that it should be [Transferable], and be restricted to "at most one" successful attachment to the underlying `MediaSource` by a handle. Further, if a `Window` `MediaSource` was previously attached using a legacy MSE object URL, subsequent attempts to use that `MediaSource` instance's `MediaSourceHandle` for attachment must also fail. * Changes `MediaSource.getHandle()` to be a `[SameObject] readonly attribute` named `handle`. * Adds HTMLMediaElement.srcObject extension subsection. * Updates MediaSourceHandle transfer and related attachment text to more clearly indicate that transferring a handle prevents re-transfer of that instance due to the [[Detached]] internal slot's logic, and that transfer is also synchronously prevented if the handle instance's [[has ever been assigned as srcobject]] internal slot is true. Also adds multiple notes to describe intent and to clarify. * Updates the "attaching to a media element" text to indicate that a `MediaSourceHandle` can be attached successfully to at most one media element ever. Note that legacy MSE object URL attachments similarly already achieve the same intent in existing implementations. This change includes a clarification for MSE object URL attachments, since it was never the intent even for main-thread `MediaSource` attachments for them to be successfully attached to more than one `HTMLMediaElement` at a time. The switch to more clear `srcObject` usage for worker MSE attachment affords the spec an opportunity to be more clear about this intent. This refinement originates from discussion on the previous PR with @karlt (w3c#305) and from lengthy discussion on this change itself at w3c#306. This work is associated with the MSE-in-Worker spec issue: w3c#175

@karlt

Overall intent for a `MediaSourceHandle` is that it should be [Transferable], and be restricted to "at most one" successful attachment to the underlying `MediaSource` by a handle. Further, if a `Window` `MediaSource` was previously attached using a legacy MSE object URL, subsequent attempts to use that `MediaSource` instance's `MediaSourceHandle` for attachment must also fail. * Changes `MediaSource.getHandle()` to be a `[SameObject] readonly attribute` named `handle`. * Adds HTMLMediaElement.srcObject extension subsection. * Updates MediaSourceHandle transfer and related attachment text to more clearly indicate that transferring a handle prevents re-transfer of that instance due to the [[Detached]] internal slot's logic, and that transfer is also synchronously prevented if the handle instance's [[has ever been assigned as srcobject]] internal slot is true. Also adds multiple notes to describe intent and to clarify. * Updates the "attaching to a media element" text to indicate that a `MediaSourceHandle` can be attached successfully to at most one media element ever. Note that legacy MSE object URL attachments similarly already achieve the same intent in existing implementations. This change includes a clarification for MSE object URL attachments, since it was never the intent even for main-thread `MediaSource` attachments for them to be successfully attached to more than one `HTMLMediaElement` at a time. The switch to more clear `srcObject` usage for worker MSE attachment affords the spec an opportunity to be more clear about this intent. This refinement originates from discussion on the previous PR with @karlt (w3c#305) and from lengthy discussion on this change itself at w3c#306. This work is associated with the MSE-in-Worker spec issue: w3c#175

@karlt

* Refine srcObject's MediaSourceHandle behavior Overall intent for a `MediaSourceHandle` is that it should be [Transferable], and be restricted to "at most one" successful attachment to the underlying `MediaSource` by a handle. Further, if a `Window` `MediaSource` was previously attached using a legacy MSE object URL, subsequent attempts to use that `MediaSource` instance's `MediaSourceHandle` for attachment must also fail. * Changes `MediaSource.getHandle()` to be a `[SameObject] readonly attribute` named `handle`. * Adds HTMLMediaElement.srcObject extension subsection. * Updates MediaSourceHandle transfer and related attachment text to more clearly indicate that transferring a handle prevents re-transfer of that instance due to the [[Detached]] internal slot's logic, and that transfer is also synchronously prevented if the handle instance's [[has ever been assigned as srcobject]] internal slot is true. Also adds multiple notes to describe intent and to clarify. * Updates the "attaching to a media element" text to indicate that a `MediaSourceHandle` can be attached successfully to at most one media element ever. Note that legacy MSE object URL attachments similarly already achieve the same intent in existing implementations. This change includes a clarification for MSE object URL attachments, since it was never the intent even for main-thread `MediaSource` attachments for them to be successfully attached to more than one `HTMLMediaElement` at a time. The switch to more clear `srcObject` usage for worker MSE attachment affords the spec an opportunity to be more clear about this intent. This refinement originates from discussion on the previous PR with @karlt (#305) and from lengthy discussion on this change itself at #306. This work is associated with the MSE-in-Worker spec issue: #175 Multiple further changes made in response to review comments on this PR are also included: * MSE-in-Workers: Update attaching to a media element language Editorial only, based on PR comment: #306 (comment) * MSE-in-Workers: Further update the attaching to a media element text Mostly editorial, in response to PR review comments. Non-editorial change: * Adds condition on [[Detached]] internal slot of a MediaSourceHandle, preventing successful attachment start if the handle instance is already transferred (detached). Editorial changes: * Replaces "been used beyond this point" text with a new MediaSource internal slot [[has ever been attached]]. Adds that slots definition and initialization in the MediaSource section, and also updates that internal slot to be true in the attaching steps. * Removes redundant-with-readyState-closed clause "being using beyond this point". * MSE-in-Workers: Minor editorial update to extended resource fetch algorithm text Editorial change in response to PR comment: #306 (comment) Changes "these clarifications" to "these requirements" * MSE-in-Workers: Another minor editorial update to extended resource fetch algorithm text Editorial change in response to PR comment: #306 (comment) Changes "do not apply" to "shall not apply" * MSE-in-Workers: Minor editorial clarification to meaning of Detached In response to PR comment: #306 (comment) * MSE-in-Workers: editorial, add reference to definition of "agent cluster" Editorial change in response to PR comment: #306 (comment) Adds normative reference to ECMAScript's definition of agent cluster, and adds an informative reference to related concepts in HTML where agent and agent cluster concepts from ECMAScript are integrated into HTML for Web APIs. * MSE-in-Workers: Remove justification from srcObject setter extension note In response to PR comment: #306 (comment) * MSE-in-Workers: Remove a couple redundant transfer steps Editorial, in response to PR comments: #306 (comment) #306 (comment) StructuredSerializeWithTransfer already moderates transfer success of Transferable objects like MediaSourceHandle with the object's [[Detached]] internal slot, so explicit extra steps to do this just for MediaSourceHandle are unnecessary and potentially confusing. Removed.

Adds core MediaSourceHandle interface and modules implementation. Later changes will enable apps to get a handle from a dedicated worker MediaSource, post it to the main thread and attach it to an HTMLMediaElement via the srcObject attribute; they will also include test updates. References: Full prototype CL: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) BUG=878133 Change-Id: I9a3b47ea02d2755a9860999e245eafc26f864977 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3687146 Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Kentaro Hara <haraken@chromium.org> Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012115} NOKEYCHECK=True GitOrigin-RevId: 2813392442e5ff5cc8702a743bdd52a1bab82be4

Adds ability for apps to get a MediaSourceHandle from a dedicated worker MediaSource instance, gated by RuntimeEnabledFeature "MediaSourceInWorkersUsingHandle". Updates the PassKey required for creation of a concrete MediaSourceAttachmentSupplement to allow that only from either URLMediaSource::createObjectURL or the new MediaSource::getHandle method. This specificity is enabled by a new AttachmentCreationPassKeyProvider type. Later changes will enable apps to post the handle to the main thread and attach it to an HTMLMediaElement via the srcObject attribute; they will also include test updates. References: Full prototype CL: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) crbug.com/506273 is somewhat related, but not fixed by this change (it refers to directly setting MediaSource on Window context to the media element's srcObject attribute.) This change sequence is specific to enabling MSE-in-Worker attachment via srcObject using a handle object. BUG=878133,506273 Change-Id: Ic61f4cc4193080bdbc39234b98897d9a789778d6 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3688613 Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Will Cassella <cassew@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012129} NOKEYCHECK=True GitOrigin-RevId: 09e5fb1a68416f65ef2255c846b81a46f6aa8e52

Adds ability to serialize a MediaSourceHandleImpl, conditionally failing with DataCloneError if the handle is_serialized() already or if the handle is_used() (both of those are initialized to be false in the new instance created during deserialization.) Later changes will enable apps to attach the handle to a main thread HTMLMediaElement via the srcObject attribute; they will also include test updates. References: Full prototype CL: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) BUG=878133 Change-Id: I19cc8a450423964aef78e8e468776e8cd912503a Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3688599 Reviewed-by: Will Cassella <cassew@chromium.org> Reviewed-by: Kentaro Hara <haraken@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012132} NOKEYCHECK=True GitOrigin-RevId: d5d391975da969d6242e5785c68609d6cd15fda3

Adds ability to attach a dedicated-worker-owned MediaSource to an HTMLMediaElement using a MediaSourceHandle for that MediaSource instance that the app has posted over to the main thread and assigned to the srcObject attribute on the media element. Previously, only a MediaStream could be set as a srcObject in Chrome. This change adds the relevant MediaProvider IDL union type and makes it the type of that srcObject attribute. For historical reasons, the srcObject attribute of an HTMLMediaElement is implemented in modules as a partial interface; this change updates that partial interface as well as the media element implementation to enable the scenario without regressing existing ability to use MediaStream as media element's srcObject. The attachment of the worker MediaSource using the handle does not involve lookup into a URL registry; rather, an internal blob URL is built into the handle when it was retrieved from the MediaSource and that URL is used to satisfy existing media element load safety checks as well as provide the underlying WebMediaPlayer a URL that it can use when logging to devtools. Later changes will add further test updates and remove the previous ability to attach a worker MediaSource using a registered object URL. References: Full prototype CL: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) crbug.com/506273 is somewhat related, but not fixed by this change (it refers to directly setting MediaSource on Window context to the media element's srcObject attribute.) This change sequence is specific to enabling MSE-in-Worker attachment via srcObject using a handle object. BUG=878133,506273 Change-Id: I86cddd87bafae1c7cbae9e94ea4614418067012f Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3688740 Reviewed-by: Kentaro Hara <haraken@chromium.org> Reviewed-by: Elad Alon <eladalon@chromium.org> Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012202} NOKEYCHECK=True GitOrigin-RevId: 756af69b67c9e40e0a0c92cb6c0a911c96ea6835

If the MediaSourceInWorkersUsingHandle feature is enabled, this change prevents successful ability of obtaining an objectURL that would succeed in loading a worker-owned MediaSource. It changes the wpt tests to use handle for attachment and verifies expected new behavior of getHandle and that worker objectURL attachment fails (these tests run on experimental builds of Chromium with currently-experimental MediaSourceInWorkersUsingHandle feature enabled, just like the currently-experimental MediaSourceInWorkers feature.) References: Full prototype CL for the parts 1-4 that have already landed: https://chromium-review.googlesource.com/c/chromium/src/+/3515334 MSE spec issue: w3c/media-source#175 MSE spec feature updates switching from worker MSE attachment via object URL to attachment via srcObject MediaSourceHandle: * w3c/media-source#305 * further clarifications in discussion at w3c/media-source#306 (comment) BUG=878133 Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> Reviewed-by: Will Cassella <cassew@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012712} NOKEYCHECK=True GitOrigin-RevId: 6315549b8c2ece3dbbf3062c1a87347589a5e115

…bjectURL" This reverts commit 6315549b8c2ece3dbbf3062c1a87347589a5e115. Reason for revert: This is causing failures on the WebKit Linux Leak builder i.e. https://ci.chromium.org/ui/p/chromium/builders/ci/WebKit%20Linux%20Leak/39394/overview Original change's description: > MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL > > If the MediaSourceInWorkersUsingHandle feature is enabled, this change > prevents successful ability of obtaining an objectURL that would succeed > in loading a worker-owned MediaSource. > > It changes the wpt tests to use handle for attachment and verifies > expected new behavior of getHandle and that worker objectURL attachment > fails (these tests run on experimental builds of Chromium with > currently-experimental MediaSourceInWorkersUsingHandle feature enabled, > just like the currently-experimental MediaSourceInWorkers feature.) > > References: > Full prototype CL for the parts 1-4 that have already landed: > https://chromium-review.googlesource.com/c/chromium/src/+/3515334 > MSE spec issue: > w3c/media-source#175 > MSE spec feature updates switching from worker MSE attachment via > object URL to attachment via srcObject MediaSourceHandle: > * w3c/media-source#305 > * further clarifications in discussion at > w3c/media-source#306 (comment) > > BUG=878133 > > Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> > Reviewed-by: Will Cassella <cassew@chromium.org> > Cr-Commit-Position: refs/heads/main@{#1012712} Bug: 878133 Change-Id: I1e405ae1de146d1f3183592b00a43bd3c38d849d No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3695890 Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Commit-Queue: Nidhi Jaju <nidhijaju@chromium.org> Owners-Override: Nidhi Jaju <nidhijaju@chromium.org> Cr-Commit-Position: refs/heads/main@{#1012823} NOKEYCHECK=True GitOrigin-RevId: f438823c5527e6499b8dedb4eef7bb8ed43f4371

…bjectURL With the underlying leak fixed by https://chromium-review.googlesource.com/c/chromium/src/+/3704191, this change can be relanded now. Previous revert: > Revert "MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL" > > This reverts commit 6315549b8c2ece3dbbf3062c1a87347589a5e115. > > Reason for revert: This is causing failures on the WebKit Linux Leak builder > i.e. https://ci.chromium.org/ui/p/chromium/builders/ci/WebKit%20Linux%20Leak/39394/overview > > Original change's description: > > MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL > > > > If the MediaSourceInWorkersUsingHandle feature is enabled, this change > > prevents successful ability of obtaining an objectURL that would succeed > > in loading a worker-owned MediaSource. > > > > It changes the wpt tests to use handle for attachment and verifies > > expected new behavior of getHandle and that worker objectURL attachment > > fails (these tests run on experimental builds of Chromium with > > currently-experimental MediaSourceInWorkersUsingHandle feature enabled, > > just like the currently-experimental MediaSourceInWorkers feature.) > > > > References: > > Full prototype CL for the parts 1-4 that have already landed: > > https://chromium-review.googlesource.com/c/chromium/src/+/3515334 > > MSE spec issue: > > w3c/media-source#175 > > MSE spec feature updates switching from worker MSE attachment via > > object URL to attachment via srcObject MediaSourceHandle: > > * w3c/media-source#305 > > * further clarifications in discussion at > > w3c/media-source#306 (comment) > > > > BUG=878133 > > > > Change-Id: I60ddca79ee0f95c87b6d84e4f44ad9c283f359a3 > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3698231 > > Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> > > Auto-Submit: Matthew Wolenetz <wolenetz@chromium.org> > > Reviewed-by: Will Cassella <cassew@chromium.org> > > Cr-Commit-Position: refs/heads/main@{#1012712} > > Bug: 878133 > Change-Id: I1e405ae1de146d1f3183592b00a43bd3c38d849d > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3695890 > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> > Commit-Queue: Nidhi Jaju <nidhijaju@chromium.org> > Owners-Override: Nidhi Jaju <nidhijaju@chromium.org> > Cr-Commit-Position: refs/heads/main@{#1012823} Bug: 878133 Change-Id: I56e4ecd4d8b58d9d58ed3c575b0fb52f596b6fae Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3708465 Reviewed-by: Will Cassella <cassew@chromium.org> Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org> Cr-Commit-Position: refs/heads/main@{#1014755} NOKEYCHECK=True GitOrigin-RevId: 39878163d3ff90baabd342ad784671a4b54262ed

wolenetz requested a review from mwatson2 March 9, 2022 22:49

wolenetz mentioned this pull request Mar 9, 2022

MediaSource in Worker #175

Open

karlt reviewed Mar 10, 2022

View reviewed changes

mwatson2 approved these changes Mar 14, 2022

View reviewed changes

wolenetz force-pushed the srcobject_for_worker_mse branch from f9df82e to b8624a3 Compare March 30, 2022 19:40

Switch from src to srcObject for worker MSE attachment

1f22c33

Adds text to require implementations to fail attachment using a worker MediaSource's object URL. Adds a MediaSourceHandle object and describes how to use it to attach a worker MediaSource to a media element.

wolenetz force-pushed the srcobject_for_worker_mse branch from b8624a3 to 1f22c33 Compare March 30, 2022 19:49

wolenetz merged commit 89bd50e into w3c:main Mar 30, 2022

wolenetz deleted the srcobject_for_worker_mse branch March 30, 2022 19:55

wolenetz mentioned this pull request Mar 31, 2022

Refine srcObject's MediaSourceHandle behavior #306

Merged

chromium-wpt-export-bot mentioned this pull request Jun 9, 2022

MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL web-platform-tests/wpt#34364

Merged

chromium-wpt-export-bot mentioned this pull request Jun 10, 2022

Revert "MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL" web-platform-tests/wpt#34371

Merged

chromium-wpt-export-bot mentioned this pull request Jun 15, 2022

Reland: MSE-in-Workers: srcObject part 5: Conditionally fail worker objectURL web-platform-tests/wpt#34451

Merged

wolenetz mentioned this pull request Jul 27, 2022

MSE in Workers mozilla/standards-positions#547

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch from src to srcObject for worker MSE attachment #305

Switch from src to srcObject for worker MSE attachment #305

wolenetz commented Mar 9, 2022 •

edited by pr-preview bot

Loading

wolenetz commented Mar 9, 2022

wolenetz commented Mar 9, 2022

karlt Mar 10, 2022

wolenetz Mar 10, 2022

karlt Mar 11, 2022

wolenetz Mar 18, 2022

karlt Mar 20, 2022

wolenetz Mar 30, 2022

wolenetz Mar 30, 2022

Switch from src to srcObject for worker MSE attachment #305

Switch from src to srcObject for worker MSE attachment #305

Conversation

wolenetz commented Mar 9, 2022 • edited by pr-preview bot Loading

wolenetz commented Mar 9, 2022

wolenetz commented Mar 9, 2022

karlt Mar 10, 2022

Choose a reason for hiding this comment

wolenetz Mar 10, 2022

Choose a reason for hiding this comment

karlt Mar 11, 2022

Choose a reason for hiding this comment

wolenetz Mar 18, 2022

Choose a reason for hiding this comment

karlt Mar 20, 2022

Choose a reason for hiding this comment

wolenetz Mar 30, 2022

Choose a reason for hiding this comment

wolenetz Mar 30, 2022

Choose a reason for hiding this comment

wolenetz commented Mar 9, 2022 •

edited by pr-preview bot

Loading