Revive createObjectURL?

[foolip]

createObjectURL was removed in 2013:
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23091
7167f73

However, it is still supported in Chrome, Edge and Firefox, as tested with this code:

var stream = new (window.MediaStream || window.webkitMediaStream);
var url = URL.createObjectURL(stream);
console.log(url);

Rather than attempting to remove it, bringing it back in the spec may be less effort for everyone involved. Here's how it's spec'd for Blob and MediaSource:
https://w3c.github.io/FileAPI/#creating-revoking
http://w3c.github.io/media-source/#url

[alvestrand]

One reason for my strong dislike of createObjectURL is that it prevents mocking of the stream object (for code that could otherwise be debugged and tested without ever making a real stream object).

There are more.

We already have a "backwards compatibility legacy" section - if we dump all the bad ideas we've discarded into that section, we get easier backwards compatibility, but we abandon all hope of eventually leaving the bad ideas behind us.

[foolip]

Can you elaborate on the mocking, is this something a web developer would do are is it for unit testing of the implementation?

If it would help inform the decision, we could measure usage in Chrome.

[alvestrand]

My personal experience (a couple of years ago) was with trying to write an utility function that kept track of a bunch of streams and how they were connected to a HTMLMedia object.

In order to test this function, I passed it a mock HTMLMedia object and a mock stream, and attempted to run it through its paces.
I could easily mock the "srcObject" attribute on the HTMLMedia object, and detect that it was set correctly - but I could not catch the call to createObjectURL without overriding the method on the global URL object for all usages - which seemed dramatic to me at the time.

I would think usage in Chrome is pretty massive - it's not long ago that we shipped srcObject at all, and I doubt many consumers have converted so far. But use counters are a Good Thing.

[foolip]

Adding use counters in https://codereview.chromium.org/2383503002/

I would be very surprised if usage is low enough to justify removal given the current state of near-interop, but if it's not in the way we can just park this issue for a few months until we know.

[stefhak]

We had a lot of discussion about this, and decided to remove it. Reasons included:

• it was never fully specced
• implementations did differ

Also, I think it was pre-fixed in all implementations.
I don't think we have strong enough arguments for re-opening this question.

[foolip]

Whatever the state of implementations was at the time, they are now unprefixed and appear to work.

[stefhak]

Putting the Icebox label on this one. I don't see us doing anything at least until we have some input from the counters added.

[foolip]

Internal data from the last 7 days on Chrome Dev channel show usage in chromestatus.com-terms of ~0.04% for URL.createObjectURL(mediaStream), ~0.73% for URL.createObjectURL(blob), and ~1.49% for URL.createObjectURL(mediaSource).

These numbers will change for stable, and I'm OK waiting for that, but it looks very unlikely that all URL.createObjectURL variants could be removed, so even if usage of the MediaStream variant would drop to <0.01% it wouldn't seem worthwhile to attempt removal in all three engines already shipping it.

[stefhak]

Great to hear some data is coming. I note that only three persons have been involved in this discussion. @foolip would you be willing to take this to the list to get more input?

[foolip]

Sure, I've sent https://lists.w3.org/Archives/Public/public-webrtc/2016Oct/0077.html

[stefhak]

Thanks!

[jan-ivar]

@foolip Just because something is hard to drop doesn't mean it should be in the spec, or is impossible to drop. We do the usual pattern of deprecation warnings, then deprecation errors, and tell devs to move over. There's less to type, so this change should be welcome, and quite compatible. Are those numbers from Chrome? Didn't Chrome just implement srcObject a month ago? Firefox unprefixed srcObject a year ago. Let's give devs some time.

[jesup]

Mozilla has always opposed using createObjectURL for WebRTC and MediaStreams in general. The primary reason is that once you call createObjectURL, you lose any way to track usage of the resource, and must keep it available until the page is destroyed. This is why we pushed hard for srcObject, and are very glad it's now implemented in Chrome. createObjectURL was effectively cruft the beginning of the effort.

[foolip]

@jan-ivar @jesup, in Gecko, is createObjectURL more problematic for MediaStream than for MediaSource or Blob? Would removing just one variant allow for any simplification?

Let's wait for the numbers from Chrome stable channel, but if they're roughly the same, and no vendor is actively pursuing removal, then things will likely stay the same. (Just deprecation messages don't seem to reliably drive down usage, presumably because in the long tail of sites there's nobody looking at the console.)

[jan-ivar]

@foolip Two bad things beat three bad things. I believe the plan is to support srcObject for all three eventually.

Of those numbers you mention, how many call revokeObjectURL correctly? We're fixing bugs by moving people off createObjectURL. Not only is the resource not freed, the camera light stays on.

We told people WebRTC isn't stable, and people signed on knowing they would need to update their code. E.g. How many in the long tail used http?

[foolip]

There's no question that this is a bad API, but removing things is hard work and you have to prioritize. Interop problems would be at the top of my list, but if anyone is motivated to get it removed (not just deprecated) in Chrome, Edge or Firefox in a reasonable time frame, then it could prove that it's safe for the remaining two. @jan-ivar, do you plan to remove it from Firefox, and in what kind of time frame?

(The usage of getUserMedia on insecure origins when it was removed was ~0.001%.)

[jan-ivar]

@foolip Thanks for the usage data on http. Removing things take time, especially when some browsers lag on implementing good alternatives.

I see webkitGetUserMedia is almost at ~0.01 yet it's not in the spec. My point is that just existing is not a sufficient criteria to be in a spec. Specs are based on consensus, not just web compat, and sometimes we have to take the long view.

[jan-ivar]

We could turn video black-and-white or degrade resolution on sites using createObjectURL...

[jesup]

We could overlay with a "Time until this stops working" countdown.

[annevk]

[...] sometimes we have to take the long view.

That ends up creating an anti-competitive situation whereby new entrants don't know what's required to support the web platform. So WHATWG and W3C largely too have abandoned that kind of thinking quite a while ago in favor of specifying what is actually required to implement a browser.

If things can be removed after a while, great, but we shouldn't end up in a situation where something is not specified for years just because it was assumed it could be removed.

[domenic]

In particular, it's fine to mark things as deprecated, and remove them from specs once they're removed from browsers. But in the meantime, specs need to reflect the reality of what's implemented, not wishful thinking. Not speccing something doesn't make it go away faster; it just creates interop pain and prevents the spec from being useful for implementers.

[jan-ivar]

@annevk, @domenic Agree this API has lingered for way long. There's a flip-side though, e.g. we left in the legacy non-promise version of RTCPeerConnection, and just this week Edge announced they just completed a working version of "WebRTC 1.0", except... you guessed it... with a non-promise API only.

I'm hopeful they'll address this, but it doesn't help that most examples of WebRTC mentioned in questions on SO for example, keep using the legacy non-promise API, even though the authors clearly just wrote it.

[annevk]

Hmm that sucks. How is the web-platform-tests story for the promise API? (Good that all examples in the specification use promises, makes it extra weird they just ignored it.)

[foolip]

I've tested Edge 14, and navigator.mediaDevices.getUserMedia returns a promise, while navigator.getUserMedia takes a pair of callbacks. Seems to be in order?

[stefhak]

@foolip I guess @jan-ivar is referring to WebRTC APIs (such as createOffer, setLocal).

[foolip]

Oh, now I see. Maybe @ShijunS can comment, but it seems a bit unlikely that the existence of the callback variants, clearly marked as legacy, is the reason that promise variants weren't shipped.

[domenic]

I don't think the presence or absence of the non-promise versions in the spec is what prompted Edge to implement them. Rather, I think it was the need to be compatible with web content in a competitive market. Now imagine if they had to do all that without a spec---then there would be two non-promise versions that became part of the web!

[alvestrand]

This is actually related to #425 - it's a legacy piece that we decided to leave out of the spec.

[jan-ivar]

I hesitate to suggest it, but we can of course polyfill and remove URL.createObjectURL(mediaStream) while still leaving URL.createObjectURL(mediaSource) and URL.createObjectURL(blob) alone until we have srcObject or those. That's hardly pulling things off cleanly, but still an incremental improvement.

The other two may be harder to get rid of, as people using e.g. MediaRecorder may not necessarily be using adapter.js? Then again, I don't know what % of webrtc users uses adapter.js in the first place.

[foolip]

To replace URL.createObjectURL and HTMLMediaElement.prototype.src in a way that avoid calling the original URL.createObjectURL(stream) is of course possible, but it's very unlikely to make a decisive difference in the usage. Not everyone is going to be using adapter.js, and with other frameworks we typically see copies of old versions staying around for a very long time, so at best this is something that takes a few years to have the desired effect.

As for URL.createObjectURL(blob), that has much higher usage still. Even with the auto-revoking URL.createFor(blob), is there any plan to get rid of the former variant? Perhaps @inexorabletash can say.

[inexorabletash]

No plan that I'm aware of. At a SW F2F (with Microsoft, Mozilla and Google peeps in the room) there was a large collective sigh about URL.createObjectURL(blob) but concerns were raised that URL.createFor(blob) did not solve all of the problems so there was not strong vendor interest in pushing it. So I don't expect rapid action in that space.

[jan-ivar]

Is URL.createFor(blob) needed once we have elem.srcObject = blob, or does the former solve something not covered by the latter? Firefox implements neither atm.

[annevk]

@jan-ivar I think we shouldn't implement createFor() if we haven't already and indeed push for APIs that are object-oriented rather than string-based. I still think createObjectURL() was a rather big mistake and we should have just added the necessary APIs to support assigning objects to things. (The argument against that has always been that there are some places, such as CSS, where it's unlikely we'll ever get sufficient API exposure to allow for assigning of objects, but that also seems to be changing.)

[jan-ivar]

Here's a polyfill for URL.createObjectURL(stream) we could put in adapter.js FWIW.

[stefhak]

@jan-ivar will you make a PR for adapter.js?

[jan-ivar]

@stefhak I've created https://github.com/webrtc/adapter/pull/411.

[stefhak]

@jan-ivar thanks, let's see if it can be merged.

[beaufortfrancois]

I'm not sure if that's related but it would be nice if we could also assign directly a MediaSource to srcObject. As I'm not familiar with the Media spec yet, I was wondering if you could tell me where I should file this proposal.

 var audio = document.createElement('audio');
 var mediaSource = new MediaSource();
 ...
- audio.src = window.URL.createObjectURL(mediaSource);
+ audio.srcObject = mediaSource;

For now, here's the error I get:
Uncaught TypeError: Failed to set the 'srcObject' property on 'HTMLMediaElement': The provided value is not of type 'MediaStream'.

[foolip]

@beaufortfrancois, that's already supposed to work per spec, but hasn't been implemented yet. Star https://crbug.com/506273 and/or comment and it might get some traction.

[stefhak]

This was discussed at the January 25th 2017 meeting, and no one was in favor of reviving, while several argued strongly that we should not revive. I will close this Issue once the "summary of decisions" has been sent on the list (given that objections to that summary is not received).

It can be noted that implementers said they will look into adding deprecation warnings ASAP, and that people said they will go through their (example/demo/etc.) code and change to use .srcObject.

[foolip]

Which implementers are going to be actively pursuing removal, and can links to bugs tracking that work be pasted here? I would suggest that if nobody has succeeded in a year's time, this issue be reopened.

[alvestrand]

The Chrome bug is here: https://crbug.com/591719

[jan-ivar]

The Firefox bug is here: https://bugzil.la/1334564

[eric-carlson]

The WebKit bug is here: https://webkit.org/b/167518 eric

[alvestrand]

Closing per WG consensus.

[foolip]

Thanks all, I'll check back here in a year.

[stefhak]

Thanks @foolip - and BTW any input on whether the adapter.js polyfill works, how counters are affected etc. in the meantime would be interesting.

[foolip]

I've commented on https://github.com/webrtc/adapter/pull/411 and it will be interesting to see if that rolling out has any impact on the usage. I would expect that a fair chunk of the usage will remain. If anyone is very motivated to remove this, then the top thing I would try is analyzing usage in httparchive, to see how it's typically used. If someone gives me a combination of static strings to search for, perhaps ".createObjectURL" and "etUserMedia", then I can provide a list of URLs that contain these strings.

[foolip]

I'll check back here in a year

It's been over a year, and somewhat to my surprise, we're making some progress. @Orphis has deprecated createObjectURL(mediaStream), which will hopefully be followed by removal. If that works out, then there's no reason to revisit this.