Added debug id proposal

[mitsuhiko]

This is a proposal for #19. The idea is to embed globally unique debug or build IDs into source maps and transpiled JavaScript files. This enables a work flow similar to native object files and their debug companions or WASM files (WebAssembly/tool-conventions#133).

Rendered text

[Swatinem]

Its unclear to me what this means? Does it mean that only JS files have the //# debugId comment in them? Because you can have any arbitrary file as one of the "sources".

[mitsuhiko]

The original spec also leaves open CSS and other formats.

[Swatinem]

Ohhh absolutely, I always forget that this is not exclusive to JS. 🤔

[Swatinem]

Today I learned: https://en.wikipedia.org/wiki/Universally_unique_identifier#Versions_3_and_5_(namespace_name-based)

UUIDs have special version and variant tags, so they don’t use the full 128-bits.

But as you said, I would leave that up to the specific toolchain, as long as the identifier is formatted like a UUID and sufficiently unique, it will be fine :-)

[Swatinem]

Not sure if this needs to be mentioned here or anywhere. Tools like webpack have an option to have "hidden" (See https://webpack.js.org/configuration/devtool/#devtool) sourcemaps, which are created but not referenced, to create some sense of "obfuscation". As the Debug ID would just be some random bytes, they do not pose any "obfuscation" risk so to say, and tools should always put them in.

[Swatinem]

Error.stack returns an opaque string. With some luck you are able to parse a URL out of it though.

[Swatinem]

I believe this might be the most controversial point of this proposal.

While a lot of tools use the System global, I believe no "official" spec is using it?

Also, "URL that has already been loaded by the browser" might need a bit more clarification, for example:

• It should return the debug-id of scripts "that are current loaded" into the engine.
• The URL should match the one that was used to load the script, and should also match the ones pretty-printed in Error.stack output, import.meta.url, etc.
• Specifically, it should have the same query-string and fragment.
• It should not do any network IO
• On mismatch, should it return undefined or rather throw a ReferenceError or any other kind of error?
• On mismatch, should it give an explanation of the reason? Is the file not loaded at all? Do the querystring parameters not match? etc…

[Swatinem]

👍🏻 I like this, especially also implicitly forcing JS files to be UTF-8 ;-)

Should we also propose a JSON Schema along with a "$schema" field for source maps, or would that be a bit too much?

[mitsuhiko]

Mostly want to see initial feedback but a JSON schema for source maps would be very valuable by itself.

[Swatinem]

I believe we should also plan for browsers and local development servers.
Using something like Import Maps could solve this, and point the browser to the local development server to resolve the corresponding source map.

Related to my note above, this could potentially replace the various different ways that bundlers / development servers can offer source maps today. I believe offering a local symbol server would also benefit performance a bit, as the tools wouldn’t have to embed base-64 encoded sourcemaps into the development assets, but sourcemaps would only be needed to be serialized on demand.

[robpaveza]

At Microsoft we've been using the SHA-256 hash of the script contents, plus the name of the file of course.

A possible drawback here is if only the original source changes - e.g., with newlines or comments only - then there is a possibility of source maps not matching up. But I can say that in about 18 months of the Edge DevTools team using this mechanism for our own internal debugging, we haven't run into this issue at all. (Bottom line is, if the old and the new code both mapped to the same output contents, and the error existed in the old contents, then you should be able to reason about your code -- because the executing code itself didn't change. This approach is also hardened if you're including the original source inside your source maps).

That having been said, the advantage we liked about this approach was that you don't have to index the same source map if particular files don't change. For example, we're not investing much in the "Welcome" panel in DevTools, so even though we build it every day multiple times as part of the CI, it's not changing, so we don't need to index every CI run's output of this file. It also allows us to cache the source maps reasonably reliably (Edge has been caching source maps uniquely against the filename+hash tuple since Edge 99), again having showing really good results.

From a technical perspective, V8 provides an API as part of the CallSite API which is provided in the Error.prepareStackTrace callback to obtain the script hash -- getScriptHash() which will return this hash if the script isn't being treated as opaque.

[mitsuhiko]

@robpaveza we are definitely quite interested in supporting that at the very least in addition, as this is already something that can be supported on browsers today. For what it's worth that requires using Error.prepareStackTrace which is quite tricky to use for a tool like us (as we have no guarantee that nobody else is patching that) and if the browser does not support the script hash, it's hard to fall back to.

In theory the fallback for no browser support on debug IDs is just making an XHR request to the file and parsing the debugId comment with a regex.

[littledan]

This is a very good starting point for continued discussion, so I'm landing this, and will land other analogous proposal documents without expecting them to reflect the consensus of everyone involved.