Use sourcemap sources for Go to Implementation and show them in Explorer pane (like Chrome devtools does), even when no debug session is active

[justingrant]

Summary

VS Code should treat the sourcesContent inside source map files (and inline sourcemaps too) as if they were real, read-only, virtual files that exist in the IDE whether or not a debug session is running, similar to how Chrome dev tools exposes sourcemap code.

The user should be able to see sourcemap code as files in the IDE's Explorer pane, should be able to set breakpoints in them without the debugger running, should be able to use Go to Implementation to read library source code, and more.

This feature is a superset of microsoft/vscode#144424, which is a debugger-only, inline-sourcemap-only request.

Justification

A large and increasing portion of packages in the npm registry are transpiled or bundled. Often this is because the original source is TypeScript. Or because the author wants to write ES2021+ code and rely on babel or other transpilers for downlevel support. Or for many other reasons. Package authors have gotten much better about also shipping sourcemaps with their packages.

However, frequently package authors DON'T include the original source files of their package in their NPM distribution. They only ship bundled and/or transpiled JS code, and sourcemaps. Often this is based on strong recommendations from other package authors to omit sources. For example:

My point here isn't to hassle @jakebailey (he later amended this to soften the recommendation), but rather to point out that this "don't ship source files" sentiment is common among JS package maintainers.

Sadly, multiple VS Code features don't work well when original source files are unavailable. In particular:

• Go to Implementation doesn't work, even if the package has a declaration map. This makes it much harder for users to find where the implementation actually is in a complex package, and it makes it harder for users to dig into library code to figure out how they work.
• If a package is bundled or transpiled then the resulting JS is really hard to read. Especially for less-skilled developers who are the majority of developers worldwide. And *especially* if the package uses some features like async/await whose downlevel transpilation is almost impossible to understand.
• You can't set a breakpoint on library code before starting a debug session is currently active. This makes it harder to debug into library code, or to debug problems that only happen at startup.
• It creates an unnecessarily big difference between the debugging UX in Chrome dev tools (where sourcemap code shows up as real-looking files in the sources pane) and the explorer pane in VS Code where sourcemap code is invisible.

But despite these challenges that users have with missing sources in packages, there are many of reasons why package authors don't want to ship sources in their npm packages. Here's a few I've heard:

• It's not needed, because sourcemaps with sourcesContent are sufficient for debugging in Chrome dev tools.
• Deploying more files in NPM packages fills up developers hard drives with unnecessary bloat.
• Transpiled JS code is "usually readable enough for me".
• If I ship sources, then users will mistakenly think they can patch my TS code and complain when that does nothing.
• If I ship sources, then users will mistakenly import .ts files directly e.g. import { foo } from 'some-package/src/bar.ts'; and when I change my TS code, e.g. to upgrade to a new TS version, they'll file bugs and complain that I broke their app even though they're not using my public API.
• Bundlers may be configured to compile TS code in node_modules.
• Poorly-configured packages may include TS sources and transpiled JS in the same folder, leading the TS file to be resolved first by import or require.
• Because they feel like it, e.g. "Just ship dist dir, that's common sense.".
• Because there's no clear guidance in the TypeScript docs telling them otherwise.

Some of these reasons are technical problems that cause real user bug reports and complaints, which package maintainers always want to avoid. Other reasons stem from the fact that package maintainers are usually much more experienced JS developers than package users, so it's harder for them to understand the profound difficulty that novice JS developers have with basic tasks like setting breakpoints in library code, finding the implementation of a particular function, or even reading transpiled code.

Over the last 4 years, I've filed ~20 PRs to try to get package maintainers to ship sources and to fix broken sourcemaps. But, to be honest, I've barely made a dent.

So I think it's time for VS Code to recognize that many package maintainers will *never* ship their sources. But they will ship Chrome-dev-tools-friendly sourcemaps! Can we use those sourcemaps to help developers read and understand library code?

Proposal

VS Code should start treating sourcesContent and inline sourcemaps as read-only, virtual source files that are first-class citizens in the IDE.

Top Priority cases:

• Show sourcemap code as a read-only file in the explorer pane, just as if it were a real file on disk (although perhaps with a different visual treatment like Chrome does when it changes the background color of the file icon). This will allow users to explore a package and read the original source code to understand how the package works.
• Allow breakpoints to be set in one of these "files", even when the debugger isn't running. This supports a more natural debugging workflow than having to wait to start a debug session before seeing the original source code, and enables breakpoints on startup code.
• Allow Go To Implementation to navigate from an imported symbol, to its associated declaration map, and finally into a read-only sourcemap code. For example, if dist/index.d.ts.map includes "sources":["../src/index.ts"], and dist/index.js.map includes "sources":["../src/index.ts"], then that should be enough for VS Code (or the TS Server) to be able to navigate the user to the TS source.
  • An added bonus is that once the user has found one implementation file, they can navigate sideways in the explorer pane to find other implementation files. Many packages have deeply-nested and/or confusing folder structures (e.g. different folders for ESM vs. CJS), so finding one file is often the key to unlocking the whole package for readability.
• Continue to show the sourcemap-generated files in the explorer pane when a debug session starts, to provide that familiar-like-Chrome-devtools user experience.

Lower-priority cases (these are less important to users and/or may have performance issues that may make them really hard to implement)

• If the sourcemap's files property has invalid paths, ideally there'd be some indication in the IDE to help the user understand why they aren't seeing sources
• Ideally there'd be some way to navigate from a particular location in sourcemap code back to the transpiled JS that corresponds to it, e.g. a "Go to Transpiled JS" context menu option when right-clicking on sourcemap code. This would be particularly useful for advanced debugging cases or if something is wrong with the library's source-mapping config.
• Support these "files" with Find in Files (although not Replace in Files because they're read-only).
• Explorer pane filtering should work, so I can type the name of the source file and I'd expect it to find it in the explorer pane.
• Other IDE features that rely on file paths should also work. (This is a hand-wavey catch-all that would need to be fleshed out more, because I'm not sure what other feature there are that rely on file paths.)

There would have to be some limits and constraints:

• Only sourcemaps that expose relative paths in sources would be supported. No file:// paths or other URLs.
• For security reasons, no relative navigation should be allowed deeper than the root of the installed package.
• If a file already exists at that location (i.e. the package did install its source), then don't create the fake read-only files.
• I don't think you should be able to copy/paste these "virtual files", or at least not without some sort of opt-in decision by the user that they want to paste files that weren't actually there in the source folder.
• It'd be reasonable if many extensions didn't see these "files" at all, or otherwise there were limited opt-in support for these virtual files among extensions.

What I'm asking for here might be very expensive if VS Code doesn't currently have a "virtual file" capability in the IDE. But after 4 years of trying to convince maintainers to ship their sources to enable a better VS Code experience, I'm starting to believe that the easiest and most realistic solution might be to change VS Code instead of trying to convince every maintainer one at a time.

Thanks for listening!

[roblourens]

This is really a feature request for the ts/js language server. It might belong in the Typescript repo

[justingrant]

This is really a feature request for the ts/js language server. It might belong in the Typescript repo

@roblourens Does VS Code's file explorer pane currently have the ability (like Chrome Dev Tools does) to show "virtual" files that don't exist on disk? Or is every file in the explorer pane required to exist on disk?