Automatic Package Acquisition and Full Project Loads for TypeScript on Web

[DanielRosenwasser]

TypeScript in VS Code has already been able to support workloads across TypeScript and JavaScript projects in web scenarios, such as on https://vscode.dev and https://github.dev. Unfortunately, it suffers from two problems:

1. It only supports a "partial semantic" mode, and only operates on open files.
2. It cannot provide rich navigation and authoring based on external libraries, due to a lack of either automatic type acquisition or in-browser package management.

Over the past months, @sandersn, @zkat, and others have experimented with enabling these experiences on the web through a few separate efforts, and we think VS Code in the browser should be able to adopt them in a near-term iteration.

Full Project Loads

Editing or exploring TypeScript and JavaScript projects is decent today; however, cross-file navigation, auto-completion, and more are extremely limited from not having the full project graph available. Without files from the same program or configuration files, we can't give anything close to the experience we provide outside of the browser.

Currently, there are means to experimentally enable a full project load by using a custom server host in the browser. This is powered by @vscode/sync-api-client. In https://vscode.dev, the TypeScript extension should be able to leverage this custom host when it creates a custom tsserver instance.

Automatic Package Acquisition

Even when we load the full program graph in TypeScript, we'll still be missing the declaration files from external libraries, which are a huge part of the editing experience. This is a familiar problem - when TypeScript started supporting JavaScript projects, managing declaration files was a process specific to TypeScript projects. That's why TypeScript supports a process called "automatic type acquisition". When TSServer is started, it can be given the path to a "typings installer" process, and in certain cases, when TypeScript's project system encounters an unresolved import, it will send a message to that typings installer process. For files that should appear in an unresolved package like foo, declaration files from @types/foo are downloaded to a cache and projects then have the option to use these declaration files when a path fails to resolve.

This is close to what a user needs for a TypeScript-in-the-web experience, except that

1. ATA is usually never triggered for a TypeScript project (unless a user enables it (don't do this))
2. ATA doesn't cover installing any @types packages
3. ATA works via best-effort, and just grabs the latest version

What we want is something closer to what one might call "automatic package acquisition" that can fetch the actual package in the most accurate way.

We've made some more of our APIs public in the TSServer library. Over at zkat#1, we have a proof-of-concept doing exactly this using

1. a custom typingsInstaller.js that can be run on the web
2. an npm dependency evaluator/downloader that we can run in web contexts

This combo works pretty well in our cursory experimentation! Still, there's some more polish that will need to go into this.

Current Next Steps

As you can see from the checklist over at zkat#1, there's still a bit of work to be done.

1. We need to decide on whether the current proof of concept is sufficient, or if it can serve as an inspiration for whatever is the "right" implementation.
2. We should identify a (reasonable) target date for when we'd like to get this work rolled out by so that we can work backwards and figure out what to correct and prioritize.
3. There are several TODOs in the PR, that are either bugs or possibly-missing features. If we choose the path of a new implementation, they may be "automatically"; but regardless, we should make sure they are tested. Of those, there's a rough way to divide remaining tasks work based on affinity/expertise:
   • vscode.dev mechanics
     • Decide on a caching mechanism (e.g. localStorage)
     • Support installation for private registries via CORS configuration
   • Typings Installer and TSServer mechanics
     • Investigate package.json File Watching issues
     • Investigate Go-to-Source-Definition issues
     • Investigate First ATA Events Being Blocked on File Close

[sandersn]

Here's a draft PR based on @zkat's work: #182801

I wrote up a short overview of the changes with pointers into the code.

[Abdallatif]

Not sure if this is the best place to report the bug but I'm going to post here since this is a work-in-progress. I'm trying to setup vscode on the web with typescript support using https://github.com/CodinGame/monaco-vscode-api. My app has private npm packages (node_modules) that are already mounted in the vscode filesystem tree. I found that typescript server ignores my node_modules directory and always tries to download the package from npm.

I was able to fix the issue by disabling this behavior by commenting this line. You can find more details about this issue here.

Do you think it makes sense for AutoInstallerFs to be initialized from the filesystem node_modules folder and maybe check the filesystem before trying to download the package from a registry.

[mjbvz]

Should be working. Have run into a few edge cases but overall seems to be working well

[mjbvz]

Marking as verified because we actually went through testing last iteration: #226653