Do not perserve references in declaration emit, unless preserve=true

[jakebailey]

At the moment, declaration emit does a funky mixture of preserving some references, eliding unused references, adding of referenced files for ambients (like node's builtins), but also sometimes imports (react), and so on. For external implementations of isolatedDeclarations (and honestly, the human brain), this is either difficult to reason about or totally impossible to replicate.

This PR greatly changes the way we handle references. References are no longer preserved in the output from the input, nor are they ever added based on any analysis. If a reference is intended to be preserved, one must write preserve="true" to have it copied to the output file. This gives a "what you see is what you get" format, not unlike what verbatimModuleSyntax does for imported values.

This is technically a break, as packages which re-export from something like @types/node will no longer have implicitly added reference directives for those types, however this change is in line with how we handle lib.d.ts; it's assumed that the user will have actually included the environment's types into their project. Testing in prep for this change (#57569, #57656) showed that most instances of added references fit into this bucket and should continue to work (and why we are shipping this without a new flag).

This change seems to improve performance a bit (1-4% in benchmarks with dts emit enabled), likely due to the reduced bookkeeping.

The only non-obvious thing that remains is the rewriting of file references, which includes the changing of import extensions (funny) and the shifting of paths to be relative to the output paths. This specific analysis is more reasonable to implement externally as it's just a function of file paths and compiler options.

Fixes #57439 (though does the opposite by default)

It also fixes these issues by nature of us no longer synthesizing new references, or just keeping the references as written:

Fixes #15488
Fixes #48143
Fixes #52110
Fixes #56590
Fixes #57482

Then in terms of design, the direction is now "we don't generate references out of thin air anymore", in which case this also closes:

Closes #28195

Related: #56571

[typescript-bot]

Looks like you're introducing a change to the public API surface area. If this includes breaking changes, please document them on our wiki's API Breaking Changes page.

Also, please make sure @DanielRosenwasser and @RyanCavanaugh are aware of the changes, just as a heads up.

[jakebailey]

A significant number of our tests break because they don't include the new references; is this an indicator that the default should actually be preserve=true? Or that our tests are "weird code heavy"?

[sheetalkamat]

@jakebailey our tests are written such that just including d.ts files should not result in error. You would need to look at what all changes need to happen in the config for building those declaration file.

[jakebailey]

The errors are declaration errors in the emit, since references that were needed were not preserved by default.

[jakebailey]

@typescript-bot pack this

[typescript-bot]

Starting jobs; this comment will be updated as builds start and complete.

Command	Status	Results
pack this	✅ Started	✅ Results

[typescript-bot]

Hey @jakebailey, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/160635/artifacts?artifactName=tgz&fileId=F9E2BFE8B70C68AFEF79932BEEF547DC885DD9935ECCD0249B1D1A4B70EC193402&fileName=/typescript-5.5.0-insiders.20240320.tgz"
    }
}

and then running npm install.

---

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.5.0-pr-57681-18".;

[jakebailey]

Tested jupyterlab out; adding preserve=true fixes all problems. All of the other errors seem like oddities with how we attempt to build the project without actually invoking their build system.