scopes: Encode original scope trees and use live ranges for bindings

[szuend]

This PR is intended to kick-off discussion on how to fix the spec to properly support the scenarios pointed out in here and here.

The two large changes are:

1. In addition to the generated scopes, we now also encode the original scope tree for ever sources entry. definition was changed to point back to the corresponding original scope instead. This fixes the missing outer scope information for inlined function of the current proposal.
2. Bindings are handled as "live ranges" instead: Each original scope specifies which variables/symbols it declares. For each symbol we specify the generated code ranges where this symbol is available and which expression to use to retrieve it's values. This fixes cases like constant propagation where "per scope bindings" are ambiguous.

The PR also proposes an additional ("conceptual") specification of the scope information. What information to include is somewhat orthogonal to how to encode it. A TypeScript definition style format should facilitate better discussion on the former.

There are also a couple of smaller changes:

• isFunction flag was removed in favor of a "scope kind". Together with a name for non-function scopes (such as classes) would give debuggers the opportunity to build a complete scope view without having to look at anything other than the sourcemap.
• inherit parent bindings is no longer necessary as binding live ranges spell this out explicitly.
• skip when stepping and collapse is also not really necessary: If a generated scope has a corresponding original scope we should step through it and show it in a stack trace. I'm not sure if we need "per scope" decision for this outside of ignoreList. Maybe someone has an example?

Note that for originalScopes I opted not to group the scope items by line given that original source files are normally not minified and spelling out the line numbers might be more efficient. But encoding (as the rest) is very much up for debate.

[hbenl]

This PR is intended to kick-off discussion

Some initial thoughts:

• except for "live ranges", the existing proposal does support the scenarios you've linked to since it would also contain all original scopes (except those whose code doesn't appear in the generated code, e.g. due to tree shaking)
• encoding original scopes the way it's done in this PR could be more efficient because with the current scopes proposal information about original scopes may have to be repeated (see here)
• we also need a way to specify the generated ranges for original scopes that have been removed from the generated source (e.g. original: if (b) { this(); } else { that(); }, generated: b ? this() : that()) - with this PR we could only link from generated scopes to original scopes but not from a generated source range without a scope
• do we need arbitrary "live ranges"? In most cases those will be identical to the scope ranges, so repeating them would be inefficient. We could keep the bindings in the generated scope information but allow multiple values separated by locations, so a binding [name, [value1, location1, value2, location2, value3]] would mean: variable name has the value
  • value1 in the source range from the start of the scope to location1
  • value2 in the source range from location1 to location2
  • value3 in the source range from location2 to the end of the scope

A TypeScript definition style format should facilitate better discussion

Agreed.

[szuend]

Thanks for the feedback :)

we also need a way to specify the generated ranges for original scopes that have been removed from the generated source (e.g. original: if (b) { this(); } else { that(); }, generated: b ? this() : that()) - with this PR we could only link from generated scopes to original scopes but not from a generated source range without a scope

I forgot to spell this out explicitly. But GeneratedScope doesn't necessarily need to correspond to a "real" scope in the generated JS. It can also correspond to just a generated range.

do we need arbitrary "live ranges"? In most cases those will be identical to the scope ranges, so repeating them would be inefficient. We could keep the bindings in the generated scope information but allow multiple values separated by locations

Agreed. I came up with a similar change when I prototyped this PR in DevTools:

1. The authored scope only has an array of variables it declares
2. The corresponding generated scope has an array of the same length that describes the bindings. Each entry is either:
   - Some sentinel to mark "unavailable"
   - If the variable is alive in the whole scope just the expression to get it's value
   - Full ranges + expressions otherwise.

In Typescript terms: bindings: Array<null|string|{from: Pos, to: Pos, expr: string}[]>.

For the conceptual TypeScript style definition should we stick closer to the the encoding (with e.g. scope start/end markers), or rather on a higher level as proposed here? I can open a separate PR to add the .d.ts for the current proposal.

[hbenl]

I have updated my implementation using a scopes format similar to this PR. The main differences are:

• I've added ScopeKind to generated scopes and use a new kind "reference" for "generated scopes" that only reference an original scope but don't correspond to a scope in the generated code
• I've moved the bindings' expressions to the generated scopes referencing the original scopes (which contain the bindings' variable names)
• I haven't implemented "live ranges" yet (but that should be easy to add)

I prefer this format over what I had implemented previously for several reasons:

• shoehorning the information about original scopes into the generated scopes in the old format felt awkward
• the old format required duplicating original scope information in some situations
• the new algorithm for computing original frames and scopes feels more straightforward

[hbenl]

I think we should merge this PR to clarify that this is the current state of the scopes proposal. Details can be discussed in follow-up PRs.

[connor4312]

I don't see any definition for what the from and to in the BindingRange is intended to encompass, clarification may be useful. "the expression for the bindings value in this range" implies but doesn't state that it's the generated range where the binding is applicable

[connor4312]

Eager to use this :)

[pfaffe]

Would it be sensible to rename GeneratedScope to GeneratedRange to make it more obvious that it does not refer to a lexical scope?