[dart2wasm] Empty 'names' field in generated source maps

[buenaflor]

Environment

• Dart 3.5.2
• Flutter 3.24.2
• macOS 14.4.1

Description

When building a project for web with wasm, the generated source map file has an empty "names" field. This won't allow mapping symbol names to stack frames, which hinders debugging capabilities.

I have checked this commit: 10742d9
which asserts the function name in the source_map_simple_test.dart so it should work (or at least used to work)

Maybe I'm also missing some setup detail.

Steps to Reproduce

Example 1: Flutter

1. Create a new Flutter web project for example through Android Studio
2. Run the command flutter build --wasm --source-maps
3. Examine the generated main.dart.wasm.map file

Example 2: Dart

1. Create a Dart Web project with dart create -t web mywebapp
2. Compile WASM: dart compile wasm web/main.dart
3. Examine the generated main.wasm.map file

Expected Behaviour

The "names" field in the main.dart.wasm.map file should contain symbol names that can be used to map to stack frames.

Actual Behaviour

The "names" field in the main.dart.wasm.map file is empty.

Example Sourcemap Snippet

{
  "version": 3,
  "sources": [
  	"... etc",
    "org-dartlang-sdk:///lib/_internal/wasm/lib/date_patch_patch.dart"
  ],
  "names": [],
  "mappings": [
	"... etc",
    "0r8BAuM+C,IAAd,EAAK,C,yBA4BvB,CACI,WAAF,CAAoB,WAAH,CAqE9B"
  ]
}

[dart-github-bot]

Summary: The generated source map for Dart/Flutter web projects compiled to WASM has an empty "names" field, preventing symbol name mapping and hindering debugging. This issue occurs despite a recent commit that should have ensured function names are included in the source map.

[mkustermann]

/cc @osa1

[osa1]

We used to generate names but wasm-opt doesn't support them and Chrome devtools doesn't use them, so I removed them in 1b1740e.

I had also asked around internally before removing them to see if there are any tools that use those names but couldn't find one.

@buenaflor which tool are you using that makes use of the names in source maps?

[buenaflor]

We used to generate names but wasm-opt doesn't support them and Chrome devtools doesn't use them, so I removed them in 1b1740e.

Ah okay got it.

which tool are you using that makes use of the names in source maps?

I was investigating the effort to use sourcemaps for flutter web wasm stacktraces for us at sentry.io. that's the main use case

related issue

[buenaflor]

I wanted to check out how much info the current wasm sourcemap will provide us at sentry but our sentry-cli validates sourcemaps so I cannot upload it, e.g error: bad reference to name #0

[osa1]

Could you share the instructions to generate the source map file that causes this error?

We don't generate names in the mappings (as described above) so I'm not sure how you can see a reference to a name.

[buenaflor]

We don't generate names in the mappings (as described above) so I'm not sure how you can see a reference to a name

Sry my bad, I tried it again and used the wrong sourcemap before. Works as intended.

Dunno if it's feasible for you to re-enable generating names generally or through an additional flag?

[osa1]

@buenaflor How do you plan to use the names and what how do you want the generated code to be named? (e.g. name code with the name of the function it belongs, or something else?)

[buenaflor]

In our Javascript SDKs and Flutter Web we use it to unminify the stacktrace for crash and error reporting.
So we are aiming for providing useful stacktraces to users. Ideally the stackframes would be able to map to its original function name

Example from our Flutter Web dart2js stacktrace (don't mind the stacktrace formatting):

StateError: Bad state: try catch
  at Object.c(main.dart.js:3494:19)
  at <fn>(main.dart.js:28168:9)
  at aEx.a(main.dart.js:4857:63)
  at aEx.$2(main.dart.js:38020:14)
  at Object.u(main.dart.js:4843:10)
  at V_.aFZ(main.dart.js:28185:10)
  at Gj.WL(main.dart.js:51878:14)
  at tear_off.<fn>(main.dart.js:3629:79)
  at ht.Xg(main.dart.js:48625:9)
  at ht.d3(main.dart.js:48630:21)

// unminified to:

StateError: Bad state: try catch
  at wrapException(org-dartlang-sdk:///dart-sdk/lib/_internal/js_runtime/lib/js_helper.dart:1198:19)
  at tryCatch(../../../lib/main.dart:786:5)
  at _wrapJsFunctionForAsync(org-dartlang-sdk:///dart-sdk/lib/_internal/js_runtime/lib/async_patch.dart:303:19)
  at dartProgram(main.dart.js:329:4)
  at _asyncStartSync(org-dartlang-sdk:///dart-sdk/lib/_internal/js_runtime/lib/async_patch.dart:233:3)
  at asyncThrows(../../../lib/main.dart:793:3)
  at _InkResponseState.handleTapCancel(../../../../../../../../Documents/flutter/packages/flutter/lib/src/material/ink_well.dart:1175:5)
  at A.aYf(org-dartlang-sdk:///dart-sdk/lib/_internal/js_runtime/lib/js_helper.dart:2257:9)
  at GestureRecognizer.invokeCallback(../../../../../../../../Documents/flutter/packages/flutter/lib/src/foundation/assertions.dart:407:10)
  at dartProgram(main.dart.js:338:6)

[osa1]

If you build your Flutter app with --no-strip-wasm you should see function names in your stack traces.

Example:

f() {
  g();
}

g() {
  throw 'hi';
}

void main() {
  f();
}

Output:

hi
    at Error._throwWithCurrentStackTrace (wasm://wasm/0008a6b2:wasm-function[119]:0xe5d3)
    at g (wasm://wasm/0008a6b2:wasm-function[587]:0x1ceeb)
    at f (wasm://wasm/0008a6b2:wasm-function[586]:0x1cedf)
    at main (wasm://wasm/0008a6b2:wasm-function[583]:0x1cec7)
    at main tear-off trampoline (wasm://wasm/0008a6b2:wasm-function[585]:0x1ced9)
    at _invokeMain (wasm://wasm/0008a6b2:wasm-function[70]:0xb0ff)
    at InstantiatedApp.invokeMain (/usr/local/google/home/omersa/dart/sdk/test/test.mjs:361:37)
    at main (/usr/local/google/home/omersa/dart/sdk/sdk/pkg/dart2wasm/bin/run_wasm.js:416:21)
/usr/local/google/home/omersa/dart/sdk/sdk/pkg/dart2wasm/bin/run_wasm.js:346: [object WebAssembly.Exception]

You can see the names Error._throwWithCurrentStackTrace, g, f, main.

Flutter drops the relevant part of the .wasm files so you have to pass --no-strip-wasm to see the names, e.g. flutter build web --wasm --no-strip-wasm.

Does this work for you?

[buenaflor]

Flutter drops the relevant part of the .wasm files so you have to pass --no-strip-wasm to see the names, e.g. flutter build web --wasm --no-strip-wasm.

It definitely works as a temporary solution (generally this issue is not a very high priority for us currently so it's fine right now), but it does impact the wasm file size quite a bit for users so imo long term it'd be more ideal to strip the symbols and leverage the sourcemaps

[osa1]

Do you care about only the .wasm file size or size of the whole package with the .wasm file and the .map file?

If you care about the whole package we could add a flag for just putting function names in the names section and avoiding the other names (names for globals, types, ...). That way the total package size would be very similar the package with no names in .wasm but names in the .map.

[buenaflor]

We mainly care about things people are deploying to production, we don't necessarily want to force them to not strip symbol names and thus increase their app size.

I think your proposed solution for adding a flag sounds good

[osa1]

Just to confirm, assume that function names will add the same amount of bytes to either the .wasm files or the .wasm.map files.

Is there a difference for you between adding them to the .wasm files vs. to .wasm.map files?

[buenaflor]

At Sentry for processing it does not make a difference, but for our customers who use our Flutter SDK for error and crash reporting it does matter.

When you add this flag we would set up the guide like so: In order to symbolicate your stacktraces for Flutter Web WASM you need to build the app with the command: flutter build web --wasm --new-flag

So for the end user's sake it's preferrable to add them to wasm.map files. Hope that makes sense

[osa1]

I still don't understand why you prefer the names to be in the .map files.. Since end users seem to be shipping both of these files (otherwise adding them to the .map files wouldn't make any difference) what is the advantage of adding function names to the .map files instead of .wasm files?

[buenaflor]

Sry for the confusion, here would be the full process:

1. Sentry SDK usage

• A developer creates a Flutter WASM app and integrates the Sentry SDK.
• They build their app for production using flutter build web --wasm.
• The developer uploads the sourcemaps (e.g., main.dart.wasm.map) to Sentry

2. Deployment

• The production build (without sourcemaps) is deployed to the web server.
• This production build has names stripped out, resulting in smaller file sizes.

3. Error Handling

• When an error occurs in production, the Sentry SDK captures it.
• The error data, including the minified/unsymbolicated stacktrace, is sent to Sentry's servers.

4. Symbolication:

• Sentry uses the previously uploaded sourcemaps to symbolicate the minified stacktrace.
• This process turns the unreadable stacktrace into a human-readable one with function names, line, column.

The advantage of having the names in the sourcemap in this case is no size overhead in production: The deployed WASM files are smaller because it is stripped.

[osa1]

Thanks for the detailed explanation.

I agree that for this use case it makes sense to add names to the source maps.

Just for the record, I tried to disable all other names and only leave function names, but the binary size overhead is still huge. In devtools: (optimized)

• Without any names: 5,203,573 bytes.
• With only function names: 9,269,110 bytes. (+78%)
• With all names: 9,666,761 bytes. (+4.2%)

So we really need to drop all names in the release builds.

One question is what in the generated code give names to.

AFAIK dart2js adds names to functions and (s (in ES6 methods) in the generated file.

In .wasm files we could map the length prefix of each function, or first byte of the function code (i.e. the first byte after the length prefix of a function).

@buenaflor Given a stack trace from a browser and a source map, how do you search for the names? Do you start with an offset in the stack trace, and scan backwards in the generated code (.js or .wasm) until you see a mapped code with a name?

EDIT: I think it makes sense to just map the whole function code to a name. It adds only one byte to each mapping in the function.

[buenaflor]

Given a stack trace from a browser and a source map, how do you search for the names? Do you start with an offset in the stack trace, and scan backwards in the generated code (.js or .wasm) until you see a mapped code with a name?

Sry for the late response, do you still need this info? I'm not too familiar with our symbolizer, I'll have to ask another team for that

[sigmundch]

Do you start with an offset in the stack trace, and scan backwards in the generated code (.js or .wasm) until you see a mapped code with a name?

FWIW, this is common practice for JavaScript soruce-maps, and we use this in our deobfuscation techniques both internally and in the sample implementation in our dart2js_tools/lib/deobfuscate_stack_trace.dart tool. The deobfuscation method we provide does precisely the backward lookup that you just described.

@osa1 - This reminds me something I wanted to ask. Does dart2wasm or wasm-opt do any method inlining? If so, it may be worth discussing mechanisms to help unwind the inlined frames in the symbolized/deobfuscated stack trace. In dart2js we had to introduce source-map extensions to support it.

[osa1]

Thanks for the details @sigmundch.

Does dart2wasm or wasm-opt do any method inlining? If so, it may be worth discussing mechanisms to help unwind the inlined frames in the symbolized/deobfuscated stack trace. In dart2js we had to introduce source-map extensions to support it.

Both dart2wasm and wasm-opt do it and currently we don't have a way of preserving the frames when inlining.

I think a problem with this is we would have to implement a dart2wasm-specific source map feature in binaryen as well, which is a generic tool and may not be open to adding language/toolchain specific features.

Is there a description of dart2js's source map extension somewhere?

[sigmundch]

Is there a description of dart2js's source map extension somewhere?

Yes, our extensions are documented in pkg/compiler/doc/sourcemap_extensions.md. I describe them briefly on this comment #52483 (comment). We also have an internal document with more background and motivation explaining the design at go/robust-dart2js-deobfuscation.

It's worth noting that there is appetite for evolving the source maps spec to support inilining data and other information as part of the official source-map format. Inline data is tracked in tc39/ecma426#40. However, there is an active proposal in https://github.com/tc39/source-map/blob/main/proposals/scopes.md under consideration that includes that among other features. Depending on how that progresses, I could see this evolving in a direction where dart2wasm and binaryen could support this in a language agnostic way.

[osa1]

With dart2wasm we also have the option of using DWARF instead of source maps, which supports inlining.

As far as I can gather from this comment Chrome may already be supporting debugging with DWARF. Binaryen supports it as well, but with caveats. Since we are discussing debugging optimized builds (otherwise we can keep using source maps and just not inline), and for optimizations we rely on Binaryen, I don't know which one would be more feasible: (1) implementing dart2wasm-specific things in Binaryen (2) improving Binaryen's DWARF handling in optimizations.

I suspect (2) makes more sense. It would help other projects that use Binaryen, too.

Between tc39/ecma426#40 and DWARF, I don't know which one would be preferable for dart2wasm.

[osa1]

@buenaflor we now generate function names in the mappings. You can follow the tags in the commit referenced above to see which SDK versions will have the commit.

[buenaflor]

@osa1 thanks for the effort 🙇