feat: support --via-ir flag when running forge coverage

[colinnielsen]

Component

Forge

Have you ensured that all of these are up to date?

• Foundry
• Foundryup

What version of Foundry are you on?

forge 0.2.0 (d896050 2022-10-21T00:08:06.615745Z)

What command(s) is the bug in?

forge coverage --via-ir --report lcov

Operating System

macOS (Apple Silicon)

Describe the bug

Looks like COMPILER OPTIONS are listed when forge coverage -h is run, however it appears that the coverage command does not build through the YUL pipeline.

[colinnielsen]

Looks like forge coverage also ignores the via_ir flag in the foundry.toml

[mattsse]

this is unfortunately a limitation, because --via-ir transforms the AST so there is no good way of keeping track afterwards.

[colinnielsen]

Shoot... Is there a way around this? I'm worried that this feature would be bricked once solidity moves to the IR pipeline.

[Oighty]

I've been having this same error without using --via-ir. My project compiles fine with forge build and forge test, but throws Stack too deep when running forge coverage. Also on Apple Silicon with forge 0.2.0 (7f4fb5 2022-11-02).

[mattsse]

this is unfortunately a limitation, because --via-ir transforms the AST so there is no good way of keeping track of the original AST afterward right now.

we should make this clearer on the website.

[frank-lim-partior]

this is unfortunately a limitation, because --via-ir transforms the AST so there is no good way of keeping track afterwards.

Is the need for tracking AST isolated to the test contracts? Or does it apply to the primary source code as well?

[0xrusowsky]

this is unfortunately a limitation, because --via-ir transforms the AST so there is no good way of keeping track afterwards.

I'm facing the same issue. Having this limitation in mind, do we have any alternative to see test coverage when getting Stack too deep error?

[PaulRBerg]

we should make this clearer on the website.

@mattsse what do you mean by "website"? The Foundry Book? Because if yes, I don't see any documentation about coverage on there.

I am not sufficiently experienced with test coverage software to understand why it is difficult (or impossible) to run coverage over IR-optimized code, but I agree with the commend made by @colinnielsen above - if coverage is not made to work with IR, it will not be as popular as it could be.

[haythemsellami]

My code compile without --via-ir, and tests works fine, but still getting stack too deep in coverage.

forge 0.2.0 (08a629a 2023-06-03T00:04:10.749254000Z)

[haythemsellami]

My code compile without --via-ir, but still getting stack too deep in coverage.

forge 0.2.0 (08a629a 2023-06-03T00:04:10.749254000Z)

This is an annoying error and probably will be a big blocker for Foundry adoption.Also, how to get more output from the CLI command, it does not show to me where the stack too deep error exactly is:

[⠰] Compiling...
[⠒] Compiling 100 files with 0.8.20
[⠘] Solc 0.8.20 finished in 4.24s
Error:
Compiler run failed:
Error: Compiler error (/solidity/libyul/backends/evm/AsmCodeGen.cpp:68):Stack too deep. Try compiling with `--via-ir` (cli) or the equivalent `viaIR: true` (standard JSON) while enabling the optimizer. Otherwise, try removing local variables. When compiling inline assembly: Variable headStart is 1 slot(s) too deep inside the stack. Stack too deep. Try compiling with `--via-ir` (cli) or the equivalent `viaIR: true` (standard JSON) while enabling the optimizer. Otherwise, try removing local variables.

[smak0v]

I have the same issue with forge coverage command.

[ethereumdegen]

I've been having this same error without using --via-ir. My project compiles fine with forge build and forge test, but throws Stack too deep when running forge coverage. Also on Apple Silicon with forge 0.2.0 (7f4fb5 2022-11-02).

i have the exact same issue

[golden-expiriensu]

Hi, I have the exact same issue, is this fix planned?

[kevinsslin]

I can pass forge build without --via-ir, but still occur the same error while running forge coverage.

Does anyone know how to fix it?

[Pai-Sho]

Same issue here

[QEDK]

Do we have any workarounds for this issue yet?

[Pai-Sho]

forge coverage --ir-minimum works

[sanbir]

forge coverage --ir-minimum is a life saver!

I also had to replace all my assembly { with assembly ("memory-safe") { for it to start working.

[NoahMarconi]

forge coverage --ir-minimum is a life saver!

I also had to replace all my assembly { with assembly ("memory-safe") { for it to start working.

@sanbir To confirm, your assembly blocks are in fact memory safe before you add the annotation?

[zerosnacks]

Looks like this is a current limitation of forge coverage because --via-ir transforms the AST so there is no good way of keeping track afterwards. Converted the ticket into a feature request so we can look into how we could possible overcome this limitation.

[mohib231]

forge coverage --ir-minimum is runnning with some limitations on --via-ir. we can use --ir-minimum as an alternate for sometime

[kartojal]

anyone facing memory leaks with --ir-minimum?

forge coverage --ir-minimum causes memory leaks while is compiling a large project, it runs out of memory with a 32gb machine

forge 0.2.0 (ea3ba89 2024-09-05T00:20:31.915792582Z)

logs:

$ forge coverage --ir-minimum
Warning! "--ir-minimum" flag enables viaIR with minimum optimization, which can result in inaccurate source mappings.
Only use this flag as a workaround if you are experiencing "stack too deep" errors.
Note that "viaIR" is only available in Solidity 0.8.13 and above.
See more: https://github.com/foundry-rs/foundry/issues/3357
[⠒] Compiling...
[⠢] Compiling 341 files with Solc 0.8.22
[⠰] Compiling 90 files with Solc 0.7.6
[⠒] Solc 0.7.6 finished in 61.74ms
Error: 
solc exited with signal: 9 (SIGKILL)
<empty output>

[siguint]

anyone facing memory leaks with --ir-minimum?
i have the same issue

[anajuliabit]

λ forge coverage --ir-minimum --skip test --skip script
Warning! "--ir-minimum" flag enables viaIR with minimum optimization, which can result in inaccurate source mappings.
Only use this flag as a workaround if you are experiencing "stack too deep" errors.
Note that "viaIR" is only available in Solidity 0.8.13 and above.
See more: https://github.com/foundry-rs/foundry/issues/3357
[⠊] Compiling...
[⠒] Compiling 311 files with Solc 0.8.19
[⠢] Compiling 43 files with Solc 0.6.12
[⠆] Compiling 12 files with Solc 0.8.10
[⠰] Solc 0.8.10 finished in 969.96ms
[⠔] Solc 0.6.12 finished in 614.18s
[⠒] Solc 0.8.19 finished in 614.78s
Error: 
Compiler run failed:
Error: Unknown key "inliner

--ir-minimum for me fails with Error: Unknown key "inliner

[grandizzy]

--ir-minimum should work, mind though that coverage disables optimizer (that is equivalent to setting optimizer = false in config file) so you could hit the behavior where test compiles but coverage fails (see #8840 (comment)). The Unknown key "inliner issue was solved with #9341
Also re memory issues when running coverage - there are big improvements in latest builds, please update and retry.