Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_runner: print coverage threshold errors in red #55911

Closed
wants to merge 1 commit into from

Conversation

avivkeller
Copy link
Member

Fixes nodejs/help#4504 by printing coverage threshold errors in red.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Nov 18, 2024
Copy link

codecov bot commented Nov 18, 2024

Codecov Report

Attention: Patch coverage is 20.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 88.00%. Comparing base (a2edde4) to head (b778f1b).
Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/test_runner/test.js 20.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #55911      +/-   ##
==========================================
- Coverage   88.49%   88.00%   -0.50%     
==========================================
  Files         653      653              
  Lines      187735   187739       +4     
  Branches    36181    35877     -304     
==========================================
- Hits       166141   165223     -918     
- Misses      14819    15700     +881     
- Partials     6775     6816      +41     
Files with missing lines Coverage Δ
lib/internal/test_runner/test.js 96.76% <20.00%> (-0.22%) ⬇️

... and 95 files with indirect coverage changes

---- 🚨 Try these New Features:

reporter.diagnostic(nesting, loc, `Error: ${NumberPrototypeToFixed(actual, 2)}% ${name} coverage does not meet threshold of ${threshold}%.`);
reporter.diagnostic(nesting,
loc,
`${red}Error: ${NumberPrototypeToFixed(actual, 2)}% ${name} ` +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we emit colors in the reporter events anywhere else? Doesn't doing so defeat the purpose of having a generic reporter interface?

I am in favor of highlighting this text in reporters that use colors though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. In the coverage table, colors are used to show whether something has a close to 100% coverage or not

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh ☹️ We should not do that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. In the coverage table, colors are used to show whether something has a close to 100% coverage or not

If I'm not mistaken we colorise during the reporting, not directly in the emit.
I think that the problem here is that we shouldn't colorise the content of the emit

Copy link
Member

@MoLow MoLow Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should add a level parameter in diagnostics (i.e debug/info/warn/error) so reporters can implement coloring or other things

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, this PR isn’t needed, and a new one should be open implementing that, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

@@ -69,6 +69,7 @@ const { TIMEOUT_MAX } = require('internal/timers');
const { fileURLToPath } = require('internal/url');
const { availableParallelism } = require('os');
const { innerOk } = require('internal/assert/utils');
const { red, reset } = require('internal/util/colors');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the previous comment: util/colors was not used here before because the color is added in the reporting phase

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Test runner coverage thresholds must show up in red
5 participants