You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Tests should have failures logged in a format that Build Analysis can easily reasoned about and surfaced to the check tab.
Goals
Be able to identify and record catastrophic failures to Build Analysis:
Native Crashes
Hangs and Timeouts
Work Items
Look into the other format (other than trx) used by dotnet test and understand how it works.
A Yaml version was decided, so it has to be implemented in the test source generators. Helix compatibility with older OS's would be broken by adding new stuff, so this work item was changed to a XML log fixer.
Implement accordingly so the Helix script can easily understand and surface it.
Implement a watchdog to be able to easily identify when a test has taken too long and/or frozen. Suggested ideas: * [ ] Add it as a new feature in corerun: Create a new flag like --timeout. Have corerun exit and report the failure if this value is exceeded.
Add it as a new mini-app: This mini-app will receive the test's corerun command, and the timeout value as parameters. It will spawn a separate process to run the test in, and monitor it itself. If it exceeds the time limit, then i
t will kill the process and report the failure accordingly.
The text was updated successfully, but these errors were encountered:
Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.
Issue Details
Merge-on-Red Subproject
Tests should have failures logged in a format that Build Analysis can easily reasoned about and surfaced to the check tab.
Goals
Be able to identify and record catastrophic failures to Build Analysis:
Native Crashes
Hangs and Timeouts
Work Items
Look into the other format (other than trx) used by dotnet test and understand how it works.
Implement accordingly so the Helix script can easily understand and surface it.
Implement a watchdog to be able to easily identify when a test has taken too long and/or frozen. Suggested ideas:
Add it as a new feature in corerun: Create a new flag like --timeout. Have corerun exit and report the failure if this value is exceeded.
Add it as a new mini-app: This mini-app will receive the test's corerun command, and the timeout value as parameters. It will spawn a separate process to run the test in, and monitor it itself. If it exceeds the time limit, then i
t will kill the process and report the failure accordingly.
hoyosjs
changed the title
[Merge-on-Red] - Accurately log catastrophic test failures and freezes
[Merge-on-Red] - Accurately log catastrophic test failures and freezes in source-gened test infrastructure
Nov 1, 2022
Merge-on-Red Subproject
Tests should have failures logged in a format that Build Analysis can easily reasoned about and surfaced to the check tab.
Goals
Work Items
dotnet test
and understand how it works.A Yaml version was decided, so it has to be implemented in the test source generators.Helix compatibility with older OS's would be broken by adding new stuff, so this work item was changed to a XML log fixer.* [ ] Add it as a new feature in corerun: Create a new flag like--timeout
. Have corerun exit and report the failure if this value is exceeded.t will kill the process and report the failure accordingly.
The text was updated successfully, but these errors were encountered: