ci: re-run failed test steps with debug logging enabled

[JKRhb]

This PR introduces a retry mechanism with enabled debug logging for failed CI testing steps as discussed in #1184 (comment).

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (be1fd88) 76.66% compared to head (b60a0b5) 76.64%.
Report is 8 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1189      +/-   ##
==========================================
- Coverage   76.66%   76.64%   -0.02%     
==========================================
  Files          80       80              
  Lines       16813    16821       +8     
  Branches     1618     1619       +1     
==========================================
+ Hits        12890    12893       +3     
- Misses       3893     3898       +5     
  Partials       30       30              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[JKRhb]

Enabling debug logging on a second try does seem to work :) See this run for example: https://github.com/eclipse-thingweb/node-wot/actions/runs/7165425102/job/19507350139?pr=1189

[relu91]

Should we go for test:only the second time? or do we want always the coverage? even for debugging?

[danielpeintner]

That may become very handy!

[JKRhb]

Should we go for test:only the second time? or do we want always the coverage? even for debugging?

Hmm, I would assume that if the retry should succeed using the test:only script. this would cause the codecov step to fail instead 🤔 However, always failing the retry eventually to signal that something went wrong might be the right approach here anyway, I suppose?

[relu91]

However, always failing the retry eventually to signal that something went wrong might be the right approach here anyway, I suppose?

Yes, I think we should always fail if we re-try... but maybe not because the coverage step should fail... we should signal it better. Is it possible to configure the action to fail if it re-tries?

[JKRhb]

However, always failing the retry eventually to signal that something went wrong might be the right approach here anyway, I suppose?

Yes, I think we should always fail if we re-try... but maybe not because the coverage step should fail... we should signal it better. Is it possible to configure the action to fail if it re-tries?

This should now be incorporated :) "Unfortunately", the MQTT tests have not been failing anymore so the behavior in this specific case could not really be tested by me so far. However, after triggering a "manual" the updated action step (more or less) showed the correct behavior.

Update: I now managed to successfully test the expected behavior in this workflow run (with a slightly adjusted workflow file): https://github.com/eclipse-thingweb/node-wot/actions/runs/7173674190/job/19533532357?pr=1189

[JKRhb]

Hmm, as the failed CI run shows, the current approach does not work for Windows yet and needs to be slightly adjusted...