CI: Improve error reporting with surefire reports

[luck3y]

Signed-off-by: Ken Wills kwills@redhat.com

[stuartwdouglas]

I don't think this works for PR's, as PRs run under a limited security context.

[luck3y]

I don't think this works for PR's, as PRs run under a limited security context.

The run in the context of the forked repo, not the one the PR is opened against. So does that mean it can't update the PR here? I've run into some stuff like that before, and its somewhat irritating.

I guess I can put a check in for when its running against its own fork, that would be helpful for local dev maybe?

[luck3y]

Also, yeah you're right, I'd forgotten PRs don't get to see any secrets, e.g. secrets.GITHUB_TOKEN, so this won't work like this, the proper way would be to use a personal access token maybe.

On the plus side, I can leave it in, and check that secrets.GITHUB_TOKEN isn't empty as a check for if it should run.

[luck3y]

Yeah, this will need more testing, I'll close for now.

[stuartwdouglas]

I had an idea about how to fix this, and it was to write a GH app that you install in the repo, and it reacts to the uploaded surefire result artifacts. No idea how possible/practical it is but its something I want for Quarkus where we have the same problem. I might spend some time on it.

[luck3y]

I think another approach could be to run a workflow in the context of the upstream repo via the cron scheduler, which looks for completed PR jobs and updates the status, that might work. The problem with a lot of this stuff is the docs are a bit thin.

You could use the same thing for labeling I think, I'll test that and see if it works.

I did something similar for building images using the deploy trigger here: https://github.com/wildfly/wildfly-s2i/blob/master/.github/workflows/deploy.yml which is triggered by exporting a PAT in your env and running https://github.com/wildfly/wildfly-s2i/blob/master/tools/github-deploy.sh

This is fine for a deployment, but it would suck for PRs, so I kind of hope the same security context would work in a workflow actually running in the upstream.

[luck3y]

Looks like cron is the suggested alternative: actions/starter-workflows#78