add codecov threshold

[ericphanson]

to help with #477

docs: https://docs.codecov.com/docs/common-recipe-list#set-status-checks-to-block-coverage

[DilumAluthge]

Personally, I want us to keep enforcing that master must have 100% code coverage.

I think the better approach would be to figure out how to write unit tests to cover the last few lines that are uncovered by unit tests currently (but are covered by our integration tests).

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (9b7249f) 99.24% compared to head (85b2189) 99.24%.
Report is 2 commits behind head on dpa/merge-queue.

Additional details and impacted files

@@               Coverage Diff                @@
##           dpa/merge-queue     #480   +/-   ##
================================================
  Coverage            99.24%   99.24%           
================================================
  Files                   12       12           
  Lines                  395      395           
================================================
  Hits                   392      392           
  Misses                   3        3           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ericphanson]

I wonder if this configuration would work for that?

[DilumAluthge]

Hmmm, so in the current state of this PR, it would allow the merge_group job to have code coverage strictly less than 100%, right?

[ericphanson]

I don't really understand if this is about the branch we are merging from (/ running on), or the branch we are merging into. Either way I think this is incomplete though. If it is the branch we are running on, then we want the merge-queue branch to have the strict checks, not just main itself. If it is the branch we are merging into, then this won't help for PRs to main.

[ericphanson]

I think maybe it does support regex actually. The docs are very incomplete but I found some old ones that suggested it does (and some older ones that suggested it didn't then)

[ericphanson]

Ok, I had to fix the syntax a bit, but this now validates:

curl -X POST --data-binary @codecov.yml https://codecov.io/validate
Valid!

{
  "coverage": {
    "status": {
      "project": {
        "main": {
          "branches": [
            "^main$",
            "^gh-readonly-queue/main/.*"
          ],
          "target": 100.0
        },
        "default": {
          "target": 98.0
        }
      }
    }
  }
}

[DilumAluthge]

How would we go about testing this to make sure it works the way we want?

[ericphanson]

I think the easiest way is just by merging things. We can test the happy path by merging this and your PR down. Then we can test the failure path by deleting the integration tests and seeing if that merges down. If so we can revert back.