Generate CI matrix at runtime

[zanieb]

Generates the CI matrix dynamically so we can run subsets on pull requests.

• Moves the matrix definition out of GitHub workflows into a ci-targets.yaml file
• Adds a ci-matrix.py script which parses the ci-targets.yaml file and outputs a JSON matrix
• Updates the GitHub Actions workflows to use the script to generate a matrix dynamically
• Uses the labels on the pull request to allow subsetting of the matrix

For example, you can run the matrix generator locally to see the label subsetting in action:

❯ uv run ci-matrix.py --label arch:x86_64,platform:linux,libc:gnu,build:freethreaded,build:lto | jq
Reading inline script metadata from `ci-matrix.py`
{
  "include": [
    {
      "arch": "x86_64",
      "target_triple": "x86_64-unknown-linux-gnu",
      "platform": "linux",
      "libc": "gnu",
      "run": "true",
      "python": "3.13",
      "build_options": "freethreaded+pgo+lto"
    },
    {
      "arch": "x86_64",
      "target_triple": "x86_64_v2-unknown-linux-gnu",
      "platform": "linux",
      "arch_variant": "v2",
      "libc": "gnu",
      "run": "true",
      "python": "3.13",
      "build_options": "freethreaded+pgo+lto"
    },
    {
      "arch": "x86_64",
      "target_triple": "x86_64_v3-unknown-linux-gnu",
      "platform": "linux",
      "arch_variant": "v3",
      "libc": "gnu",
      "run": "true",
      "python": "3.13",
      "build_options": "freethreaded+pgo+lto"
    },
    {
      "arch": "x86_64",
      "target_triple": "x86_64_v4-unknown-linux-gnu",
      "platform": "linux",
      "arch_variant": "v4",
      "libc": "gnu",
      "python": "3.13",
      "build_options": "freethreaded+lto"
    }
  ]
}

I'll add labels for

• platform:darwin
• platform:linux
• platform:windows
• python:3.9
• python:3.10
• python:3.11
• python:3.12
• python:3.13
• build:debug
• build:pgo
• build:lto
• build:noopt
• build:freethreaded
• arch:x86_64
• arch:aarch64
• arch:armv7
• arch:s390x
• arch:ppc64le
• arch:x86
• libc:gnu
• libc:musl

In a follow-up, I'll update this to use different runners in forks as noted in #426

[indygreg]

I did not realize this magic was possible. Otherwise I would have done this months ago.

[zanieb]

It's fairly new (though at least a year old). I'm glad you're into it!

[zanieb]

Another note for the future, we could roll these into a single workflow. I don't know if it's worth it though. We can also use this to automatically split the matrix if we exceed the matrix job limit again (i.e., when adding new variants).

[zanieb]

I want to validate that all the expected artifacts are here before merging, seems like a bit of a pain but I don't want to drop something?

edit: I validated these manually, oof.

[indygreg]

Do we need single quotes here to avoid shell injection/escape?

[zanieb]

As in, via the labels? I don't think people can create arbitrary labels in the organization, and I don't think commits on main could be targeted — but I'm happy to switch to single quotes to make it more robust anyway.