Feat: add support for multi-systems

[GaetanLepage]

This PR addresses #400 by allowing to evaluate a PR/revision for multiple systems at once.
You can now provide the --systems option with a list of systems to evaluate:

• --systems current (default): legacy behavior
• --systems "x86_64-linux aarch64-darwin": evaluate for both intel linux and ARM darwin
• --systems all: evaluate on all four main platforms

Progress

• Overall refactoring, addition of the necessary abstractions
• Adapted all "output" features (report.json, report.md, stdout...)
• Support for --eval ofborg
• Support for --eval local (sequential)
• Support for --eval local (optionally parallel) -> Let's do this in a following PR. This one is already big enough.
• Flesh out documentation
• Final code polishing

Note

This is still a WIP. The tool works fine when using ofborg eval but I have not yet adapted to local evaluation.
The code is quite dirty for now and will require more polish once everything will be working.
Also, I am fully open to suggestions regarding the solution and coding style.

I am eager to receive some feedback !

Technical details

• Basically, adding this feature requires a pretty heavy refactoring.
  Every place where there was a list of attributes/packages/reports, we now need a dict mapping each system to this list.
• I also needed to change the code for the nix shell to build as it now combines the packages to build for all platforms in the final attribute list.
• The current --system option has been softly deprecated (warning + forwarding the value to --systems) to avoid breaking the workflow of existing users.
• Note: I also had to change the layout of the report.json.

cc @Mic92 @natsukium

[Mic92]

Changing report.json will need changes in nipkgs-update as well.

[Mic92]

From the code it looks like you handle local evaluation as well, no?

[GaetanLepage]

From the code it looks like you handle local evaluation as well, no?

No, the build_commit method of Review is broken and has not been adapted.

[GaetanLepage]

I have a sequential version working for --eval local.
Basically, it calls nix-env for each system, one after the other.
We could work on calling them all in parallel, but we would need to provide an option as having 4 of them simultaneously requires a lot of RAM (sometimes even 64GB is not enough).

[GaetanLepage]

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 342734

---

x86_64-linux

⏩ 2 packages marked as broken and skipped:

• python311Packages.bambi
• python311Packages.bambi.dist

✅ 16 packages built:

• python311Packages.arviz
• python311Packages.arviz.dist
• python311Packages.corner
• python311Packages.corner.dist
• python311Packages.flowmc
• python311Packages.flowmc.dist
• python311Packages.numpyro
• python311Packages.numpyro.dist
• python311Packages.pymc
• python311Packages.pymc.dist
• python312Packages.arviz
• python312Packages.arviz.dist
• python312Packages.corner
• python312Packages.corner.dist
• python312Packages.numpyro
• python312Packages.numpyro.dist

---

aarch64-linux

⏩ 2 packages marked as broken and skipped:

• python311Packages.bambi
• python311Packages.bambi.dist

✅ 14 packages built:

• python311Packages.arviz
• python311Packages.arviz.dist
• python311Packages.corner
• python311Packages.corner.dist
• python311Packages.numpyro
• python311Packages.numpyro.dist
• python311Packages.pymc
• python311Packages.pymc.dist
• python312Packages.arviz
• python312Packages.arviz.dist
• python312Packages.corner
• python312Packages.corner.dist
• python312Packages.numpyro
• python312Packages.numpyro.dist

---

x86_64-darwin

⏩ 16 packages marked as broken and skipped:

• python311Packages.arviz
• python311Packages.arviz.dist
• python311Packages.bambi
• python311Packages.bambi.dist
• python311Packages.corner
• python311Packages.corner.dist
• python311Packages.numpyro
• python311Packages.numpyro.dist
• python311Packages.pymc
• python311Packages.pymc.dist
• python312Packages.arviz
• python312Packages.arviz.dist
• python312Packages.corner
• python312Packages.corner.dist
• python312Packages.numpyro
• python312Packages.numpyro.dist

---

aarch64-darwin

⏩ 6 packages marked as broken and skipped:

• python311Packages.bambi
• python311Packages.bambi.dist
• python311Packages.flowmc
• python311Packages.flowmc.dist
• python311Packages.pymc
• python311Packages.pymc.dist

❌ 8 packages failed to build:

• python311Packages.arviz
• python311Packages.arviz.dist
• python311Packages.corner
• python311Packages.corner.dist
• python312Packages.arviz
• python312Packages.arviz.dist
• python312Packages.corner
• python312Packages.corner.dist

✅ 4 packages built:

• python311Packages.numpyro
• python311Packages.numpyro.dist
• python312Packages.numpyro
• python312Packages.numpyro.dist