| layout | title | permalink |
|---|---|---|
page |
About |
/about/ |
This site contains reports generated using nix-review-tools for evaluations of major Hydra jobsets, which list the packages that failed to build by platform and problematic dependencies (packages whose failure to build caused other packages to fail to build due to being a dependency of that package). Reports are added/updated hourly.
The homepage contains a list of reports grouped by jobset. The date associated with each report is the date that jobset evaluation was started by Hydra.
For those unfamiliar with Hydra jobsets, each jobset usually corresponds to a branch/channel of nixpkgs. See the Nix channels article on the NixOS Wiki for more information on the relationship between Hydra jobsets and branches/channels, and Nix Channel Status for which major branches/channels correspond to which jobsets. For jobsets not listed on Nix Channel Status, see the description on the jobset list for the nixpkgs and nixos Hydra projects.
I'd like this to be a useful resource to the Nix community, as such, contributions and feedback are very welcome. Feel free to open an issue if you encounter a bug, have a feature request, or just want to provide some other type of feedback.
I'll do my best to fix bugs promptly, but can't guarantee I'll have the time to implement new features, etc. However, I'd certainly be happy to review/merge PRs with new features and improvements.
If there's a jobset you think would be valuable to add to the site, let me know by opening an issue, or better yet, submit a PR with the jobset added as a new job in gen-reports.yml.
Code for this project is on GitHub at malob/nix-review-tools-reports. The site itself is hosted using GitHub Pages, which uses Jekyll to automatically generate a static site based on the contents of the repository.
Scripts used to generate/manage the site's contents are located in flake.nix, which also includes a devShell output that creates a shell environment that contains said scripts, along with ruby (which includes bundler) required for local development of the Jekyll site.
Run nix develop to load the shell environment.
The site is updated automatically using GitHub workflows.
The gen-reports.yml workflow runs hourly. This workflow generates reports for the latest evaluation of Hydra jobsets by calling the reusable workflow, gen-report.yml, with a Hydra project and jobset name.
If a report for the latest evaluation of a jobset,
- hasn't been generated yet, it's created and added to the site by pushing a new commit which adds the new report to the _posts/ directory;
- has already been generated and the evaluation,
- was still in progress the last time the report was generated, the report is updated;
- had finished when the report was previously generated, the workflow does nothing since the report is already up to date.
When a report for an evaluation is added/updated, if that evaluation had finished prior to generating the report, "(succeeded)" is added to the end of the report title. Note that if a new evaluation for a jobset is started before the prior evaluation succeeds, the report for the prior evaluation won't be updated. As such, if a report on the site does not contain "(succeeded)" this does not necessarily mean that the evaluation didn't eventually succeed/finish.
When generating a report for a jobset evaluation, nix-review-tools downloads the Hydra report for each failed build. It's useful to keep a cache of these files since they take a while to download. If not cached, they would need to be re-downloaded whenever a report for a previously unfinished evaluation is updated (which can greatly increase report generation time, as well as increase the load on the Hydra servers unnecessarily). Failed builds also often carry over to new jobset evaluations, so maintaining a cache not only speeds up the process of updating existing reports, but also reports for new evaluations of a given jobset.
The gen-report script downloads these files into a data/ directory. Originally this directory was committed to the repository. However, this quickly resulted in the repository growing to many GBs in size, so this practice was discontinued in favor of using GitHub's cache action in this workflow.
(Note that nix-review-tools also downloads the Hydra report for the jobset evaluation itself, but these reports are deleted by gen-report so that they are re-downloaded on every workflow run. If these files were left in the cache, reports would never update since nix-review-tools would use the cached version of the Hydra evaluation report.)
The gen-report-manual.yml is run manually via a workflow_dispatch event, which can be triggered on its GitHub Action page, or with GitHub's CLI tool.
It has 3 inputs:
- Hydra project, e.g.,
nixpkgs. - Hydra jobset for project, e.g.,
staging-next. - Delete the build report cache for the jobset before report generation, which is a boolean.
Sometimes the cache of Hydra build reports restored by GitHub's cache action might contain reports for builds that failed, but were subsequently rerun manually on Hydra and succeeded. In those cases, the reports generated by nix-review-tools won't reflect this, and deleting the cache is required to resolve the discrepancy. (Note that report generation will take substantially longer, sometimes a few hours if the jobset evaluation contains a large number of failed builds, with the cache deleted.)
The rm-reports.yml workflow runs daily. This edits the Git history to remove all traces of reports generated over a week ago using the rm-reports-older-than script.
Old reports are removed because they, aren't particularly valuable, clutter up the site, and increase the time it takes Jekyll to generate the site. The Git history is edited rather than just deleting the files since otherwise the size of the repository grows very large over time.
To test/develop the site locally, load the development environment by running nix develop. You'll need to run bundle install the first time you do so to install the required Ruby dependencies (listed in Gemfile). A serve command is provided (effectively an alias for bundle exec jekyll serve --incremental), to quickly build and serve the site locally for testing.
This site currently uses Jekyll's default theme, Minima, it's not the prettiest but it does the job. Reports are placed in the _posts/ directory which Jekyll interprets as blog posts. The gen-report script runs nix-review-tools to generate reports (which are in Markdown format) and places the generated report in the _posts/ directory, prepended with a front matter metadata block which includes the following keys,
title, which is of the formtitle: [jobset name] [evaluation id] ["" or "(succeeded)"], e.g.,title: nixos:trunk-combined 1748954 (succeeded); andcategories, which contains the jobset name, e.g.,categories: nixos:trunk-combined, where categories are used to group evaluation reports for a given jobset together on the homepage.
Jekyll also expects the front matter block to include a layout key to indicate the layout the post should use, but this sites Jekyll configuration includes a plugin jekyll-default-layout which automatically sets the appropriate layout for any pages/posts that don't include the layout key in their front matter blocks.
Files in _posts/ must be of the form [YYYY-MM-DD]-[title].[markup], e.g., 2022-03-15-nixos_trunk-combined_1748954.md. The gen-report script uses the date the jobset evaluation was started when naming the file for a given report. The date in the file name is used by Jekyll as metadata for the post.
By default the Minima theme's homepage lists all posts in reverse chronological order based on the date mentioned above. Seeing the list of reports for each evaluation of each jobset in this way isn't particularly helpful. As such, the homepage's layout is overridden by _layouts/home.html to group all posts (reports) by the category (jobset).
Scripts used to generate/manage the sites contents are located in flake.nix. To load a shell environment containing these scripts run nix develop. Scripts can also be run using nix run, .e.g.:
❯ nix run .#jobset-latest-successful-eval-id -- nixpkgs trunk
1749064Outputs the ID of the latest successful/finished evaluation of a given jobset.
Usage: jobset-latest-successful-eval-id PROJECT JOBSET
❯ jobset-latest-successful-eval-id nixpkgs trunk
1749064Outputs the ID of the latest evaluation of a given jobset.
Usage: jobset-latest-eval-id PROJECT JOBSET
❯ jobset-latest-eval-id nixpkgs trunk
1749156Outputs the date that a given jobset evaluation was started.
Usage: jobset-eval-date EVALUATION_ID
❯ jobset-eval-date 1749156
2022-03-15Uses nix-review-tools to generate a report for the latest evaluation of a given jobset. The Hydra report for the evaluation and Hydra build reports (for failed builds during that evaluation) downloaded by nix-review-tools are placed in a data/ directory, and the report is output to a file in placed in a _posts/ directory with a name of the following form [YYYY-MM-DD]-[project]_[jobset]_[evaluation id].md (where the date is the date output by jobset-eval-date for the evaluation) prepended with the front matter metadata block as outline in the Jekyll section above. The Hydra evaluation report downloaded by nix-review-tools is then deleted.
Usage: gen-report PROJECT JOBSET
❯ gen-report nixpkgs trunkEdits the Git history to remove all traces of files in the _posts/ directory that were first added to the repository before a certain time.
Usage: rm-reports-older-than QUANTITY UNIT_OF_TIME
❯ rm-reports-older-than 2 weeks