Merge branch 'main' into feat/cache-retrieval-of-remote-recipes

.github/workflows/build-and-rebuild.yaml

@@ -12,6 +12,10 @@ env:
   RATTLER_BUILD_ENABLE_GITHUB_INTEGRATION: true
   # Name of the production database
   REPRO_DB_NAME: repro.db
+  # Set the directory where the docs are stored
+  # because we try to be local first this needs to
+  # be explicitly set in CI
+  REPRO_DOCS_DIR: docs
   # For windows builds
   PYTHONUTF8: 1
 

.gitignore

@@ -182,3 +182,4 @@ build_outputs/*
 
 # local sql database
 repro.local.db
+docs.local/**

README.md

@@ -33,14 +33,14 @@ This is similar to what other projects in the [Reproducible Builds](https://repr
 If you have any more suggestions for variations, please let us know!
 
 ## Getting Started 🚀
-
 This project uses [pixi](https://github.com/prefix-dev/pixi) for project management.
 
 ### Prerequisite
 Complete the following steps only once:
 
 1. Install Pixi by following the instructions on https://prefix.dev/
-2. Clone the repository and navigate to the root directory:
+2. For building of local `rattler-build` versions install [rust](https://rustup.rs/). We might include this in the pixi configuration in the future.
+3. Clone the repository and navigate to the root directory:
     ```bash
     git clone https://github.com/prefix-dev/reproducible-builds # or ssh
     cd reproducible-builds
@@ -53,24 +53,51 @@ To see if the project is setup correctly, run the following command:
 ```bash
 # Build/Rebuild the boltons package
 pixi r reproduce boltons
+# .. rattler-build output ..
+# ...🕒🕒🕒🕒🕒
+# ... a while later ...
+Done: 'boltons'
+Verifying if rebuilds are reproducible...
+                                       Build Info
+┏━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
+┃ Platform ┃ Version ┃ Rattler Build Hash                                               ┃
+┡━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
+│ darwin   │ 22.3.0  │ 1c47eafc2fbd6bc6d8d629b40c4d2950e7de35a3f26712dd5d4eeba26dd33aba │
+└──────────┴─────────┴──────────────────────────────────────────────────────────────────┘
+ Is the Recipe Repro?
+┏━━━━━━━━━┳━━━━━━━━━━━┓
+┃ Name    ┃ Is Repro? ┃
+┡━━━━━━━━━╇━━━━━━━━━━━┩
+│ boltons │ Yes       │
+└─────────┴───────────┘
+# We have now successfully built the boltons package and verified that it is reproducible.
+```
+Once this is done we can generate the html page to view the results, similar as to what is shown at [Results](https://prefix-dev.github.io/reproducible-builds/). To do this run the following command:
+
+```bash
+pixi r serve-html
+✨ Pixi task (serve-html in html): livereload -d ./docs.local/
+Serving on http://127.0.0.1:35729
 ```
 
+You can now view the html page at http://127.0.1.1:35729. Changes to this page will automatically be picked up.
+
 ### Configuration
 #### Config.yaml (what recipes to build) ⚙️
 The `config.yaml` file contains the configuration for the project. You can add `remote` or `local` recipes, that are local to the project or remote repositories.
 The config also contains the rattler-build version that its using, this way we can depend on unreleased versions. This project automatically sets this up for you.
 
-### SQLite Database (stores the build information) 📕
+#### SQLite Database (stores the build information) 📕
 This project uses a Sqlite database to store the build information. The database is created automatically when the project is setup.
 When running locally a local version of the database is created, this will ensure that you have a clean database to work with.
 You can also use the `--in-memory-sql` flag to use an in-memory database, which is useful for testing.
 E.g `pixi run repror --in-memory-sql build-recipe boltons`, this will build the boltons recipe in an in-memory database.
 
-### Running locally 🏃‍♂️
+## Running locally 🏃‍♂️
 This project exposes a Python CLI called `repror` to interact with the project. We also re-expose the CLI using pixi tasks.
 
-#### Using the pixi tasks 📋
-Currently the following tasks are available for building/reproducing:
+### Using the pixi tasks 📋
+Currently the following tasks are available for **building/reproducing**:
 * `reproduce` builds and rebuilds a recipe for the current platform.
 * `build-recipe <name>` to build a recipe.
 * `build-recipe-skip` same as above but uses the `rattler-build` defined in the `pixi.toml`.
@@ -79,28 +106,32 @@ Currently the following tasks are available for building/reproducing:
 * `check` checks the database which recipes are reproducible.
 Note, that you can use the `--force` flag to force a rebuild of the recipe.
 
-There is also the following task for converting recipes:
+#### Recipe conversion
+There is also the following task for **converting recipes**:
 * `convert-recipe` that converts a `conda-build` `meta.yaml` to a `rattler-build` `recipe.yaml`.
 
+#### HTML generation
 A static html page is generated with the results of the builds, this can be found in the `docs` folder.
-to create the html page run the following task:
-* `generate-html` to generate the html page.
+to **create the html** page run the following task:
+* `generate-html` to generate the html page, from your local database.
+* `serve-html` to serve the html page. This will generate the html page first.
+There are additional `-prod` tasks that work on the production data, e.g. `generate-html-prod`.
 
+#### Testing
 Testing the repository can be done by running the following task:
 * `test` to run the pytests.
 
-
-#### Using the Python CLI directly 🐍
+### Using the Python CLI directly 🐍
 Sometimes, the tasks do not cover the full functionality of the CLI. In such cases, you can use the CLI directly. Make sure that the environment has been installed using `pixi install` or by running one of the tasks.
 To use the CLI directly, run the following command:
 
 ```bash
 Use `pixi r repror`
 ```
 
-This should give you some help on how to use the CLI.
+This should give you some `--help` on how to use the CLI.
 
-### Running on CI 🌎
+## Running on CI 🌎
 The project is setup to run on CI, and will build the recipes for the different platforms.
 
 ### Caching strategy
@@ -115,7 +146,7 @@ The CI has the following stages:
 * `patch-db` because the database is a SQLite database we cannot update per job, so we create metadata files that are `patched` into the database, the database is pushed to `main`, this step also generates the `index.html` file.
 
 
-### Contributing 🤝
+## Contributing 🤝
 Easiest way to contribute is to create a PR with a new recipe, by adding it to the `config.yaml` file, either through a `remote` or a `local` source. This way we can check
 the reproducibility for this specific recipe. This should also help us find `rattler-build` changes that we can make to prodoce more reproducible builds.