Unnecessary downloading of other repos when their notebooks are not being run #30
Comments
tlvu
added a commit
that referenced
this issue
Jan 23, 2025
…mmatically (#138) This is done via the `CONFIG_PARAMETERS_SCRIPT_URL` override. See the sample `jenkins-params-external-repos.include.sh` where I run external notebooks from https://github.com/roocs/rook/tree/master/notebooks. Changes: * Do not download repos not enabled to run (fixes #30). * Do not delete `setup.cfg` `tox.ini` and `pyproject.toml` files. * Do not hardcode the list of `archiveArtifacts` in `Jenkinsfile` because the list of repos is dynamic now. * (Breaking) Archived build artifacts of original notebooks moved to under `buildout/` dir, together with their corresponding `*.output.ipynb` files. Previously they were at the same checkout path as their repo. * This artifact filename format can be override by `CONFIG_OVERRIDE_SCRIPT_URL`, see demo in `test-override/jenkins-params-external-repos.include.sh` * `downloadrepos` is both a script and a library to centralize all helper functions in one place. Otherwise we would have helpers for downloading repos in one file and helpers for other tasks in other files. Backward compatibility is preserved when `downloadrepos` is used as a script in autodeploy of tutorial notebooks to the Jupyter env. * New ability to run post-processing steps at the end of Jenkins build, see demo in `test-override/jenkins-params-external-repos.include.sh` * `NBVAL_SANITIZE_CFG_FILE` and `DEFAULT_PRODUCTION_HOST` now can also be overridden by the external repo if needed. This PR is almost fixing #57. There is no easy way to programmatically change the Jenkins "build request UI" to add on-the-fly the new params for the external repo. I think we will have to have another separate `Jenkinsfile.other_repo`. This PR would help testing bird-house/birdhouse-deploy#455. @mishaschwartz This would be useful to you if you have notebooks for your own WPS services.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently raven and esgf-compute-api notebooks are not tested by default, but their repos are still being downloaded, wasting build time and network transfer for nothing.
The text was updated successfully, but these errors were encountered: