data.table Reverse Dependency (revdep) Checks
This is code for checking all reverse dependencies of R package data.table. It is designed to be run in the context of a SLURM cluster such as Monsoon at Northern Arizona University. Results are computed every day, and results for the past few days are available for inspection in the analyze directory.
params.R launches the revdep checks, by first building data.table from github master, and optionally rebuilding R itself, then
- using sbatch with a job array, each task for a different revdep to check via check_one.R
- using sbatch with
--depend=afterany:JOBIDto run analyze.R to make the output web page after all the checks finish.
So that the job is run every day, there are two options
- create a crontab entry which runs sbatch_params.sh, which will ask the cluster to run params.sh, which is generated in https://github.com/tdhock/data.table-revdeps/blob/master/params.R#L319 (this is what we do currently).
- using sbatch with
--begin=tomorrowat 00:01 to run params.R again, which will start the checks tomorrow (on a compute node, not done currently because we use the cron option above instead).
cron runs params.sh with a minimal environment which can be simulated for testing purposes via
env -i bash --noprofile --norc ~/bin/params.sh The scripts are currently highly specific to data.table and to the NAU Monsoon cluster. System libraries required to build R packages are installed in my conda environment and/or home directory:
th798@wind:~/src$ ls QuantLib-1.28 icu openssl-1.1.1s rasqal-0.9.33 SYMPHONY-5.6.17 icu4c-70_1-src.tgz raptor2-2.0.15 redland-1.0.17
ICU 70 is installed under $HOME (for redland), and ICU 58 is installed under $CONDA_PREFIX (for base R).
This software is highly experimental and has not been extensively tested.
- https://github.com/Rdatatable/data.table/blob/master/.dev/revdep.R implements revdep checking for data.table on your local computer.
- https://github.com/r-lib/revdepcheck has parallel processing for RStudio employees in the AWS cloud.
- https://github.com/r-devel/recheck runs on GitHub Actions, avoids long compilation times using pre-built ubuntu binaries of dependencies.
tools::check_packages_in_dir()is apparently what is used on CRAN.- in the table below “diff” means the software checks for differences in revdep check results, using the CRAN and GitHub versions of your package.
| Method | diff | R | OS |
|---|---|---|---|
| data.table-revdeps (this repo) | yes | release,devel | Monsoon=CentOS |
| revdep.R | ? | local | local |
| revdepcheck | yes | local | local |
| recheck | no | release | ubuntu:latest |
CRAN tools::check_packages_in_dir() | no | devel | debian:testing |