Compressed source seems to break CI workflows that install duckdb

[andrewGhazi]

Hi there -- I have a Carpentries training module that depends on duckdb and installs the package on a Github runner image to generate the online version. However the relatively recent addition of the xz compressed source directory seems to be breaking the Github actions that orchestrate this process. See the log here:

https://github.com/carpentries-incubator/bioc-scrnaseq/actions/runs/12036979601/job/33559404345#step:6:2590

By the time it gets to tar cvJf duckdb.tar.xz duckdb that directory seems to be gone. I guess it's deleted early somehow, maybe it's the cleanup script somehow running before configure or something? It's hard to tell exactly what's going on, but it does seem to be a relatively recent bug as another workflow from a few weeks ago that used duckdb_1.1.2 from CRAN succeeded.

Maybe this is my workflow's problem, but I thought I would flag the issue here since it was pretty tricky to track down. It would be nice not to have to pin the module to an old duckdb version.

[krlmlr]

Thanks. It looks like we need to make ./cleanup reentrant: check if .tar.xz exists, if yes, do nothing.

[andrewGhazi]

Hi there, unfortunately the update in 1.1.13-1 doesn't seem to have fixed the issue for me. I wish I could be of more help but I can't tell why the compression command is getting run given the update in #634 ... The error log is pretty much the same as last time:

  
  * installing *source* package ‘duckdb’ ...
  ** package ‘duckdb’ successfully unpacked and MD5 sums checked
  ** using staged installation
  + [ -f duckdb.tar.xz ]
  + which xz
  + [ -d .git ]
  + cd src
  + tar cvJf duckdb.tar.xz duckdb
  tar: duckdb: Cannot stat: No such file or directory
  tar: Exiting with failure status due to previous errors
  + dirname ./configure
  + cd ./src
  + [ -f  ]
  + cp include/to-tar.mk Makevars.duckdb
  + [ -f duckdb.tar.xz ]
  + tar xJf duckdb.tar.xz
  ** libs
  using C++ compiler: ‘g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0’
  using C++17
  make: *** No rule to make target 'duckdb/ub_src_catalog.o', needed by 'duckdb.tar'.  Stop.
  ERROR: compilation failed for package ‘duckdb’
  * removing ‘/home/runner/work/bioc-scrnaseq/bioc-scrnaseq/renv/profiles/lesson-requirements/renv/library/linux-ubuntu-jammy/R-4.4/x86_64-pc-linux-gnu/.renv/1/duckdb’
  install of package 'duckdb' failed [error code 1]

[PingpingZheng]

@andrewGhazi I had the same issue when install.packages("duckdb"): make: *** No rule to make target 'duckdb/ub_src_catalog.o', needed by 'duckdb.tar'. Stop. and install of package 'duckdb' failed [error code 1]. Somehow, the remotes::install_version("duckdb") works well for me. The downloaded version is also 1.1.3.1 Downloading package from url: https://cloud.r-project.org/src/contrib/duckdb_1.1.3-1.tar.gz

[krlmlr]

Forgot that ./cleanup starts in the package root, not in src .

[krlmlr]

Can you please check if #940 works for you?

[andrewGhazi]

Unfortunately the github workflows my project uses (which are required to conform to the rest of the Carpentries organization) doesn't allow me to point to or install specific commits from Github, it has to pull them from CRAN. So I can't test it at will unfortunately, but I will report back once the change percolates through to that point. Sorry I can't be of more help on this :(

[krlmlr]

Could you adapt the workflow in a fork?

You can get the current version from r-universe, which is a CRAN-like repository: https://duckdb.r-universe.dev/duckdb .

[andrewGhazi]

The version from r-universe seems to work 🎉

Details: With some customized actions that point at that repo, it pulled from r-universe and it did seem to get past the initial "can't find the source directory" problem and install successfully. The workflow then proceeded to turn around and attempt to install a second time from 1.1.3-1 on CRAN, which failed as expected, but that's the workflow's fault for ignoring the repos setting. I'm not sure why it does that, but it's not duckdb's fault. I'm going to fiddle around with it some more, but all that being said I'm 99% sure this issue is fixed by #940 . Thanks for your help!