renv::snapshot fails to capture development package installed from custom gitlab repo

[alketh]

Hey there,

I am developing a package which uses renv to snapshot the dependencies to generate docker images for developing/testing.

For some reason I am not able to automatically add the package itself to the renv::snapshot routine.

I use the following library paths during package development:

> .libPaths()
[1] "/home/alexander_keth/ivgtools/renv/library/R-3.6/x86_64-pc-linux-gnu"         
[2] "/home/alexander_keth/ivgtools/renv/sandbox/R-3.6/x86_64-pc-linux-gnu/f23d1376"

I use a deps.R file with

library(ivgtools)
renv::install("gitlab::ivg-analytics/packages/ivgtools@1.2.3", rebuild = T)

to ensure that renv::snapshot does add the dependency. Based on the R console log ivgtools@1.2.3 is correctly installed into the library:

> renv::install("gitlab::ivg-analytics/packages/ivgtools@1.2.3", rebuild = T)
Retrieving '<PATH_TO_GITLAB>' ...
	OK [downloaded 539.8 Kb in 0.26 seconds]
Installing ivgtools [1.2.3] ...
	OK [built from source in 7.1 seconds]
Moving ivgtools [1.2.3] into the cache ...
	OK [moved to cache in 12 milliseconds]
* Installed 1 package in 8.7 seconds.
	OK [built from source in 7.1 seconds]
Moving ivgtools [1.2.3] into the cache ...
	OK [moved to cache in 12 milliseconds]
* Installed 1 package in 8.7 seconds.

Nonetheless, calling renv::snapshot always results in ivgtools being droped from the renv.lock file:

> renv::snapshot()
The following required packages are not installed:

	ivgtools  [required by ivgslackr]

Consider reinstalling these packages before snapshotting the lockfile.

[kevinushey]

Can you confirm the expected library paths are being used when renv::install() is being invoked as well?

[kevinushey]

Also, do you not get an installation summary giving the library path to be used? For example, I see:

> renv::install("rlang")
# Downloading packages -------------------------------------------------------
- Downloading rlang from CRAN ...               OK [1.8 Mb in 0.13s]
Successfully downloaded 1 package in 2 seconds.

The following package(s) will be installed:
- rlang [1.1.4]
These packages will be installed into "~/Library/R/arm64/4.4/library".

# Installing packages --------------------------------------------------------
- Installing rlang ...                          OK [installed binary and cached in 0.13s]
Successfully installed 1 package in 0.19 seconds.

Note that the library being used is reported.

[alketh]

Thanks so much for the quick reply!

As it turns out I was using renv 0.17.3, I guess the paths are not reported here?

I upgraded to renv_1.0.11 and the paths to the lib are now reported correctly:

> renv::install("dplyr@1.1.4")
The following package(s) will be installed:
- dplyr     [1.1.4]
- glue      [1.8.0]
- lifecycle [1.0.4]
- rlang     [1.1.4]
These packages will be installed into "~/ivgtools/renv/library/R-3.6/x86_64-pc-linux-gnu".

Do you want to proceed? [Y/n]: Y

# Installing packages --------------------------------------------------------
- Installing glue ...                           OK [linked from cache]
- Installing rlang ...                          OK [linked from cache]
- Installing lifecycle ...                      OK [linked from cache]
- Installing dplyr ...                          OK [linked from cache]
Successfully installed 4 packages in 25 milliseconds.

I can also confirm that the correct version from ìvgtools 1.2.3 is installed in the library path (installed via renv 0.17.3):

cat ~/ivgtools/renv/library/R-3.6/x86_64-pc-linux-gnu/ivgtools/DESCRIPTION | grep Version
Version: 1.2.3

However, when I try to install with renv 1.0.11 I get the following error:

> renv::install("gitlab::ivg-analytics/packages/ivgtools@1.2.3", rebuild = T)
# Downloading packages -------------------------------------------------------
- Downloading ivgtools from GitLab ...          OK [539.8 Kb in 0.28s]
Error: package 'arrow' is not available
In addition: Warning messages:
1: In value[[3L]](cond) : Could not find any credentials
2: In value[[3L]](cond) : Could not find any credentials
Traceback (most recent calls last):
22: renv::install("gitlab::ivg-analytics/packages/ivgtools@1.2.3", 
        rebuild = T)
21: renv_retrieve_impl(packages)
20: handler(package, renv_retrieve_impl_one(package))
19: renv_retrieve_impl_one(package)
18: renv_retrieve_gitlab(record)
17: renv_retrieve_package(record, url, path)
16: renv_retrieve_successful(record, path)
15: (function() {
        repos <- if (is.null(desc$biocViews)) 
            getOption("repos")
        else renv_bioconductor_repos()
        renv_scope_options(repos = repos)
        renv_retrieve_successful_recurse(deps)
    })()
14: renv_retrieve_successful_recurse(deps)
13: renv_retrieve_successful_recurse_impl(remote)
12: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote), 
        force = force)
11: the$dynamic_objects[[id]] %||% {
        dlog("dynamic", "memoizing dynamic value for '%s'", id)
        value
    }
10: renv_retrieve_successful_recurse_impl_one(remote)
 9: renv_retrieve_impl_one(remote)
 8: withCallingHandlers(renv_available_packages_latest(package), 
        error = function(err) stopf("package '%s' is not available", 
            package))
 7: renv_available_packages_latest(package)
 6: stopf("package '%s' is not available", package)
 5: stop(sprintf(fmt, ...), call. = call.)
 4: .handleSimpleError(function (err) 
    {
        ...
    }, "package 'arrow' is not available", base::quote(NULL))
 3: h(simpleError(msg, call))
 2: stopf("package '%s' is not available", package)
 1: stop(sprintf(fmt, ...), call. = call.)

Which is odd because arrow is already available in the library:

cat ~/ivgtools/renv/library/R-3.6/x86_64-pc-linux-gnu/arrow/DESCRIPTION | grep Version
Version: 14.0.0.2

[kevinushey]

Is it possible that ivgtools itself depends on arrow, or has something like Remotes: <arrow> declaring an alternate upstream remote for the arrow package? Or could it depend on a newer (or alternate) version of the arrow package?

[alketh]

arrow is in fact the first dependency listed as Imports in ivgtools:

Imports:
    arrow,
    assertthat,
    aws.s3,
    blastula,
    coin,
    data.table,
    DBI,
    dplyr,
    fasttime,
    ...

[kevinushey]

I wonder if the fact that the version of 'arrow' on CRAN requiring R (>= 4.0) is tripping up renv here, since you're using an older version of R...

https://cran.r-project.org/web/packages/arrow/index.html

I can try to test and confirm if I see something similar.

[kevinushey]

Does your project have a DESCRIPTION file? I think I can reproduce with a similar example, using R 3.6.3:

renv::init()
renv::install("cpp11@0.4.7")
writeLines("Imports: cpp11", con = "DESCRIPTION")
renv::install("cpp11armadillo")

When I run this, I see:

> renv::install("cpp11armadillo")
Error: package 'cpp11' is not available

That is, this appears to be a bug in renv -- I'll try to learn more.