Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

package BH boost errors env #405

Closed
HannesOberreiter opened this issue May 27, 2020 · 8 comments
Closed

package BH boost errors env #405

HannesOberreiter opened this issue May 27, 2020 · 8 comments

Comments

@HannesOberreiter
Copy link

HannesOberreiter commented May 27, 2020

Hi everyone,

I try to build a docker container (FROM rocker/tidyverse:4.0.0) which should use env for version control with caching and the package qpcR.

While installing / compiling the needed packages I run into some trouble, see error message below.

Don't know if this is a problem coming with the ROCKER container.

Thanks! Cheers Hannes

* installing *source* packagelater...
** packagelatersuccessfully unpacked and MD5 sums checked
** using staged installation
** libs
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG -pthread -DSTRICT_R_HEADERS -I'/home/rstudio/project/renv/library/R-4.0/x86_64-pc-linux-gnu/Rcpp/include' -I'/home/rstudio/project/renv/library/R-4.0/x86_64-pc-linux-gnu/BH/include' -I/usr/local/include   -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG -pthread -DSTRICT_R_HEADERS -I'/home/rstudio/project/renv/library/R-4.0/x86_64-pc-linux-gnu/Rcpp/include' -I'/home/rstudio/project/renv/library/R-4.0/x86_64-pc-linux-gnu/BH/include' -I/usr/local/include   -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c callback_registry.cpp -o callback_registry.o
In file included from callback_registry.cpp:1:
/home/rstudio/project/renv/library/R-4.0/x86_64-pc-linux-gnu/BH/include/boost/bind.hpp:22:10: fatal error: boost/bind/bind.hpp: No such file or directory
   22 | #include <boost/bind/bind.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [/usr/local/lib/R/etc/Makeconf:176: callback_registry.o] Error 1
ERROR: compilation failed for packagelater* removing/home/rstudio/project/renv/staging/1/laterInstalling promises [1.1.0] ...
	OK [built from source]
Installing httpuv [1.5.2] ...
	FAILED
Error installing package 'httpuv':

I also added my DOCKERFILE below.

FROM rocker/tidyverse:4.0.0

LABEL maintainer="hoberreiter@gmail.com"

COPY git_config.sh /etc/cont-init.d/gitconfig
COPY . /home/rstudio/project
RUN chown -R rstudio /home/rstudio/project

ENV RENV_VERSION 0.10.0
RUN Rscript -e "install.packages('remotes', repos = c(CRAN = 'https://cloud.r-project.org'))"
RUN Rscript -e "remotes::install_github('rstudio/renv@${RENV_VERSION}')"

# rgl package, used in qpcR package
RUN apt-get update -qq && apt-get -y --no-install-recommends install \
	libglu1-mesa-dev

# httpuv https://github.com/rstudio/shiny/issues/2073
RUN apt-get -y --no-install-recommends install \
	gfortran libreadline6-dev libx11-dev libxt-dev libcairo2-dev libbz2-dev liblzma-dev libcurl4-openssl-dev cmake

RUN mkdir /home/rstudio/.local
RUN mkdir /home/rstudio/.local/share
RUN mkdir /home/rstudio/.local/share/renv
RUN mkdir /home/rstudio/.local/share/renv/binary

# TODO dont chmod a+rwx? Cannot install binaries and project wihout it (env::init())
RUN chmod a+rwx /home/rstudio/.local
RUN chown -R rstudio /home/rstudio/.local
@eddelbuettel
Copy link
Member

I do not why the build error of a "randomly chosen" package off CRAN would be a Rocker bug.

Maybe it is renv, maybe it is later, maybe it is something else. I have no idea. I happen to also maintain BH but I see no issue here.

(You could use a little bit of Unix + Docker refinment. Instead of

RUN mkdir /home/rstudio/.local
RUN mkdir /home/rstudio/.local/share
RUN mkdir /home/rstudio/.local/share/renv
RUN mkdir /home/rstudio/.local/share/renv/binary

so a single

RUN mkdir -p /home/rstudio/.local/share/renv/binary

Ditto for

RUN chmod a+rwx /home/rstudio/.local
RUN chown -R rstudio /home/rstudio/.local

which I would write as

RUN chmod a+rwx /home/rstudio/.local \
     && chown -R rstudio /home/rstudio/.local

as each RUN becomes one layer.

That's not your bug though which is a different issue. I also do not understand why one would want renv (which freezes versions) inside Docker (which already does that) but I digress....

Lastly, we offer several venues to take advantage of prebuilt binaries to avoid whatever building issue you may run into...

I suggest we close this as I see no Rocker bug here (yet, so feel free to reformulate if you think Rocker is at fault).

@HannesOberreiter
Copy link
Author

HannesOberreiter commented May 27, 2020

Hi,

thanks for the response! Pretty new to the Docker environment and really appreciate your help.

I follow the guide https://rstudio.github.io/renv/articles/docker.html to use renv and docker, because I thinks it is a good idea.

Lastly, we offer several venues to take advantage of prebuilt binaries to avoid whatever building issue you may run into...

Could you give me some hints how i can do this?

Closing this Issue as you said.

Cheers
Hannes

@eddelbuettel
Copy link
Member

Here is one way:

edd@rob:~$ docker run --rm -ti rocker/r-ubuntu:18.04 bash
root@a70e39f754ff:/# apt update
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]                                                                       
Get:3 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic InRelease [15.4 kB] 
Get:4 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [931 kB]                          
Get:5 http://ppa.launchpad.net/marutter/rrutter3.5/ubuntu bionic InRelease [15.4 kB]                                                         
Get:6 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [59.3 kB]                                                                
Get:7 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [8,815 B]                                               
Get:8 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [852 kB]                        
Get:9 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main Sources [1,818 kB]                        
Get:10 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]        
Get:11 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]        
Get:12 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Get:13 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1,344 kB]      
Get:15 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]          
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [73.6 kB]    
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,228 kB]          
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [20.1 kB]     
Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,385 kB]      
Get:20 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [8,286 B]         
Get:21 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [8,158 B]     
Get:22 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 Packages [877 kB]    
Get:23 http://ppa.launchpad.net/marutter/rrutter3.5/ubuntu bionic/main Sources [9,707 B]
Get:24 http://ppa.launchpad.net/marutter/rrutter3.5/ubuntu bionic/main amd64 Packages [8,457 B]
Fetched 20.7 MB in 5s (3,765 kB/s)                   
Reading package lists... Done
Building dependency tree       
Reading state information... Done
100 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@a70e39f754ff:/# apt install r-cran-httpuv
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  r-cran-bh r-cran-later r-cran-magrittr r-cran-promises r-cran-r6 r-cran-rcpp r-cran-rlang
The following NEW packages will be installed:
  r-cran-bh r-cran-httpuv r-cran-later r-cran-magrittr r-cran-promises r-cran-r6 r-cran-rcpp r-cran-rlang
0 upgraded, 8 newly installed, 0 to remove and 100 not upgraded.
Need to get 12.4 MB of archives.
After this operation, 133 MB of additional disk space will be used.
Do you want to continue? [Y/n]  
Get:1 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-bh all 1.72.0-3-1cran1ppabionic0 [7,431 kB]
Get:2 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-rcpp amd64 1.0.4.6-1cran1ppabionic0 [1,936 kB]                                                                                   
Get:3 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-r6 all 2.4.1-1cran1ppabionic0 [61.9 kB]                                                                                          
Get:4 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-rlang amd64 0.4.6-1cran1.1804.0 [996 kB]                                                                                         
Get:5 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-later amd64 1.0.0-1cran1ppabionic0 [105 kB]                                                                                      
Get:6 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-magrittr all 1.5-5cran2ubuntu2~ubuntu18.04.1~ppa1 [149 kB]                                                                       
Get:7 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-promises amd64 1.1.0-1cran1ppabionic0 [1,197 kB]                                                                                 
Get:8 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 r-cran-httpuv amd64 1.5.2-1cran1ppabionic0 [510 kB]                                                                                     
Fetched 12.4 MB in 20s (632 kB/s)                                                                                                                                                                                 
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package r-cran-bh.
(Reading database ... 19372 files and directories currently installed.)
Preparing to unpack .../0-r-cran-bh_1.72.0-3-1cran1ppabionic0_all.deb ...
Unpacking r-cran-bh (1.72.0-3-1cran1ppabionic0) ...
Selecting previously unselected package r-cran-rcpp.
Preparing to unpack .../1-r-cran-rcpp_1.0.4.6-1cran1ppabionic0_amd64.deb ...
Unpacking r-cran-rcpp (1.0.4.6-1cran1ppabionic0) ...
Selecting previously unselected package r-cran-r6.
Preparing to unpack .../2-r-cran-r6_2.4.1-1cran1ppabionic0_all.deb ...
Unpacking r-cran-r6 (2.4.1-1cran1ppabionic0) ...
Selecting previously unselected package r-cran-rlang.
Preparing to unpack .../3-r-cran-rlang_0.4.6-1cran1.1804.0_amd64.deb ...
Unpacking r-cran-rlang (0.4.6-1cran1.1804.0) ...
Selecting previously unselected package r-cran-later.
Preparing to unpack .../4-r-cran-later_1.0.0-1cran1ppabionic0_amd64.deb ...
Unpacking r-cran-later (1.0.0-1cran1ppabionic0) ...
Selecting previously unselected package r-cran-magrittr.
Preparing to unpack .../5-r-cran-magrittr_1.5-5cran2ubuntu2~ubuntu18.04.1~ppa1_all.deb ...
Unpacking r-cran-magrittr (1.5-5cran2ubuntu2~ubuntu18.04.1~ppa1) ...
Selecting previously unselected package r-cran-promises.
Preparing to unpack .../6-r-cran-promises_1.1.0-1cran1ppabionic0_amd64.deb ...
Unpacking r-cran-promises (1.1.0-1cran1ppabionic0) ...
Selecting previously unselected package r-cran-httpuv.
Preparing to unpack .../7-r-cran-httpuv_1.5.2-1cran1ppabionic0_amd64.deb ...
Unpacking r-cran-httpuv (1.5.2-1cran1ppabionic0) ...
Setting up r-cran-rcpp (1.0.4.6-1cran1ppabionic0) ...
Setting up r-cran-r6 (2.4.1-1cran1ppabionic0) ...
Setting up r-cran-bh (1.72.0-3-1cran1ppabionic0) ...
Setting up r-cran-rlang (0.4.6-1cran1.1804.0) ...
Setting up r-cran-magrittr (1.5-5cran2ubuntu2~ubuntu18.04.1~ppa1) ...
Setting up r-cran-later (1.0.0-1cran1ppabionic0) ...
Setting up r-cran-promises (1.1.0-1cran1ppabionic0) ...
Setting up r-cran-httpuv (1.5.2-1cran1ppabionic0) ...
root@a70e39f754ff:/# 

@HannesOberreiter
Copy link
Author

Thank you very much @eddelbuettel it works now. Build from from source now also works, probably was missing some dependancies.

Need to have a look how I can use the caching from renv in my workflow with the prebuild binaries, but you were a big help!

@cboettig
Copy link
Member

@HannesOberreiter can you provide a bit more information about what you are doing, e.g. your renv lockfile? Your Dockerfile above should build just fine, but it does not seem to be trying to install the qpcR package you mention.

Note that you can get binary installation on the image you're using with:

docker run --rm -ti rocker/tidyverse:4.0.0 bash
install2.r qpcR

This works because that image uses a CRAN mirror which has pre-built linux binaries for this distro. You can change the mirror to a snapshot so that you get the same versions each time if you prefer. We do that automatically for you on rocker versioned stack once a newer version of R is released. See https://packagemanager.rstudio.com/client/#/repos/1/overview

Of course this should work with renv too, but I need more context to be able to reproduce the error you show above.

@HannesOberreiter
Copy link
Author

@cboettig thank you for your answer. I try to build a workflow were I can use my own renv library cache but still produce reproducible code with smaller size containers.

I had in mind that the libraries should install on runtime renv::restore() and use my cache or if no cache is given it should pull the binaries from CRAN.

At the moment my "work-in-progress" project looks like this: github project with docker file connected to the dockerhub were it builds my github file automatically /r/hannesoberreiter/melt_graz.

I start up the container and link my cache folder, start renv::restore() and it should use the cache or load it from CRAN mirror.

I think there is something happening at the renv project were it should fetch prebuild binaries, but I do not fully understand if this is exactly my problem. rstudio/renv#430

@cboettig
Copy link
Member

@HannesOberreiter

RSPM is still quite new and it looks like the dust hasn't settled yet in renv (e.g. the open issue you've linked). Your renv lockfile has RSPM as the repository in some places, though RSPM is not defined at the top of the repositories list in the lockfile, which only defines CRAN. I'm not quite up-to-speed on renv at the moment, but that seems potentially problematic.

@HannesOberreiter
Copy link
Author

@cboettig thank you very much for the explanation! It sheds a little bit more light into my dark nights to understand how to best work with docker and rocker. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants