Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix docker builds #195

Closed
2 of 3 tasks
casperdcl opened this issue Jan 23, 2019 · 17 comments · Fixed by #209 or #214
Closed
2 of 3 tasks

fix docker builds #195

casperdcl opened this issue Jan 23, 2019 · 17 comments · Fixed by #209 or #214
Assignees
@casperdcl
Labels
in-review
Milestone
v2.0

Comments

@casperdcl
Copy link
Member

casperdcl commented Jan 23, 2019 •
edited
Loading

depends on #212 -> #213
@casperdcl casperdcl self-assigned this Jan 23, 2019
@casperdcl casperdcl mentioned this issue Jan 23, 2019
Merged
@casperdcl
Copy link
Member Author

casperdcl commented Feb 15, 2019 •
edited
Loading

ah this is actually caused by https://travis-ci.org/CCPPETMR/SIRF-SuperBuild/jobs/483407710#L5089

@paskino what's the current quickest way to install with SuperBuild? I was using ubuntu16.04 with as many system libs as possible. Will I have to -DUSE_SYSTEM_Boost=OFF or can I get away with upgrading to 18.04 and using system libs?

casperdcl added a commit that referenced this issue Feb 15, 2019
@casperdcl
ensure docker doesn't silently fail
b211772 
related to #195
@casperdcl casperdcl added this to the v1.1 milestone Feb 15, 2019
@casperdcl
Copy link
Member Author

sigh #91 would've fixed this issue

@KrisThielemans
Copy link
Member

I'm fine with upgrading the docker build to 18.04. Otherwise loads of trouble with Gadgetron. In any case, it's a container, so why not?

@KrisThielemans KrisThielemans modified the milestones: v1.1, v2.0 Feb 15, 2019
@KrisThielemans
Copy link
Member

@casperdcl, we're now ready for Ubuntu 18.04. Please use that (and the system boost and hdf5 etc). Currently, the docker build has a timeout.

@KrisThielemans
Copy link
Member

Currently, the docker build has a timeout.
Ignore this. I was checking the SIRF builds where the last job has a time-out. sorry

@KrisThielemans KrisThielemans mentioned this issue Feb 25, 2019
Merged
@casperdcl casperdcl reopened this Feb 25, 2019
@casperdcl
Copy link
Member Author

@KrisThielemans from .travis.yml, where do the following dependencies come from?

- libopenblas-dev
- libatlas-base-dev

Are they required for gadgetron?

@KrisThielemans
Copy link
Member

Not 100% sure. They're not mentioned on https://github.com/gadgetron/gadgetron/wiki/Linux-Installation. I think they are optional and will make some Gadgetron stuff faster. I see we have them in the VM as well (which I'm sure we copied into .travis.yml).

@rijobro has to use openblas on OSX as otherwise it found Apple's Accelerate library for BLAS which apparently had a bug. See gadgetron/gadgetron#719

@casperdcl
Copy link
Member Author

hmm they're not in docker atm, nor are -DUSE_ITK or ACE. Was trying to update things. Such #91 much fun.

@KrisThielemans
Copy link
Member

ACE has to be there somehow. Otherwise it will fail AFAIK (could be by default I guess) .

ITK is optional but useful to read nifti's and Dicoms etc with STIR.

what about for docker cloning CCPPETMR_VM and use some of its scripts? I guess we could do the same for Travis but I suppose it'll be faster to list them explicitly (not sure).

@casperdcl
Copy link
Member Author

I don't know - I'd be willing to deal with #91 now if you/@paskino are ready to merge the VM repo into this one, then it would be easier

@casperdcl
Copy link
Member Author

argh I'm also running into conda/conda#6030 now

@KrisThielemans
Copy link
Member

re merging CCPPETMR_VM here. I guess we can do this while keeping the old repo, so I don't mind really. However moving everything to the proposed structure seems quite (too) ambitious... I'm afraid I cannot help really. no time. sorry.

@casperdcl
Copy link
Member Author

OK I now have:

$ docker-compose -f docker-compose.yml -f docker-compose.nix.yml -f docker-compose.devel.yml build --build-arg UID=1000 --build-arg GROUPS=1000 sirf
Step 1/50 : FROM ubuntu:18.04 as base
...
Step 9/50 : FROM base as core
...
Step 31/50 : FROM core as sirf
...
Step 46/50 : RUN bash user_sirf-ubuntu.sh
$ git clone https://github.com/CCPPETMR/SIRF-SuperBuild --recursive -b master /opt/SIRF-SuperBuild
Cloning into '/opt/SIRF-SuperBuild'...
$ pushd /opt/SIRF-SuperBuild
/opt/SIRF-SuperBuild /home-away
$ echo $BUILD_FLAGS $EXTRA_BUILD_FLAGS
-DCMAKE_BUILD_TYPE=Release -DBUILD_STIR_WITH_OPENMP=ON -DUSE_SYSTEM_ACE=ON -DUSE_SYSTEM_Armadillo=ON -DUSE_SYSTEM_Boost=ON -DUSE_SYSTEM_FFTW3=ON -DUSE_SYSTEM_HDF5=ON -DUSE_ITK=ON -DBUILD_siemens_to_ismrmrd=ON -DUSE_SYSTEM_SWIG=ON -DDEVEL_BUILD=ON
$ cmake $BUILD_FLAGS $EXTRA_BUILD_FLAGS .
-- The C compiler identification is GNU 7.3.0
-- The CXX compiler identification is GNU 7.3.0
-- Check for working C compiler: /opt/pyvenv/bin/x86_64-conda_cos6-linux-gnu-cc
-- Check for working C compiler: /opt/pyvenv/bin/x86_64-conda_cos6-linux-gnu-cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /opt/pyvenv/bin/x86_64-conda_cos6-linux-gnu-c++
-- Check for working CXX compiler: /opt/pyvenv/bin/x86_64-conda_cos6-linux-gnu-c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- EXTERNAL_PROJECT_BUILD_TYPE: Release
-- Found PythonInterp: /opt/pyvenv/bin/python (found version "2.7.15")
-- Found PYTHON_EXECUTABLE=/opt/pyvenv/bin/python
-- Python version 2.7.15
-- Found PythonLibs: /opt/pyvenv/lib/libpython2.7.so (found version "2.7.15")
-- Found PYTHON_INCLUDE_DIRS=/opt/pyvenv/include/python2.7
-- Found PYTHON_LIBRARIES=/opt/pyvenv/lib/libpython2.7.so
-- Could NOT find Matlab (missing:  Matlab_INCLUDE_DIRS Matlab_MEX_LIBRARY Matlab_MEX_EXTENSION Matlab_ROOT_DIR MAIN_PROGRAM)
-- Python libraries found
-- SIRF and/or STIR Python modules will be installed in /opt/SIRF-SuperBuild/INSTALL/python
-- SuperBuild - First pass
-- Matlab_ROOT_DIR=
-- STIR_DIR=
-- NiftyReg_Binary_DIR=
-- SuperBuild - First pass - done
-- SuperBuild - CCPPETMR => Requires SIRF, STIR, Gadgetron, siemens_to_ismrmrd, NiftyReg,
-- Matlab_ROOT_DIR=
-- STIR_DIR=
-- NiftyReg_Binary_DIR=
-- SuperBuild -   SIRF => Requires STIR, Boost, HDF5, ISMRMRD, FFTW3, SWIG, NiftyReg,
-- SuperBuild -     STIR => Requires Boost, ITK,
-- SuperBuild -       Boost[OK]
-- Boost version: 1.65.1
-- USING the system Boost, using BOOST_LIBRARY_DIR=
-- SuperBuild -       ITK => Requires HDF5,
-- SuperBuild -         HDF5[OK]
-- HDF5: Using hdf5 compiler wrapper to determine C configuration
-- Found HDF5: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so;pthread;sz;z;dl;m (found suitable version "1.10.0.1", minimum required is "1.8")
-- USING the system HDF5, found HDF5_INCLUDE_DIRS=/usr/include/hdf5/serial, HDF5_C_LIBRARY_hdf5=/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so,HDF5_LIBRARIES=/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so;pthread;sz;z;dl;m
-- SuperBuild -       ITK[OK]
-- Adding project ITK
-- SuperBuild -     STIR[OK]
-- Adding project STIR
-- SuperBuild -     ISMRMRD => Requires HDF5[INCLUDED], Boost[INCLUDED], FFTW3,
-- SuperBuild -       FFTW3[OK]
FFTW3 UNIX libraries: FFTW3_LIBRARY-NOTFOUND
CMake Error at /opt/cmake/share/cmake-3.7/Modules/FindPackageHandleStandardArgs.cmake:138 (message):
  Could NOT find FFTW3 (missing: FFTW3_LIBRARY)
Call Stack (most recent call first):
  /opt/cmake/share/cmake-3.7/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  CMake/FindFFTW3.cmake:114 (find_package_handle_standard_args)
  SuperBuild/External_FFTW3.cmake:87 (find_package)
  CMake/ExternalProjectDependency.cmake:593 (include)
  SuperBuild/External_ISMRMRD.cmake:29 (ExternalProject_Include_Dependencies)
  CMake/ExternalProjectDependency.cmake:593 (include)
  SuperBuild/External_SIRF.cmake:37 (ExternalProject_Include_Dependencies)
  CMake/ExternalProjectDependency.cmake:593 (include)
  CMake/ExternalProjectDependency.cmake:639 (ExternalProject_Include_Dependencies)
  SuperBuild.cmake:210 (ExternalProject_Include_Dependencies)
  CMakeLists.txt:18 (include)

-- Configuring incomplete, errors occurred!
See also "/opt/SIRF-SuperBuild/CMakeFiles/CMakeOutput.log".
Makefile:45: recipe for target 'build' failed

Is this #147? I've already installed libfftw3-dev

@casperdcl
Copy link
Member Author

nvm, seems to be caused by conda install gxx_linux-64 which was my attempted fix of conda/conda#6030

@casperdcl
Copy link
Member Author 

_reg_maths_eigen.cpp:(.text._Z3svdIfEvPPT_mmPS2_S3_S3_[_Z3svdIfEvPPT_mmPS2_S3_S3_]+0x5e9): undefined reference to `GOMP_parallel'
/opt/SIRF-SuperBuild/INSTALL/lib/lib_reg_maths.a(_reg_maths_eigen.cpp.o):_reg_maths_eigen.cpp:(.text._Z3svdIfEvPPT_mmPS2_S3_S3_[_Z3svdIfEvPPT_mmPS2_S3_S3_]+0x610): more und
efined references to `GOMP_parallel' follow
collect2: error: ld returned 1 exit status
src/Registration/cReg/CMakeFiles/sirf_tensor_split_join.dir/build.make:126: recipe for target 'src/Registration/cReg/sirf_tensor_split_join' failed
make[5]: *** [src/Registration/cReg/sirf_tensor_split_join] Error 1
CMakeFiles/Makefile2:758: recipe for target 'src/Registration/cReg/CMakeFiles/sirf_tensor_split_join.dir/all' failed
make[4]: *** [src/Registration/cReg/CMakeFiles/sirf_tensor_split_join.dir/all] Error 2
make[4]: *** Waiting for unfinished jobs....
[ 67%] Building CXX object src/xSTIR/pSTIR/CMakeFiles/_pystir.dir/opt/SIRF-SuperBuild/sources/STIR/src/modelling_buildblock/modelling_registries.cxx.o
[ 68%] Building CXX object src/xSTIR/pSTIR/CMakeFiles/_pystir.dir/opt/SIRF-SuperBuild/sources/STIR/src/spatial_transformation_buildblock/spatial_transformation_registries.c
xx.o
[ 69%] Linking CXX shared module _pystir.so
[ 69%] Built target _pystir
make[3]: *** [all] Error 2
Makefile:138: recipe for target 'all' failed
CMakeFiles/SIRF.dir/build.make:118: recipe for target 'builds/SIRF/stamp/SIRF-build' failed
make[2]: *** [builds/SIRF/stamp/SIRF-build] Error 2
make[1]: *** [CMakeFiles/SIRF.dir/all] Error 2
CMakeFiles/Makefile2:306: recipe for target 'CMakeFiles/SIRF.dir/all' failed
Makefile:94: recipe for target 'all' failed
make: *** [all] Error 2

@KrisThielemans
Copy link
Member

KrisThielemans commented Feb 26, 2019 •
edited
Loading

hmmm. I suspect this is a NiftyReg issue. Maybe someone with Ubuntu 18.04 (non-docker) could confirm this.

Their CMake file doesn't add linker flags. I had the same in STIR where the -fopenmp flag wasn't passed, and added a work-around, but later found out this was the wrong thing to do: UCL/STIR#292.

Suggested work-around for now: add -DUSE_OPENMP:BOOL=OFF to NiftyReg build. If that works, could you then create a separate issue to resolve this later?

@casperdcl
Copy link
Member Author

Hah that solved it.

This was referenced Feb 27, 2019
Closed
Merged
casperdcl added a commit that referenced this issue Mar 2, 2019
@casperdcl
Merge branch 'docker'
6a896c5 
- fixes #194
- fixes #195
- reverts #209
- closes #214
- depends on #212 -> #213
paskino added a commit to paskino/SIRF-SuperBuild that referenced this issue Oct 25, 2021
@paskino
Build full cil (SyneRBI#195)
09dbbc4 
closes SyneRBI#192 
closes SyneRBI#181
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@casperdcl casperdcl

Labels
in-review
Projects
None yet
Milestone
v2.0
2 participants
@KrisThielemans @casperdcl