Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update recipe (build and testing) #40

Merged
merged 2 commits into from
Jul 12, 2019
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Update recipe (build and testing)
* Do not remove --as-needed from LDFLAGS
* Add mpiexec.sh wrapper to workaround O_NONBLOCK issues
* Add test for mpiexec with shell script
* Add test for mpifort compiler wrapper
  • Loading branch information
dalcinl committed Jul 4, 2019
commit 495c068a26c9221500e038f06eb0d27ead22df8e
19 changes: 8 additions & 11 deletions recipe/build-mpich.sh → recipe/build-mpi.sh
Original file line number Diff line number Diff line change
@@ -4,24 +4,20 @@
# with a fatal deprecation message pointing to FC
unset F90 F77

# remove --as-needed, which causes problems for downstream builds,
# seen in failures in petsc, slepc, and hdf5 at least
export LDFLAGS="${LDFLAGS/-Wl,--as-needed/}"
minrk marked this conversation as resolved.
Show resolved Hide resolved
export FCFLAGS="$FFLAGS"

# avoid absolute-paths in compilers
export CC=$(basename "$CC")
export CXX=$(basename "$CXX")
export FC=$(basename "$FC")

# from anaconda recipe, not sure if it matters
export FCFLAGS="$FFLAGS"

# avoid recording flags in compilers
# See Compiler Flags section of MPICH readme
export MPICHLIB_CFLAGS=$CFLAGS
unset CFLAGS
# TODO: configure ignores MPICHLIB_LDFLAGS
export MPICHLIB_CPPFLAGS=$CPPFLAGS
unset CPPFLAGS
export MPICHLIB_CFLAGS=$CFLAGS
unset CFLAGS
export MPICHLIB_CXXFLAGS=$CXXFLAGS
unset CXXFLAGS
export MPICHLIB_LDFLAGS=$LDFLAGS
@@ -33,19 +29,20 @@ unset FCFLAGS

# set some specific flags that we *do* want recorded in the compilers
# only the bare minimum of prefix-awareness here
export CFLAGS="-I$PREFIX/include"
export CPPFLAGS="-I$PREFIX/include"
export CFLAGS="-I$PREFIX/include"
export CXXFLAGS="-I$PREFIX/include"
export FFLAGS="-I$PREFIX/include"
export FCFLAGS="-I$PREFIX/include"
export LDFLAGS="-L$PREFIX/lib -Wl,-rpath,$PREFIX/lib"

export LIBRARY_PATH="$PREFIX/lib"

./configure --prefix=$PREFIX \
--disable-dependency-tracking \
--disable-wrapper-rpath \
--enable-cxx \
--enable-fortran
--enable-fortran \
--disable-wrapper-rpath

make -j"${CPU_COUNT:-1}"
make install
14 changes: 8 additions & 6 deletions recipe/meta.yaml
Original file line number Diff line number Diff line change
@@ -11,12 +11,12 @@ source:
sha256: 5db53bf2edfaa2238eb6a0a5bc3d2c2ccbfbb1badd79b664a1a919d2ce2330f1

build:
number: 1012
skip: True # [win]
number: 1013
skip: True # [win]

outputs:
- name: mpich
script: build-mpich.sh
script: build-mpi.sh
build:
run_exports:
- {{ pin_subpackage('mpich', max_pin='x.x') }}
@@ -32,9 +32,8 @@ outputs:
test:
script: run_test.sh
files:
- tests/test_exec.py
requires:
- python >=3
- mpiexec.sh
- tests/helloworld.sh

- name: mpich-mpicc
build:
@@ -51,6 +50,7 @@ outputs:
test:
script: run_test.sh
files:
- mpiexec.sh
- tests/helloworld.c

- name: mpich-mpicxx
@@ -68,6 +68,7 @@ outputs:
test:
script: run_test.sh
files:
- mpiexec.sh
- tests/helloworld.cxx

- name: mpich-mpifort
@@ -84,6 +85,7 @@ outputs:
test:
script: run_test.sh
files:
- mpiexec.sh
- tests/helloworld.f
- tests/helloworld.f90

4 changes: 4 additions & 0 deletions recipe/mpiexec.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash
set -euo pipefail
# pipe stdout, stderr through cat to avoid O_NONBLOCK issues
exec mpiexec "$@" 2>&1</dev/null | cat
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future, I think we can remove this. It's supposedly been fixed in mpich 3.2.1, but no harm keeping it for now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. After this PR is merged, we have to update to mpich 3.3.1, I was planning to do it by then.

Or do you prefer to jump now to 3.3.1 and forget about getting fresh builds for 3.2.1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the openmpi recipe, we still need the script to pass the --allow-run-as-root. Maybe it is a good idea to keep the mpiexex script in mpich as well, as a future last resort in case we are in need of similar tweaks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. No real difference between the script and the function that was here before. Still some arguments to consolidate. Without redirecting, an alias in run_test.sh ought to be enough.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, the use the function that was here before was broken, better to use a script in which we can control bash options. See this log https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=45392 and look for ./helloworld_c: error while loading shared libraries: libgfortran.so.4: cannot open shared object file: No such file or directory. So eventually the PR was merged but the package was broken.

64 changes: 37 additions & 27 deletions recipe/run_test.sh
Original file line number Diff line number Diff line change
@@ -1,48 +1,58 @@
#!/bin/bash
set -x

command -v mpichversion
mpichversion

command -v mpicc
mpicc -show

command -v mpicxx
mpicxx -show

command -v mpif90
mpif90 -show

command -v mpiexec
set -ex

export HYDRA_LAUNCHER=fork
MPIEXEC="${PWD}/mpiexec.sh"

pushd "tests"

function mpi_exec() {
# use pipes to avoid O_NONBLOCK issues on stdin, stdout
mpiexec -launcher fork $@ 2>&1 </dev/null | cat
}

if [[ $PKG_NAME == "mpich" ]]; then
mpi_exec -n 4 python test_exec.py
command -v mpichversion
mpichversion

command -v mpiexec
$MPIEXEC -n 1 mpivars
$MPIEXEC -n 4 ./helloworld.sh
fi

if [[ $PKG_NAME == "mpich-mpicc" ]]; then
command -v mpicc
mpicc -show

mpicc $CFLAGS $LDFLAGS helloworld.c -o helloworld_c
mpi_exec -n 4 ./helloworld_c
$MPIEXEC -n 4 ./helloworld_c
fi

if [[ $PKG_NAME == "mpich-mpicxx" ]]; then
command -v mpicxx
mpicxx -show

mpicxx $CXXFLAGS $LDFLAGS helloworld.cxx -o helloworld_cxx
mpi_exec -n 4 ./helloworld_cxx
$MPIEXEC -n 4 ./helloworld_cxx
fi

if [[ $PKG_NAME == "mpich-mpifort" ]]; then
mpif77 $FFLAGS $LDFLAGS helloworld.f -o helloworld_f
mpi_exec -n 4 ./helloworld_f
command -v mpifort
mpifort -show

mpifort $FFLAGS $LDFLAGS helloworld.f -o helloworld1_f
$MPIEXEC -n 4 ./helloworld1_f

mpifort $FFLAGS $LDFLAGS helloworld.f90 -o helloworld1_f90
$MPIEXEC -n 4 ./helloworld1_f90

command -v mpif77
mpif77 -show

mpif77 $FFLAGS $LDFLAGS helloworld.f -o helloworld2_f
$MPIEXEC -n 4 ./helloworld2_f

command -v mpif90
mpif90 -show

mpif90 $FFLAGS $LDFLAGS helloworld.f90 -o helloworld2_f90
$MPIEXEC -n 4 ./helloworld2_f90

mpif90 $FFLAGS $LDFLAGS helloworld.f90 -o helloworld_f90
dalcinl marked this conversation as resolved.
Show resolved Hide resolved
mpi_exec -n 4 ./helloworld_f90
fi

popd
5 changes: 5 additions & 0 deletions recipe/tests/helloworld.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash
set -eu
rank=$PMI_RANK
size=$PMI_SIZE
printf "Hello, World! I am process %d of %d.\n" $rank $size
2 changes: 0 additions & 2 deletions recipe/tests/test_exec.py

This file was deleted.