Update recipe (build and testing) #40

dalcinl · 2019-06-28T23:10:28Z

Checklist

Used a fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)

conda-forge-linter · 2019-06-28T23:10:33Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

dalcinl · 2019-06-29T12:00:18Z

@ax3l Here you have. I'm removing some legacy workarounds in this PR. The aarch64 tests fail badly because installing the mpich package does not bring in the lib{gcc|gfrotran|stdcxx}-ng dependencies. I guess we can easily workaround things here for now. Do you know where should we report this issue?

ax3l · 2019-06-29T21:05:47Z

Sounds good to me. Looks like the same error we see downstream is now triggered here as well.

recipe/build-mpi.sh

dalcinl · 2019-07-02T10:20:30Z

@minrk I think the linux failures are a problem in conda-build=3.18.6.

Look at the Azure build log for linux-64 , you will see this line:

INFO (mpich,lib/libmpi.so.12.1.1): Needed DSO x86_64-conda_cos6-linux-gnu/sysroot/lib/libgfortran.so.4 found in CDT/compiler package defaults::libgfortran-ng-7.3.0-hdf63c60_0

So conda-build should add a runtime dependency to mpich on libgfortran-ng automatically, right?

But then, at the test stage, when the package is installed in the test env, we get:

The following NEW packages will be INSTALLED:

    libgcc-ng:    9.1.0-hdf63c60_0    defaults   
    libstdcxx-ng: 9.1.0-hdf63c60_0    defaults   
    mpi:          1.0-mpich           conda-forge
    mpich:        3.2.1-hc99cbb1_1013 local

so this means that libgfortran-ng is not listed as a runtime dependency in the mpich package.

I can reproduce a similar issue locally (using plain conda-build in my terminal, no docker). However, the missing dependency is libstdcxx-ng.

minrk · 2019-07-02T11:16:43Z

@dalcinl you're right, that's conda/conda-build#3583 it's being worked on right now. We should hold off any aarch64 builds until that gets fixed.

recipe/run_test.sh

recipe/build-mpi.sh

* Do not remove --as-needed from LDFLAGS * Add mpiexec.sh wrapper to workaround O_NONBLOCK issues * Add test for mpiexec with shell script * Add test for mpifort compiler wrapper

minrk · 2019-07-04T09:19:43Z

recipe/mpiexec.sh

+#!/bin/bash
+set -euo pipefail
+# pipe stdout, stderr through cat to avoid O_NONBLOCK issues
+exec mpiexec "$@" 2>&1</dev/null | cat


In the future, I think we can remove this. It's supposedly been fixed in mpich 3.2.1, but no harm keeping it for now.

Yes. After this PR is merged, we have to update to mpich 3.3.1, I was planning to do it by then.

Or do you prefer to jump now to 3.3.1 and forget about getting fresh builds for 3.2.1?

In the openmpi recipe, we still need the script to pass the --allow-run-as-root. Maybe it is a good idea to keep the mpiexex script in mpich as well, as a future last resort in case we are in need of similar tweaks.

Sure. No real difference between the script and the function that was here before. Still some arguments to consolidate. Without redirecting, an alias in run_test.sh ought to be enough.

However, the use the function that was here before was broken, better to use a script in which we can control bash options. See this log https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=45392 and look for ./helloworld_c: error while loading shared libraries: libgfortran.so.4: cannot open shared object file: No such file or directory. So eventually the PR was merged but the package was broken.

minrk · 2019-07-04T09:24:02Z

I'm happy with this, but I'd like to wait to merge until the conda-build issues get squared away, since this shouldn't be affecting the outputs I'm not pressured to get it done quickly. It looks like the older conda-build after the latest builds were marked as broken also has an issue with aarch64, so we probably just need to wait for a fixed conda-build 3.18

ax3l · 2019-07-10T14:18:10Z

@conda-forge-admin, please rerender

(so we get the fixed conda-build 3.18.7 with conda/conda-build#3596 )

…da-forge-pinning 2019.07.04

conda-forge-linter · 2019-07-10T14:19:55Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to re-render for you, but it looks like there was nothing to do.

conda-forge-linter · 2019-07-10T14:20:26Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to re-render for you, but it looks like there was nothing to do.

ax3l · 2019-07-10T14:20:38Z

sorry, still too old in the bot

minrk · 2019-07-11T14:22:11Z

Restarted aarch since the fixed image was pushed today (independent of rerender, I think)

minrk · 2019-07-12T07:50:49Z

All working again

dalcinl requested a review from ocefpaf as a code owner June 28, 2019 23:10

dalcinl force-pushed the update branch 2 times, most recently from d944d3c to 35bd244 Compare June 29, 2019 00:48

dalcinl requested a review from ax3l June 29, 2019 12:00

dalcinl force-pushed the update branch from 35bd244 to e761c56 Compare June 29, 2019 15:45

dalcinl changed the title ~~Update testing~~ Update recipe (build and testing) Jun 29, 2019

dalcinl force-pushed the update branch from e761c56 to a140054 Compare June 29, 2019 16:27

dalcinl force-pushed the update branch from a140054 to 1f6886f Compare June 30, 2019 09:45

minrk reviewed Jul 2, 2019

View reviewed changes

recipe/build-mpi.sh Show resolved Hide resolved

minrk reviewed Jul 2, 2019

View reviewed changes

recipe/run_test.sh Show resolved Hide resolved

minrk reviewed Jul 2, 2019

View reviewed changes

recipe/build-mpi.sh Outdated Show resolved Hide resolved

dalcinl force-pushed the update branch from 1f6886f to fe35438 Compare July 2, 2019 16:31

Update recipe (build and testing)

495c068

* Do not remove --as-needed from LDFLAGS * Add mpiexec.sh wrapper to workaround O_NONBLOCK issues * Add test for mpiexec with shell script * Add test for mpifort compiler wrapper

dalcinl force-pushed the update branch from fe35438 to 495c068 Compare July 4, 2019 09:11

minrk reviewed Jul 4, 2019

View reviewed changes

minrk approved these changes Jul 4, 2019

View reviewed changes

MNT: Re-rendered with conda-build 3.17.8, conda-smithy 3.4.0, and con…

374631d

…da-forge-pinning 2019.07.04

minrk merged commit 6ecec2d into conda-forge:master Jul 12, 2019

dalcinl deleted the update branch July 12, 2019 10:39

ax3l mentioned this pull request Aug 20, 2019

Arch: Add aarch64 and ppc64le, Version 3.0.2 conda-forge/mpi4py-feedstock#22

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update recipe (build and testing) #40

Update recipe (build and testing) #40

dalcinl commented Jun 28, 2019

conda-forge-linter commented Jun 28, 2019

dalcinl commented Jun 29, 2019

ax3l commented Jun 29, 2019 •

edited

Loading

dalcinl commented Jul 2, 2019

minrk commented Jul 2, 2019

minrk Jul 4, 2019

dalcinl Jul 4, 2019

dalcinl Jul 4, 2019

minrk Jul 4, 2019

dalcinl Jul 4, 2019

minrk commented Jul 4, 2019

ax3l commented Jul 10, 2019 •

edited

Loading

conda-forge-linter commented Jul 10, 2019

conda-forge-linter commented Jul 10, 2019

ax3l commented Jul 10, 2019

minrk commented Jul 11, 2019

minrk commented Jul 12, 2019

Update recipe (build and testing) #40

Update recipe (build and testing) #40

Conversation

dalcinl commented Jun 28, 2019

conda-forge-linter commented Jun 28, 2019

dalcinl commented Jun 29, 2019

ax3l commented Jun 29, 2019 • edited Loading

dalcinl commented Jul 2, 2019

minrk commented Jul 2, 2019

minrk Jul 4, 2019

Choose a reason for hiding this comment

dalcinl Jul 4, 2019

Choose a reason for hiding this comment

dalcinl Jul 4, 2019

Choose a reason for hiding this comment

minrk Jul 4, 2019

Choose a reason for hiding this comment

dalcinl Jul 4, 2019

Choose a reason for hiding this comment

minrk commented Jul 4, 2019

ax3l commented Jul 10, 2019 • edited Loading

conda-forge-linter commented Jul 10, 2019

conda-forge-linter commented Jul 10, 2019

ax3l commented Jul 10, 2019

minrk commented Jul 11, 2019

minrk commented Jul 12, 2019

ax3l commented Jun 29, 2019 •

edited

Loading

ax3l commented Jul 10, 2019 •

edited

Loading