Add AppVeyor testing for Windows #347

ocefpaf · 2017-01-10T21:02:35Z

To help testing on Windows I put together this appveyor.yml file that builds netcfd-c with conda and cmake. The reason I chose conda is to easily get all the dependencies (curl, hdf4, hdf5, and zlib). However, in order to get this PR working, we need to figure out a way to use AppVeyor's cygwin macro language (m4) to provide the auto-generated source files. (Or those files could be committed into the repo?)

PS: The repo adims must activate AppVeyor. Here is the first run on my account: https://ci.appveyor.com/project/ocefpaf/netcdf-c/build/1.0.8

ocefpaf · 2017-01-11T23:54:17Z

It seems that the msys2 present in AppVeyor was all we needed to get the build to complete.

https://ci.appveyor.com/project/ocefpaf/netcdf-c/build/1.0.13

This can be improved to run some tests too, right now it only builds the master branch using 3 versions of Visual Studio (vc 9, 10, and 14).

cc @dopplershift who may be interested in this PR.

dopplershift · 2017-01-31T22:20:16Z

I added netcdf-c to Unidata's AppVeyor, and have given Unidata's netCDF team what should be proper permissions.

Power-cycling the PR to hopefully get AppVeyor to run.

WardF · 2017-01-31T22:29:27Z

Thanks @dopplershift for jumping in on this!

dopplershift · 2017-01-31T22:41:26Z

Well, the infrastructure works--but the build failed.

WardF · 2017-01-31T22:43:24Z

I'll dig into it but at a glance it looks like it is using visual studio 9, which is not a version we have tested against. I know that a lot of effort goes into supporting versions 11 and 12, as there are differences from 14 (the latest, I think). I'll look and figure out the specifics.

dopplershift · 2017-01-31T22:44:26Z

What's strange it that it worked for @ocefpaf's.

WardF · 2017-01-31T22:47:19Z

It may be environmental; I see the following:

ncgen.l(123) : error C2065: '_FILLVALUE' : undeclared identifier
ncgen.l(123) : error C2099: initializer is not a constant
ncgen.l(124) : error C2065: '_FORMAT' : undeclared identifier
ncgen.l(124) : error C2099: initializer is not a constant
ncgen.l(125) : error C2065: '_STORAGE' : undeclared identifier
ncgen.l(125) : error C2099: initializer is not a constant
ncgen.l(126) : error C2065: '_CHUNKSIZES' : undeclared identifier

This suggests a header isn't being seen for some reason. It could be environmental, or maybe something in cmake? I'm not certain, yet, I'll have to look. It could also be due to missing tools, potentially. I'm not sure what this config ~~requires~~ installs, but both m4 and yacc/bison/flex are required to generate some of the files. If they aren't being installed, maybe that could cause this?

DennisHeimbigner · 2017-01-31T23:01:56Z

Remember that ncgen.l and ncgen.y should not be scanned because
they are used to generate ncgenl.[ch] and ncgeny.[ch].
Make sure that it is the latter files that are being scanned and not ncgen.l and ncgen.y

ocefpaf · 2017-02-01T17:31:40Z

What's strange it that it worked for @ocefpaf's.

The test passed up until ocefpaf@fd2911a

See https://ci.appveyor.com/project/ocefpaf/netcdf-c/history

PS: Note that this PR implementes only a "can we build this" kind of test. Additional real tests should be added later.

ocefpaf · 2017-02-16T17:33:07Z

conda.recipe/meta.yaml

+test:
+  commands:
+    - ncdump -h http://geoport-dev.whoi.edu/thredds/dodsC/estofs/atlantic
+    # FIXME: the next two are failing at the moment.


xref: conda-forge/libnetcdf-feedstock#14

Thes URLS are either not reliable (geoport-dev.whoi.edu) or cycle off (the GFS URL with a specific date).
This could be a good one as:

It's at Unidata

It doesn't reference a model resolution that may disappear someday (e.g. 0.5=>0.25) and it doesn't specify a particular date

ncdump -h http://thredds.ucar.edu/thredds/dodsC/grib/NCEP/WW3/Global/Best

IMHO, it's not useful to do more than one of these.

Indeed we need reliable endpoints for a robust test. Here I only gather the URLs I knew were failing. Before we merge this we can try to find, or create, some stable ones.

rsignell-usgs · 2017-02-17T19:43:30Z

conda.recipe/meta.yaml

+test:
+  commands:
+    - ncdump -h http://geoport-dev.whoi.edu/thredds/dodsC/estofs/atlantic
+    # FIXME: the next two are failing at the moment.


Thes URLS are either not reliable (geoport-dev.whoi.edu) or cycle off (the GFS URL with a specific date).
This could be a good one as:

It's at Unidata

It doesn't reference a model resolution that may disappear someday (e.g. 0.5=>0.25) and it doesn't specify a particular date

ncdump -h http://thredds.ucar.edu/thredds/dodsC/grib/NCEP/WW3/Global/Best

IMHO, it's not useful to do more than one of these.

dopplershift · 2017-02-17T19:46:57Z

@rsignell-usgs But is it currently broken on windows with 4.4.1.1?

rsignell-usgs · 2017-02-17T20:02:13Z

Okay, actually, the latest libnetcdf and netcdf4 on my win64 are WORKING with both a local 5GB file and with this URL:

url = 'http://thredds.ucar.edu/thredds/dodsC/grib/NCEP/GFS/Global_0p5deg/GFS
_Global_0p5deg_20170217_1200.grib2'

Here's my config:

(netcdf4b) C:\Users\rsignell\Downloads>conda list -e
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: win-64
certifi=2017.1.23=py36_0
curl=7.52.1=vc14_0
hdf4=4.2.12=vc14_0
hdf5=1.8.17=vc14_9
jpeg=9b=vc14_0
libnetcdf=4.4.1.1=vc14_2
mkl=2017.0.1=0
netcdf4=1.2.7=np111py36_0
numpy=1.11.3=py36_0
pip=9.0.1=py36_0
python=3.6.0=2
setuptools=33.1.0=py36_0
vc=14=0
vs2015_runtime=14.0.25420=0
wheel=0.29.0=py36_0
wincertstore=0.2=py36_0
zlib=1.2.11=vc14_0

dopplershift · 2017-02-17T20:04:28Z

I don't think you finished that thought...

DennisHeimbigner · 2017-02-17T20:32:52Z

Where is that geoport reference coming from? The only occurrence
in netcdf-c is in .ncdap_test/test_nstride_cached.c and that one is
commented out.

rsignell-usgs · 2017-02-17T21:24:36Z

@DennisHeimbigner that was just a test for the conda recipe.

ocefpaf · 2017-05-22T17:27:49Z

Latest source is no longer build on Windows for Python 2.7 (vc9) and Python 3.4 (vc10). This is currently a blocker to build netcdf4-python on anything but Python 3.5 and 3.6.

See https://ci.appveyor.com/project/ocefpaf/netcdf-c/build/1.0.23

ocefpaf · 2017-06-12T21:24:59Z

Latest source broke all vcs available in AppVeyor.

https://ci.appveyor.com/project/ocefpaf/netcdf-c/build/1.0.25

CLAassistant · 2017-10-21T15:34:22Z

All committers have signed the CLA.

ocefpaf · 2018-04-17T11:35:29Z

This PR is stale and I do not have the bandwidth to keep updating it.

I guess that Windows will always be a second class citizen in most projects :-(

…more information.

ocefpaf added 3 commits January 10, 2017 18:07

Add AppVeyor testing for Windows

2c122b0

Merge branch 'master' into appveyor

9956438

Use AppVeyor's cmake

a493670

ocefpaf force-pushed the appveyor branch from acf74c4 to 0eb5907 Compare January 11, 2017 23:42

test appveyor msys2

0eb5907

Merge branch 'master' into appveyor

07dda2d

This was referenced Jan 23, 2017

Release 2.2.21 missing files Unidata/UDUNITS-2#46

Closed

Add AppVeyor CI Unidata/UDUNITS-2#47

Merged

dopplershift closed this Jan 31, 2017

dopplershift reopened this Jan 31, 2017

Merge branch 'master' into appveyor

fd2911a

Merge branch 'master' into appveyor

3c737c9

ocefpaf commented Feb 16, 2017

View reviewed changes

try to run more tests

54cc1d6

rsignell-usgs suggested changes Feb 17, 2017

View reviewed changes

Merge branch 'master' into appveyor

30d6445

ocefpaf added 4 commits March 2, 2017 13:08

Merge branch 'master' into appveyor

8f48184

Merge branch 'master' into appveyor

bcf372e

Merge branch 'master' into appveyor

b1bcde5

removed unstable URLs and added two more

eb2a70f

ocefpaf mentioned this pull request May 22, 2017

Can't load_remote_dataset in suite.py for OPeNDAP URL ioos/compliance-checker#500

Closed

ocefpaf added 2 commits June 2, 2017 14:42

Merge branch 'master' into appveyor

b107261

Merge branch 'master' into appveyor

2868e38

ocefpaf added 3 commits June 12, 2017 18:26

test with hdf5 1.8.18

0c30367

simplify appveyor.yml

a67fd38

remove vc 10 builds (Python 3.4)

41e2428

ocefpaf force-pushed the appveyor branch from 14763c1 to 41e2428 Compare June 12, 2017 21:35

test agaist hdf5 1.8.18 with threads safe enabled

b6064a2

WardF added the platform/windows label Jul 6, 2017

WardF added this to the 4.5.1 milestone Jul 6, 2017

WardF added the area/tests label Jul 6, 2017

WardF self-assigned this Jul 6, 2017

ocefpaf added 2 commits August 8, 2017 11:39

Merge branch 'master' into appveyor

896917a

Merge branch 'master' into appveyor

2d1f832

ocefpaf mentioned this pull request Oct 21, 2017

V4.5.0 conda-forge/libnetcdf-feedstock#26

Merged

WardF modified the milestones: 4.6.0, 4.6.1 Jan 25, 2018

WardF modified the milestones: 4.6.1, 4.7.0 Mar 20, 2018

ocefpaf closed this Apr 17, 2018

ocefpaf deleted the appveyor branch April 17, 2018 11:35

WardF added a commit that referenced this pull request Apr 24, 2018

Attempting appveyor support. Based on work by @ocefpaf, see #347 for …

f216710

…more information.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add AppVeyor testing for Windows #347

Add AppVeyor testing for Windows #347

ocefpaf commented Jan 10, 2017 •

edited

Loading

ocefpaf commented Jan 11, 2017 •

edited

Loading

dopplershift commented Jan 31, 2017

WardF commented Jan 31, 2017

dopplershift commented Jan 31, 2017

WardF commented Jan 31, 2017

dopplershift commented Jan 31, 2017

WardF commented Jan 31, 2017 •

edited

Loading

DennisHeimbigner commented Jan 31, 2017

ocefpaf commented Feb 1, 2017

ocefpaf Feb 16, 2017

rsignell-usgs Feb 17, 2017 •

edited by dopplershift

Loading

ocefpaf Feb 17, 2017

rsignell-usgs Feb 17, 2017 •

edited by dopplershift

Loading

dopplershift commented Feb 17, 2017

rsignell-usgs commented Feb 17, 2017 •

edited

Loading

dopplershift commented Feb 17, 2017

DennisHeimbigner commented Feb 17, 2017

rsignell-usgs commented Feb 17, 2017

ocefpaf commented May 22, 2017

ocefpaf commented Jun 12, 2017

CLAassistant commented Oct 21, 2017 •

edited

Loading

ocefpaf commented Apr 17, 2018

Add AppVeyor testing for Windows #347

Add AppVeyor testing for Windows #347

Conversation

ocefpaf commented Jan 10, 2017 • edited Loading

ocefpaf commented Jan 11, 2017 • edited Loading

dopplershift commented Jan 31, 2017

WardF commented Jan 31, 2017

dopplershift commented Jan 31, 2017

WardF commented Jan 31, 2017

dopplershift commented Jan 31, 2017

WardF commented Jan 31, 2017 • edited Loading

DennisHeimbigner commented Jan 31, 2017

ocefpaf commented Feb 1, 2017

ocefpaf Feb 16, 2017

Choose a reason for hiding this comment

rsignell-usgs Feb 17, 2017 • edited by dopplershift Loading

Choose a reason for hiding this comment

ocefpaf Feb 17, 2017

Choose a reason for hiding this comment

rsignell-usgs Feb 17, 2017 • edited by dopplershift Loading

Choose a reason for hiding this comment

dopplershift commented Feb 17, 2017

rsignell-usgs commented Feb 17, 2017 • edited Loading

dopplershift commented Feb 17, 2017

DennisHeimbigner commented Feb 17, 2017

rsignell-usgs commented Feb 17, 2017

ocefpaf commented May 22, 2017

ocefpaf commented Jun 12, 2017

CLAassistant commented Oct 21, 2017 • edited Loading

ocefpaf commented Apr 17, 2018

ocefpaf commented Jan 10, 2017 •

edited

Loading

ocefpaf commented Jan 11, 2017 •

edited

Loading

WardF commented Jan 31, 2017 •

edited

Loading

rsignell-usgs Feb 17, 2017 •

edited by dopplershift

Loading

rsignell-usgs Feb 17, 2017 •

edited by dopplershift

Loading

rsignell-usgs commented Feb 17, 2017 •

edited

Loading

CLAassistant commented Oct 21, 2017 •

edited

Loading