Windows Support and AppVeyor CI #173

ax3l · 2018-05-15T00:21:17Z

Adds support for MSVC builds on Windows.

Due to various constrains in dependencies, this is

Python 3 only
HDF5 only
Serial only (this is fine)

ADIOS1 on MSVC is very unlikely but ADIOS2 in the future could be a possibility, also for conda-forge.

Related to #63

To Do

clean up verbose lines from experimenting on Windows
finalize PowerShell download script
README: Add Badge
tests: does not link with Catch2 enabled
tests: python bindings build, but do they run?
install? looks ok-ish, might need to add a link to the main openPMD.dll later on as well
enable AppVeyor in main repo as well

C0nsultant · 2018-05-15T07:44:52Z

test/SerialIOTest.cpp

@@ -815,7 +815,11 @@ TEST_CASE( "hdf5_dtype_test", "[serial][hdf5]" )
        s.setAttribute("vecUint64", std::vector< uint64_t >({18446744073709551614u, 18446744073709551615u}));
        s.setAttribute("vecFloat", std::vector< float >({0.f, 3.40282e+38f}));
        s.setAttribute("vecDouble", std::vector< double >({0., 1.79769e+308}));
+#ifdef _MSC_VER
+        s.setAttribute("vecLongdouble", std::vector< long double >({0.L, 1.49769e+308L}));


All of these calls could be replaced with std::numeric_limits::max. This would make it compatible across all compilers (that conform to the C++ standard).

Yep, just a quick hack so far :)

ax3l · 2018-05-15T12:19:07Z

@C0nsultant I think the tests are not compiling because of the redefinition of private/protected to public which causes binary incompatibilities in MSVC: catchorg/Catch2#258 (comment)

Can you maybe control these "very internal unit tests" via a define so I can skip them in MSVC?

C0nsultant · 2018-05-15T14:23:50Z

Can you maybe control these "very internal unit tests" via a define so I can skip them in MSVC?

Just so I get this right - you have something like -DopenPMD_NONINVASIVE_TESTS=ON in mind, which then skip all tests that rely on #define public private?
I'd rather have some modifier like this that allows us to still test the data.

ax3l · 2018-05-15T14:29:13Z

No, really the first one.

The second solution means we would ship different access modifiers on Windows which is a hack. Instead, let us just test observable behavior on Windows - that's ok.

C0nsultant · 2018-05-15T14:34:23Z

let us just test observable behavior on Windows

Sighh...
The platform is not out main target audience anyway, so that should be fine. Core tests then can be designated for unix and assumed to represent all other platforms. Not elegant, but it does the job.

ax3l · 2018-05-18T08:15:12Z

CMakeLists.txt

+if(WIN32)
+    # generate a the openpmd_export.h header with export defines we can use for
+    # static members and other global data
+    include(GenerateExportHeader)


Comment out if we don't need this currently.

ax3l · 2018-05-18T08:15:23Z

CMakeLists.txt

+if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "MSVC")
+    target_compile_options(openPMD PUBLIC "/bigobj")
+    if(BUILD_SHARED_LIBS)
+        target_compile_options(openPMD PUBLIC "/MD")


check if we still need this block

ax3l · 2018-05-18T13:01:50Z

.appveyor.yml

+  # Compiler & Generator Selection
+  - cmd: if "%APPVEYOR_BUILD_WORKER_IMAGE%"=="Visual Studio 2015" set OPENPMD_CMAKE_GENERATOR=Visual Studio 14 2015
+  - cmd: if "%TARGET_ARCH%"=="x64" set OPENPMD_CMAKE_GENERATOR=%OPENPMD_CMAKE_GENERATOR% Win64
+#  - cmd: if "%TARGET_ARCH%"=="x86" "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64_x86


note: I keep this for docs, this line is needed if we want to build everything from a amd64 platform

C0nsultant · 2018-05-18T13:32:58Z

test/SerialIOTest.cpp

@@ -861,7 +864,9 @@ TEST_CASE( "hdf5_dtype_test", "[serial][hdf5]" )
    REQUIRE(s.getAttribute("vecUint64").get< std::vector< uint64_t > >() == std::vector< uint64_t >({18446744073709551614u, 18446744073709551615u}));
    REQUIRE(s.getAttribute("vecFloat").get< std::vector< float > >() == std::vector< float >({0.f, 3.40282e+38f}));
    REQUIRE(s.getAttribute("vecDouble").get< std::vector< double > >() == std::vector< double >({0., 1.79769e+308}));
-    REQUIRE(s.getAttribute("vecLongdouble").get< std::vector< long double > >() == std::vector< long double >({0.L, 1.18973e+4932L}));
+#if !defined(_MSC_VER)


#179 is really this bad? Sad to see as long double is part of the C++ standard.

only during read :)
I think for some reason the variant takes the double alternative and then tries to read as long double or similar

C0nsultant · 2018-05-18T13:35:10Z

test/SerialIOTest.cpp

@@ -1171,7 +1176,9 @@ TEST_CASE( "adios1_dtype_test", "[serial][adios1]" )
    REQUIRE(s.getAttribute("uint64").get< uint64_t >() == 64u);
    REQUIRE(s.getAttribute("float").get< float >() == 16.e10f);
    REQUIRE(s.getAttribute("double").get< double >() == 1.e64);
+#if !defined(_MSC_VER)
    REQUIRE(s.getAttribute("longdouble").get< long double >() == 1.e80L);


Why not use std::numeric_limits<long double>::max() here as well? Is it the initializer list causing the problems?

The problem I caused by the long double template argument during read #179
I will exclude it for now and open a PR that removes the exclusion to demonstrate.

On MSVC, the range of `long double` is just the same as `long`. This is fine, since it just needs to be "at least" double... https://msdn.microsoft.com/en-us/library/s3f49ktz.aspx https://en.wikipedia.org/wiki/Long_double

Paths, symbols and tests.

Windows version of downloading all example files for tests.

Add support for AppVeyor builds of serial openPMD-api with HDF5. Dependencies are only available with C++11 support with VS2015 ("[vc14]") toolchain. This is limited to Python 3 since Python 2.7 is only available with VS2008.

Do not print value of attribute that is read-only tried to be set in exception. Work-around did only work on VS 2017 but not on VS 2015. Just throw a generic warning on MSVC instead.

ax3l added install machine/system machine & HPC system specific issues labels May 15, 2018

ax3l force-pushed the topic-appveyor branch 2 times, most recently from 3e3dc94 to 6b6ebd3 Compare May 15, 2018 07:00

C0nsultant reviewed May 15, 2018

View reviewed changes

ax3l force-pushed the topic-appveyor branch 5 times, most recently from 4dc5229 to c5de2f4 Compare May 15, 2018 11:57

ax3l force-pushed the topic-appveyor branch from c5de2f4 to 0940fce Compare May 15, 2018 12:23

This was referenced May 15, 2018

Allow test-build that does not modify C++ keywords #175

Closed

Introduce switch for invasive tests #176

Merged

ax3l force-pushed the topic-appveyor branch 11 times, most recently from e899fba to abe0dfc Compare May 17, 2018 11:15

ax3l force-pushed the topic-appveyor branch 8 times, most recently from 19dc563 to b075673 Compare May 18, 2018 08:13

ax3l commented May 18, 2018

View reviewed changes

ax3l force-pushed the topic-appveyor branch 2 times, most recently from 05d5bdb to e8fdb59 Compare May 18, 2018 13:01

ax3l commented May 18, 2018

View reviewed changes

ax3l force-pushed the topic-appveyor branch from e8fdb59 to d902de1 Compare May 18, 2018 13:03

ax3l changed the title ~~[WIP] Windows Support and AppVeyor CI~~ Windows Support and AppVeyor CI May 18, 2018

ax3l assigned C0nsultant May 18, 2018

C0nsultant reviewed May 18, 2018

View reviewed changes

C0nsultant approved these changes May 18, 2018

View reviewed changes

ax3l force-pushed the topic-appveyor branch 2 times, most recently from e685f89 to 1628549 Compare May 18, 2018 16:12

ax3l added 5 commits May 18, 2018 19:36

MSVC: long double == long

c1766f4

On MSVC, the range of `long double` is just the same as `long`. This is fine, since it just needs to be "at least" double... https://msdn.microsoft.com/en-us/library/s3f49ktz.aspx https://en.wikipedia.org/wiki/Long_double

CMake: Windows Support

cced9ac

Paths, symbols and tests.

Download Samples: Powershell Script

ac71be8

Windows version of downloading all example files for tests.

AppVeyor: Windows CI

c7e0b3f

Add support for AppVeyor builds of serial openPMD-api with HDF5. Dependencies are only available with C++11 support with VS2015 ("[vc14]") toolchain. This is limited to Python 3 since Python 2.7 is only available with VS2008.

README: Badge AppVeyor

760a39a

ax3l force-pushed the topic-appveyor branch from 1628549 to 760a39a Compare May 18, 2018 17:36

MSVC 2015: Disable Attribute Val

c55f96f

Do not print value of attribute that is read-only tried to be set in exception. Work-around did only work on VS 2017 but not on VS 2015. Just throw a generic warning on MSVC instead.

ax3l merged commit 2a9a1da into openPMD:dev May 19, 2018

ax3l deleted the topic-appveyor branch May 19, 2018 08:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Windows Support and AppVeyor CI #173

Windows Support and AppVeyor CI #173

ax3l commented May 15, 2018 •

edited

Loading

C0nsultant May 15, 2018 •

edited

Loading

ax3l May 15, 2018

ax3l commented May 15, 2018 •

edited

Loading

C0nsultant commented May 15, 2018 •

edited

Loading

ax3l commented May 15, 2018

C0nsultant commented May 15, 2018

ax3l May 18, 2018

ax3l May 18, 2018

ax3l May 18, 2018

C0nsultant May 18, 2018 •

edited

Loading

ax3l May 18, 2018 •

edited

Loading

C0nsultant May 18, 2018

ax3l May 18, 2018 •

edited

Loading

Windows Support and AppVeyor CI #173

Windows Support and AppVeyor CI #173

Conversation

ax3l commented May 15, 2018 • edited Loading

To Do

C0nsultant May 15, 2018 • edited Loading

Choose a reason for hiding this comment

ax3l May 15, 2018

Choose a reason for hiding this comment

ax3l commented May 15, 2018 • edited Loading

C0nsultant commented May 15, 2018 • edited Loading

ax3l commented May 15, 2018

C0nsultant commented May 15, 2018

ax3l May 18, 2018

Choose a reason for hiding this comment

ax3l May 18, 2018

Choose a reason for hiding this comment

ax3l May 18, 2018

Choose a reason for hiding this comment

C0nsultant May 18, 2018 • edited Loading

Choose a reason for hiding this comment

ax3l May 18, 2018 • edited Loading

Choose a reason for hiding this comment

C0nsultant May 18, 2018

Choose a reason for hiding this comment

ax3l May 18, 2018 • edited Loading

Choose a reason for hiding this comment

ax3l commented May 15, 2018 •

edited

Loading

C0nsultant May 15, 2018 •

edited

Loading

ax3l commented May 15, 2018 •

edited

Loading

C0nsultant commented May 15, 2018 •

edited

Loading

C0nsultant May 18, 2018 •

edited

Loading

ax3l May 18, 2018 •

edited

Loading

ax3l May 18, 2018 •

edited

Loading