[Feature Request] Provide a way to build on Windows using Bazel

[terryzhao127]

Now that there is a pre-built binary of PhoenixGo on Windows, it is very kind of you to provide a way to build PhoenixGo on Windows using Bazel.

• It will be helpful for us to learn by modifying your codes.
• Now Bazel supports Windows, and the building process using Bazel is very convient compared to using Visual Studio.

[wonderingabout]

hi,

i didnt know that bazel was available for windows too, but actually : https://docs.bazel.build/versions/master/windows.html

i'm very happy with the bazel program : it is very powerful and simple to use, and after the compile it works easily
so i would also be interested in a bazel compile for windows if it is ever possible (that may also fix the avx avx2 fma avx512 issue here #74 )

however that probably requires some work to implement, but that would be a premiere in the go AI world as far as i know

@wodesuck @fiskerhuang @funionguo know better (i am only a contributor)

[terryzhao127]

Is it theoretically feasible to build PhoenixGo using current Bazel workspace on Windows? Or they should be modified to support Windows?

[wonderingabout]

@guikarist i dont know :)
we need to ask the names i mentionned in first message

but if possible, that would certainly be a great acocmplishment

[wodesuck]

I have try, but: 1. failed to use http_archive with patch parameter, 2. failed to build glog with bazel.
Need some modify for building with bazel on windows.

[terryzhao127]

@wodesuck Thanks for response. I have encountered the first problem, too. On Windows, MSYS2 is used for Bazel, so the code to run patch binary should be stripped of /usr/bin/ (the $PATH environment variable should be correctly set). I am planning to learn more about Bazel and make some modifications to Bazel files in PhoenixGo. However now I am not completely aware of what the building process of PhoenixGo really is.

[wonderingabout]

@wodesuck

yes, most likely some building settings need to be modified

but it's totally worth doing it

not only that would solve many issues, and most of all that would be a great achievement (using bazel is so easy on powerful, even a total beginner can do it, while visual studio is such a pain to use)

[wonderingabout]

@guikarist

a translation of @wodesuck comment in english using google translate (surprisingly understandable lol) :
#57 (comment)

There is trouble in compiling under Windows. .
First, you need to compile a tensorflow, see here .
Then compile glog, gflags and boost.
Then in the "Properties Manager" in Visual Studio, find the ThirdPary property, double-click to open it, find "Common Properties - User Macro", fill in the path of the above dependency.
Finally, the solution can be started.

BTW, the above operations need to be completed under Visual Studio 2015. After testing, Visual Studio 2017 can't compile tensorflow.

it may be help you figure things out

[terryzhao127]

@wonderingabout

Thanks. My colleague has done this before. But it is obviously not simple and can only support Tensorflow which is built by CMake. And Tensorflow doesn’t support CMake since 1.11.

Now I have succeeded in building libtensorflow_cc.so on Windows using Bazel which can be used after being renamed to libtensorflow_cc.dll. But other third-party lib files have .a extensions instead of .lib. There must be some problems in building Tensorflow on Windows using Bazel, too. However, I believe one day they would be solved!

[wonderingabout]

i hope too @guikarist

can you compile using visual studio 2015 rather then ?

it would be very great to have a version with :

• cuda 10.0 (already works on ubuntu, i tested it)
• cudnn 7.4.2 (same)
• supports avx avx2 fma (to cover most modern home cpu)

[terryzhao127]

@wonderingabout
It’s a pity that I have to use Visual Studio for the moment.

[wonderingabout]

@guikarist pity or not, i would appreciate if you provide a new release

the old one is so outdated and slow 😭

[terryzhao127]

@wonderingabout No problem, my colleague will do this and I will send you one as soon as he finishes.

[terryzhao127]

@wodesuck

Now Tensorflow C++ API on Windows can be built but the output files have .so or .a extensions.

The libtensorflow_cc.so file can be renamed to libtensorflow_cc.dll, and used correctly to some extent. But I have no idea how to use the built third-party files which have .a files. Do you think whether it is possible to work out a way to build PhoenixGo using Bazel with current version of Tensorflow?

[wonderingabout]

@guikarist

great !!
i cant wait really

this is the biggest highlights needed for a new windows release :

it would be very great to have a version with :

• cuda 10.0 (already works on ubuntu, i tested it)
• cudnn 7.4.2 (same)
• support for avx avx2 fma (to cover most modern home cpu)**

cuda 10.0 would support turing GPU (RTX cards), and will improve performance on Volta cards (tesla V100, titan V, etc...,), as well as improve performance on all cards

cudnn 7.4.x is significantly faster than the old cudnn 7.1.x

avx avx2 fma cpu instructions are still lacking atm, which makes the computation a lot slower

@guikarist
when your release is finished i'll test it, and if it works well i'll add it to the releases on my PR (consider sending a pull request to my branch so that you get credit for it)

https://github.com/wonderingabout/PhoenixGo/tree/faqv2-bazel-master

[wonderingabout]

@guikarist

on ubuntu bazel always downloads tensorflow 1.8

i assume PhoenixGo engine is not yet compatible with tensorflow 1.9+ (but try, maybe you can, however i would try with supported version (1.8) first :) )
@wodesuck knows better than me

[terryzhao127]

@wonderingabout @wodesuck
This issue in Tensorflow may be important for your work. And this is the corresponding PR.

@wonderingabout
My colleague is working on building a release of PhoenixGo at the moment.

[wonderingabout]

@guikarist i'm sure this will be greatly helpful for @wodesuck who manages the code, i read the main general lines, big thanks for your contribution on tensorflow for windows ! 👍

building on bazel is the ultimate ideal distant goal

the main priority atm is to get a new release for windows no matter how it is built (visual studio or bazel)

if i can help in any way i'd be glad
big thanks for your effort !! keep me updated how it goes 👍 @guikarist

[terryzhao127]

@wonderingabout My pleasure! We had been trying buiding with Bazel. What we did with CMake is something one year ago. So it may need time for my colleague to do this.

[wonderingabout]

no problem, take your time @guikarist

hope it goes well for you 👍

[terryzhao127]

@wonderingabout We have just encountered a problem on building Tensorflow with CUDA 10 using CMake. I have read serveral issues and tutorials and find that CUDA 10 is very likely not to work on building Tensorflow using CMake on Windows. Now only usable Tensorflow built with CUDA 10 on Windows is a pip package built by Bazel, which cannot be used for building PhoenixGo.

[wonderingabout]

oh i understand, thanks for the feedback 👍 @guikarist

then i guess its natural to go with cuda 9.0 instead, with cudnn 7.4.2, and the support of all modern cpu instructions in home computers (avx, avx2, fma), thta the existing build doesnt support

can you gather the data about the various issues you encountered and their fixes, it may be helpful for other people in the future

thanks again !

i'd like if you build with cuda 9.0 and the other settings i mentionned, thanks again for your effort ! 💯

there is really a need to upgrade the existing build which is so slow (40% of the speed of the ubuntu version with same settings, purpose is 80-90% of the speed)

[wonderingabout]

@guikarist

any update or news on the basic compilation for windows ?
so that most windows users can enjoy the basic avx avx2 fma cpu instructions 👍 (and cuddn 7.4.2 + cuda 9.0)

it would be really a very big help

[terryzhao127]

Sorry to have kept you waiting. My colleague hasn't worked out, and I will have a try soon.

[wonderingabout]

@guikarist

no problem really, i'm glad you're here again today 👍

forget cuda 10.0 if its too hard, let's try a more simple thing first

so to sum up we now want a more simplistic build that just supports :

• avx , avx2, fma cpu instructions same as my r7 1700 for example (current build doesnt !!!! very slow !!!)
• cuda 9.0
• cudnn 7.4.2 for cuda 9.0 (works on ubuntu, so should on windows too i hope)

so 2 builds :

• 1 : GPU version same as above
• 2 : CPU only version with just modern cpu instructions avx avx2 fma

most users are on windows, so it may seem minimal but i'm sure it will be a big big help !!! 💯
(current windows build has 40% of the speed of ubuntu build, we're aiming for 80-85% with this build now)

[wonderingabout]

maybe this can be given another try some future time, but thanks for the effort anyways 👍