Hard or impossible to compile under Windows with Buck?

[gmalartre]

After trying to build plovr under Windows (following the multiple steps to install Buck under windows) and realizing its not possible to build plovr due to Buck using Android SDK using python select.select()( ( select.select does not work under windows “preexec_fn is not supported on Windows”), I came to the conclusion it was quite a waste of time to begin with. Eventually I realized I could get plovr 6.0.0 from the npm repository by creating a blank project running "npm init" then "npm install plovr".

Could it be mentioned in the README.md that the release page is not up to date and to use the npm build instead. Also mentionning the "npm init" command would help people to save an extra 1 minute.

Thanks!

[nicks]

hmmm...i've definitely been able to compile Plovr on windows w/ buck, but let me double-check tonight and get back to you

[nicks]

hmm...I was not able to reproduce any of the problems you are having. Can you give more details? In particular,

1. "due to Buck using Android SDK using python select". I don't know what this sentence means, or what the Android SDK has to do with building Plovr. I do not have the Android SDK on my machine. Can you describe exactly what command you're running that has the error?

2. The npm package is simply a way to help other npm packages download the release jar. There's not a good reason to use it if you don't use npm. You can just download the jars from the Releases page, as described in the README.

3. I don't understand why 'npm init' would be useful, but maybe this is due to your confusion about point (2)

[gmalartre]

1. Following the Getting Started (for Windows) tutorial for Buck, I've installed the Android SDK (it seems to be android-25, not sure how their SDK version system work), then I've installed android-ndk-r14, and Python 2.7.13 with the most recent JDK 8 (121). I've found a version of Watchman for windows. Then I was able to build Buck version b8357cdc583715406dcc1b3b2d28f7fac8a30e18 and make it run (hurray!). Trying to run buck in the plovr clone return the following:
   C:\_repository\plovr>buck build plovr Using watchman. Traceback (most recent call last): File "C:\_repository\buck\bin\..\programs\buck.py", line 48, in <module> propagate_failure(main(sys.argv)) File "C:\_repository\buck\bin\..\programs\buck.py", line 41, in main return buck_repo.launch_buck(build_id) File "C:\_repository\buck\programs\buck_tool.py", line 172, in launch_buck self.launch_buckd(buck_version_uid=buck_version_uid) File "C:\_repository\buck\programs\buck_tool.py", line 334, in launch_buckd env=self._environ_for_buck()) File "C:\Python27\lib\subprocess.py", line 343, in __init__ raise ValueError("preexec_fn is not supported on Windows " ValueError: preexec_fn is not supported on Windows platforms

Re-reading this exception, the error seems to be only related to Buck, so probably Android SDK as nothing to do with it.

2. I do not use npm and yesterday the release page wasn't up to date, that is why, before bothering anybody to just produce a jar I tried my best to find it already built somewhere. And yes creating a dummy project with npm init and npm install plovr gave me access to the most up to date version of plovr yesterday.

3. Since I don't use npm and wasn't sure how to download the plovr dependency I had to figure that out, like I said it would just save one minute of google search.

Thanks!

[gmalartre]

Ok I went a bit further and found this ticket on buck repo. I tried the environment variable NO_BUCKD=1 and was able to reach an assertionError:

C:\_repository\plovr>buck build plovr
Not using buckd because NO_BUCKD is set.
[2017-04-07 10:04:06.792][error][command:null][tid:01][com.facebook.buck.cli.Main] Uncaught exception at top level
java.lang.AssertionError: C:\android-ndk-r14b\toolchains\arm-linux-androideabi-4.9\prebuilt\windows-x86_64\bin\arm-linux-androideabi-gcc doesn't exist.
        at com.facebook.infer.annotation.Assertions.assertCondition(Assertions.java:72)
        at com.facebook.buck.android.NdkCxxPlatforms$NdkCxxToolchainPaths.processPathPattern(NdkCxxPlatforms.java:1277)
        at com.facebook.buck.android.NdkCxxPlatforms$NdkCxxToolchainPaths.getToolPath(NdkCxxPlatforms.java:1377)
        at com.facebook.buck.android.NdkCxxPlatforms.getToolPath(NdkCxxPlatforms.java:697)
        at com.facebook.buck.android.NdkCxxPlatforms.getCTool(NdkCxxPlatforms.java:732)
        at com.facebook.buck.android.NdkCxxPlatforms.build(NdkCxxPlatforms.java:542)
        at com.facebook.buck.android.NdkCxxPlatforms.getPlatforms(NdkCxxPlatforms.java:184)
        at com.facebook.buck.android.NdkCxxPlatforms.getPlatforms(NdkCxxPlatforms.java:147)
        at com.facebook.buck.rules.KnownBuildRuleTypes.createBuilder(KnownBuildRuleTypes.java:306)
        at com.facebook.buck.rules.KnownBuildRuleTypes.createInstance(KnownBuildRuleTypes.java:238)
        at com.facebook.buck.rules.KnownBuildRuleTypesFactory.create(KnownBuildRuleTypesFactory.java:46)
        at com.facebook.buck.rules.Cell.lambda$0(Cell.java:88)
        at com.google.common.base.Suppliers$MemoizingSupplier.get(Suppliers.java:120)
        at com.facebook.buck.rules.Cell.getKnownBuildRuleTypes(Cell.java:113)
        at com.facebook.buck.rules.Cell.getAllDescriptions(Cell.java:186)
        at com.facebook.buck.rules.Cell.createBuildFileParserFactory(Cell.java:265)
        at com.facebook.buck.rules.Cell.createBuildFileParser(Cell.java:230)
        at com.facebook.buck.parser.PerBuildState.createBuildFileParser(PerBuildState.java:177)
        at com.facebook.buck.parser.PerBuildState.lambda$0(PerBuildState.java:123)
        at com.facebook.buck.parser.ProjectBuildFileParserPool.lambda$1(ProjectBuildFileParserPool.java:90)
        at com.facebook.buck.util.concurrent.ResourcePool.createIfAllowed(ResourcePool.java:227)
        at com.facebook.buck.util.concurrent.ResourcePool.obtainResource(ResourcePool.java:188)
        at com.facebook.buck.util.concurrent.ResourcePool.requestResource(ResourcePool.java:164)
        at com.facebook.buck.util.concurrent.ResourcePool.access$0(ResourcePool.java:163)
        at com.facebook.buck.util.concurrent.ResourcePool$1.apply(ResourcePool.java:117)
        at com.facebook.buck.util.concurrent.ResourcePool$1.apply(ResourcePool.java:1)
        at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:211)
        at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:200)
        at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:130)
        at com.google.common.util.concurrent.MoreExecutors$5$1.run(MoreExecutors.java:988)
        at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$2(WeightedListeningExecutorService.java:109)
        at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$0(WeightedListeningExecutorService.java:81)
        at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:211)
        at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:200)
        at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:130)
        at com.google.common.util.concurrent.MoreExecutors$5$1.run(MoreExecutors.java:988)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)


C:\_repository\plovr>echo %JAVA_HOME%
C:\Program Files\Java\jdk1.8.0_121

C:\_repository\plovr>echo %ANDROID_HOME%
C:\Users\gmalartre\AppData\Local\Android\sdk

C:\_repository\plovr>echo %ANDROID_NDK%
C:\android-ndk-r14b

Can't invest more time then this for today. Hope this help!

[nicks]

are you sure you're following the right instructions? it sounds like you're following the "Buck for Windows building Android" instructions rather than the "Buck for Windows building Java" instructions. The top of the page should look like this:

maybe try zeroing out the ANDROID_HOME/ANDROID_NDK env variables and trying again?

[nicks]

(also i built a fresh Buck from head yesterday without problems...it sounds like your issues are more Buck-related and so maybe posting an issue in their repo would get you better advice than I can give...)

[gmalartre]

Argg, it made a difference for sure but now I'm missing a zip cmd in my path, I might try to install the new Bash Sheel for windows 10 to see if I can mix up the zip command.

`C:_repository\plovr>buck build plovr
Not using buckd because NO_BUCKD is set.
[-] PARSING BUCK FILES...FINISHED 1.1s [100%]
No Android platform target specified. Using default: android-23

C:_repository\plovr\buck-out\gen\closure\closure-compiler\externs_zip__srcs>cd externs && zip -r C:_repository\plovr\buck-out\gen\closure\closure-compiler\externs_zip\externs.zip *

'zip' is not recognized as an internal or external command,
operable program or batch file.

BUILT 59/71 JOBS 0.2s //closure/closure-compiler:support_srcs#class-abi
Error scrubbing non-deterministic metadata from C:_repository\plovr\buck-out\gen\closure\closure-stylesheets\javacc\parser.src.zip
java.nio.file.NoSuchFileException: C:_repository\plovr\buck-out\gen\closure\closure-stylesheets\javacc\parser.src.zip
java.nio.file.NoSuchFileException: C:_repository\plovr\buck-out\gen\closure\closure-stylesheets\javacc\parser.src.zip
at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(Unknown Source)
at java.nio.channels.FileChannel.open(Unknown Source)
at java.nio.channels.FileChannel.open(Unknown Source)
at com.facebook.buck.zip.ZipScrubber.scrubZip(ZipScrubber.java:46)
at com.facebook.buck.zip.AbstractZipScrubberStep.execute(AbstractZipScrubberStep.java:57)
at com.facebook.buck.zip.ZipScrubberStep.execute(ZipScrubberStep.java:1)
at com.facebook.buck.step.DefaultStepRunner.runStepForBuildTarget(DefaultStepRunner.java:47)
at com.facebook.buck.rules.CachingBuildEngine.executeCommandsNowThatDepsAreBuilt(CachingBuildEngine.java:1538)
at com.facebook.buck.rules.CachingBuildEngine.lambda$17(CachingBuildEngine.java:354)
at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$0(WeightedListeningExecutorService.java:81)
at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:211)
at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:200)
at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:130)
at com.google.common.util.concurrent.MoreExecutors$5$1.run(MoreExecutors.java:988)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

Error scrubbing non-deterministic metadata from C:_repository\plovr\buck-out\gen\closure\closure-templates\javacc\parser.src.zip
java.nio.file.NoSuchFileException: C:_repository\plovr\buck-out\gen\closure\closure-templates\javacc\parser.src.zip
java.nio.file.NoSuchFileException: C:_repository\plovr\buck-out\gen\closure\closure-templates\javacc\parser.src.zip
at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(Unknown Source)
at java.nio.channels.FileChannel.open(Unknown Source)
at java.nio.channels.FileChannel.open(Unknown Source)
at com.facebook.buck.zip.ZipScrubber.scrubZip(ZipScrubber.java:46)
at com.facebook.buck.zip.AbstractZipScrubberStep.execute(AbstractZipScrubberStep.java:57)
at com.facebook.buck.zip.ZipScrubberStep.execute(ZipScrubberStep.java:1)
at com.facebook.buck.step.DefaultStepRunner.runStepForBuildTarget(DefaultStepRunner.java:47)
at com.facebook.buck.rules.CachingBuildEngine.executeCommandsNowThatDepsAreBuilt(CachingBuildEngine.java:1538)
at com.facebook.buck.rules.CachingBuildEngine.lambda$17(CachingBuildEngine.java:354)
at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$0(WeightedListeningExecutorService.java:81)
at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:211)
at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:200)
at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:130)
at com.google.common.util.concurrent.MoreExecutors$5$1.run(MoreExecutors.java:988)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

BUILD FAILED: //closure/closure-compiler:externs_zip failed with exit code 1:
genrule
stderr: 'zip' is not recognized as an internal or external command,
operable program or batch file.

[-] BUILDING...FINISHED 3.0s [100%]`

The NO_BUCKD=1 environment variable seem to be mandatory in my case.

Thanks for pointing the obvious out =). But still it feel like Buck is quite a weird beast, if I install Cordova to do Android app I will always have to remove-add my ANDROID_SDK environment variable when using Buck, doesn't sound right at all.

[nicks]

yay! the zip thing is our fault i think, i'll add that to the README instructions. I usually install it with choco

let me poke around at the android sdk thing as well, it's kind of weird that it's loading the android sdk when it's not building android.

[gmalartre]

Ok, I'm still progressing!

choco install zip helped now I get the following when I build:

C:\_repository\plovr>buck build plovr
Not using buckd because NO_BUCKD is set.
[-] PARSING BUCK FILES...FINISHED 15.0s [100%]
No Android platform target specified. Using default: android-23
No Android platform target specified. Using default: android-23
BUILT 59/71 JOBS 0.9s //closure/closure-compiler:externs_zip
BUILT 60/71 JOBS 0.8s //closure/closure-compiler:externs_srcs
BUILT 61/71 JOBS 0.5s //closure/closure-compiler:externs_srcs#class-abi
BUILT 62/71 JOBS 3.0s //closure/closure-stylesheets:javacc
BUILT 65/71 JOBS 3.8s //closure/closure-templates:javacc
BUILD FAILED: //closure/closure-stylesheets:closure-stylesheets failed on step javac with an exception:
No system compiler found. Did you install the JRE instead of the JDK?
com.facebook.buck.util.HumanReadableException: No system compiler found. Did you install the JRE instead of the JDK?
        at com.facebook.buck.jvm.java.JdkProvidedInMemoryJavac.createCompiler(JdkProvidedInMemoryJavac.java:63)
        at com.facebook.buck.jvm.java.Jsr199Javac.buildWithClasspath(Jsr199Javac.java:120)
        at com.facebook.buck.jvm.java.JavacStep.performBuild(JavacStep.java:154)
        at com.facebook.buck.jvm.java.JavacStep.tryBuildWithFirstOrderDeps(JavacStep.java:144)
        at com.facebook.buck.jvm.java.JavacStep.execute(JavacStep.java:106)
        at com.facebook.buck.step.DefaultStepRunner.runStepForBuildTarget(DefaultStepRunner.java:47)
        at com.facebook.buck.rules.CachingBuildEngine.executeCommandsNowThatDepsAreBuilt(CachingBuildEngine.java:1538)
        at com.facebook.buck.rules.CachingBuildEngine.lambda$17(CachingBuildEngine.java:354)
        at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$0(WeightedListeningExecutorService.java:81)
        at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:211)
        at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:200)
        at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:130)
        at com.google.common.util.concurrent.MoreExecutors$5$1.run(MoreExecutors.java:988)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

[-] BUILDING...FINISHED 16.3s [100%]

C:\_repository\plovr>
C:\_repository\plovr>echo %JAVA_HOME%
C:\Program Files\Java\jdk1.8.0_121

As you can see the JAVA_HOME does point to a JDK so I'm not sure how the closure-stylesheet build step try to get javac.

Thanks

[gmalartre]

And the java bin directory is in my Path, so java and javac do exec in cmd.exe

On another attempt I get the same error elsewhere:

C:\_repository\plovr>buck build plovr
Not using buckd because NO_BUCKD is set.
[-] PARSING BUCK FILES...FINISHED 1.1s [100%]
BUILD FAILED: //closure/closure-stylesheets:closure-stylesheets failed on step javac with an exception:
No system compiler found. Did you install the JRE instead of the JDK?
com.facebook.buck.util.HumanReadableException: No system compiler found. Did you install the JRE instead of the JDK?
        at com.facebook.buck.jvm.java.JdkProvidedInMemoryJavac.createCompiler(JdkProvidedInMemoryJavac.java:63)
        at com.facebook.buck.jvm.java.Jsr199Javac.buildWithClasspath(Jsr199Javac.java:120)
        at com.facebook.buck.jvm.java.JavacStep.performBuild(JavacStep.java:154)
        at com.facebook.buck.jvm.java.JavacStep.tryBuildWithFirstOrderDeps(JavacStep.java:144)
        at com.facebook.buck.jvm.java.JavacStep.execute(JavacStep.java:106)
        at com.facebook.buck.step.DefaultStepRunner.runStepForBuildTarget(DefaultStepRunner.java:47)
        at com.facebook.buck.rules.CachingBuildEngine.executeCommandsNowThatDepsAreBuilt(CachingBuildEngine.java:1538)
        at com.facebook.buck.rules.CachingBuildEngine.lambda$17(CachingBuildEngine.java:354)
        at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$0(WeightedListeningExecutorService.java:81)
        at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:211)
        at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:200)
        at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:130)
        at com.google.common.util.concurrent.MoreExecutors$5$1.run(MoreExecutors.java:988)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

[-] BUILDING...FINISHED 8.2s [100%]

[nicks]

hmmm...do you have javac on PATH? i don't how it searches for the JDK but i'm also surprised it's not pulling it from JAVA_HOME

[gmalartre]

Finally got it to work! I had to change the following: remove a java.exe javaw.exe javaws.exe from my C:/WINDOWS/system32/ folder that was probably putted there by the JRE installation and was messing up the ToolProvider.getSystemJavaCompiler(); used by Buck.

Thanks @nicks

[nicks]

i added some notes to the README about zip, though it sounds like most of the issues here are Buck issues that we can't do much about. closing...