WIP: build changes for ARM

[ihnorton]

This currently compiles but does not bootstrap due to jl_cpuid. I was able to stub that out before, but I think it is more deeply entangled now.

See Makefile.user.fake for other options used.

• note DYNAMIC_ARCH=0
• edit openblas-.../c_check.c and remove the -m32 compiler option, as this appears to be unsupported by the arm GCC build (at least with Ubuntu 12.04. there are some suggestions to install g++-4.8.1-multilib, but that did not help me)

See here for all of the packages I have installed.

[ViralBShah]

This branch now fails for me during the sysimg build:

error during bootstrap: LoadError(at "sysimg.jl" line 196: LoadError(at "version.jl" line 100: BoundsError()))

[throwaway-julia-arm]

On top of a "arm-make3" branch that had been rebased atop "master", I also needed

--- a/src/support/platform.h
+++ b/src/support/platform.h
@@ -83,6 +83,8 @@
 #  define _P64
 #elif defined(_CPU_X86_)
 #  define _P32
+#elif defined(_CPU_ARM_)
+#  define _P32
 #elif defined(_OS_WINDOWS_)
 /* Not sure how to determine pointer size on Windows running ARM. */
 #  if _WIN64

to get past:

In file included from jltypes.c:13:
In file included from ./julia.h:10:
In file included from support/libsupport.h:4:
support/platform.h:100:4: error: pointer size not known for your platform / compiler

Which got me as far as trying to bootstrap, where things went:

primes.jl
error during bootstrap: LoadError(at "sysimg.jl" line 131: LoadError(at "primes.jl" line 76: InexactError()))
 jl_cpuid not implemented on arm

[ihnorton]

This branch builds cleanly through bootstrap to REPL now, against LLVM release_35. svn should work too, but I had seen some random assertion failures during 2nd stage bootstrap and switched to 35, and I don't want to recompile. (but those were probably due to #8200 anyway)

FYI: OpenBLAS 0.2.11 builds cleanly on ARM, but I didn't change it in the inc file yet. With USE_SYSTEM_LAPACK it is necessary to install liblapack3gf and then create a symlink from /usr/lib/liblapack.so to /usr/lib/libpack.so.3gf. But there are some missing functions, so some linalg tests fail.

Unless someone has an objection, I am going to squash and force-push to this branch later today to clean things up a little bit.

[sjkelly]

@ihnorton This is great! I will start a build tomorrow on a BBB and report back the results.

[ViralBShah]

Please squash. Should we also rebase to master? I am building it right now.

[ViralBShah]

If we can get basic julia to build with all the USE_SYSTEM_* flags first, that would be great. Then, we can slowly enable all the libraries one at a time. Quite likely, there will be openblas issues, since the ARM version is not well tested yet.

[ViralBShah]

It gave me a prompt successfully. This is huge! Tests are another story, but we can go step by step. Rebasing on master and trying again.

[ViralBShah]

[ihnorton]

I already rebased and pushed.

Very cool. Good to have independent confirmation that it was not a 'shop :p

[ihnorton]

I kind of cheated with the cpuid thing. The only way to get cpuid on ARM is via procinfo, because the registers require privileged mode on the CPU. There are a couple implementations that we could crib from, either in Android or OpenBLAS, but it is annoying because there is some subtle variability in the procinfo output depending on kernel version (I think).

[pao]

Is this something that we can fix by switching from calling cpuid to using the hwloc library?

[ihnorton]

I squashed and rebased already.

Suitesparse-dev seems to be missing on Ubuntu 12.04, so I have that set to
build locally.
On Sep 6, 2014 6:02 AM, "Viral B. Shah" notifications@github.com wrote:

Please squash. Should we also rebase to master? I am building it right now.

—
Reply to this email directly or view it on GitHub
#7662 (comment).

[ViralBShah]

Ah - it does exist, but not for arm.

[ViralBShah]

Should we merge this into master and close this PR, and then work on getting all tests to pass separately?

[ViralBShah]

BTW, @ihnorton did you forget to push after squashing? I still see 9 commits.

[tkelman]

shouldn't this be contingent on ifeq ($(ARCH), arm) ?

[ihnorton]

yeah, good point.

[ivarne]

Also, this whole section is unnecessary if people follow the advise in the new README.arm and have include $(JULIAHOME)/ARM.inc in their Make.user file

[ViralBShah]

@tkelman the inclusion of ARM.inc is only on ARM boxes. Are you thinking about cross-compiling on ARM machines to something else?

Eventually, ARM.inc will go away, and for now, the way it is included, I can't fully understand what it will break. Can you explain a bit more?

[ViralBShah]

@ihnorton I suggest keeping LLVM_ASSERTIONS around until we get all tests to pass and all libraries to build.

[ihnorton]

I think we should just keep ARM.inc and tell people to include it into Make.user (I just updated the README to say as much).

Good point on assertions, that makes sense. It is great that system MPFR works, but I noticed that there are some missing functions in the system libblas.

[tkelman]

@tkelman the inclusion of ARM.inc is only on ARM boxes. Are you thinking about cross-compiling on ARM machines to something else?

Eventually, ARM.inc will go away, and for now, the way it is included, I can't fully understand what it will break. Can you explain a bit more?

No, nothing to do with ARM, the existing cross-compiles from Linux to Windows would be broken by rearranging ARCH detection and anything dependent on XC_HOST to before the inclusion of Make.user.

[ihnorton]

@tkelman I got rid of that commit.

[ViralBShah]

@ihnorton I am going through the libraries one by one. I expect all of them to build, and we do not have to depend on system provided ones.

[ihnorton]

@ViralBShah yes, I think they should all build cleanly (there was a tiny patch required for OpenBLAS, but it is in the latest version).

[ihnorton]

@pao hwloc seems to only provide CPU topology, not the CPU model.

[tkelman]

I looked at hwloc a couple weeks ago and it can give you CPU model (at least on the x86 machines I tried it on), but kind of buried and not as its primary output. I had to ask for everything about a given processor and some of the CPU model info was in the output.

[ihnorton]

Hmm, have been playing with hwloc-info and lstopo with hwloc compiled on the arm box, and I didn't see any model info yet. hwloc-calc just hangs 🌵

[tkelman]

How much output do you get from ./hwloc-info --ancestors core:0 ?

[ihnorton]

Ah there we go - now more than I ever wanted to know! Seems very useful.

[ViralBShah]

The FFTW tests after building FFTW (not the Julia FFT) fail for me.

[ViralBShah]

This is the error. @stevengj Do you know about this on ARM?

Executing "/home/viral/julia/deps/fftw-3.3.4-single/tests/bench --verbose=1   --verify 
'ok11e01x11o10x5e11' --verify 'ik11e01x11o10x5e11' --verify 'obr5x12v13' --verify 'ibr5x12v13' --verify
 'ofr5x12v13' --verify 'ifr5x12v13' --verify '//obc5x12v13' --verify '//ibc5x12v13' --verify '//ofc5x12v13' --verify
 '//ifc5x12v13' --verify 'obc5x12v13' --verify 'ibc5x12v13' --verify 'ofc5x12v13' --verify 'ifc5x12v13' --verify
 'ok9bx11bx5e01v7' --verify 'ik9bx11bx5e01v7' --verify '//obr9x12x7x15' --verify '//ofr9x12x7x15' --verify
 'obr9x12x7x15' --verify 'ibr9x12x7x15' --verify 'ofr9x12x7x15' --verify 'ifr9x12x7x15' --verify
 '//obc9x12x7x15' --verify '//ibc9x12x7x15' --verify '//ofc9x12x7x15' --verify '//ifc9x12x7x15' --verify
 'obc9x12x7x15' --verify 'ibc9x12x7x15' --verify 'ofc9x12x7x15' --verify 'ifc9x12x7x15' --verify
 'ok5o11x11o11x3bv9' --verify 'ik5o11x11o11x3bv9' --verify 'obrd4x9v18' --verify 'ibrd4x9v18' --verify
 'ofrd4x9v18' --verify 'ifrd4x9v18' --verify '//obcd4x9v18' --verify '//ibcd4x9v18' --verify '//ofcd4x9v18' --verify
 '//ifcd4x9v18' --verify 'obcd4x9v18' --verify 'ibcd4x9v18' --verify 'ofcd4x9v18' --verify 'ifcd4x9v18'"
Segmentation fault (core dumped)

[sjkelly]

Should this still be ppc64?

[ihnorton]

there is a condition above it for ARM..

[ViralBShah]

I think this is good to merge now.

[ihnorton]

I just noticed that libuv has uv_cpu_info which should be sufficient for our purposes, so I am going to switch to that on ARM. hwloc is overkill, although I understand there may be some other reasons to add it as a dependency eventually.

[ViralBShah]

Sounds reasonable.