Linking to blas on openSUSE for MXNet in Julia 0.4.6

[colbec]

Currently, on the openSUSE repo version of Julia 0.4.6 it is difficult to work with MXNet where the Pkg.add, checkout, and build all perform well but the tests fail due to a doubtful link to the correct version of BLAS. Julia version 0.6 however has no problem installing and running the tests, indicating that it is the build of 0.4.6 for openSUSE which is perhaps at the root of the matter.

I opened in error an issue at apache/mxnet#3244, where we learn that MXNet requires openblas. There is a suggestion that a hard link to a fixed version of openblas might help, rather than letting the openSUSE system decide which library to use.

(cc: @ronisbr)

[yuyichao]

I don't think we build the OpenSUSE repo version. What's the action item here?

[colbec]

Oh, in that case I must have misunderstood the situation. Permit me to close the issue and I will take it up elsewhere. Thanks.

[colbec]

@tkelman FWIW I deleted my openSUSE repo version of 0.4.6 and wiped the cache. I then installed the 0.4.6 and v 0.5r3 from "Generic Linux" binaries on the Julia page. The 0.4.6 installs but will not build MXNet from checkout master, stops at the -lcblas again. But 0.5r3 is good. It builds MXNet and passes tests like 0.6. A few depwarns but that is future stuff.

[tkelman]

who is trying to link against cblas? julia doesn't ship that, so back to being an mxnet.jl issue.

on opensuse packaging, who is the maintainer there? i guess they use bugzilla for tracking opensuse package issues?

[andreasnoack]

OpenBLAS includes CBLAS, we use it and it seems that MXNet is also calling it. Probably for row major support.

[colbec]

I guess it must be MXNet trying to link. It is when I try to build MXNet that it crops up.
openSUSE packaging: one of the maintainers is ronisbr in Brazil; in my OP please note my cc to him.
But since both the Julia generic binary and the opensuse repo versions both stop with the same issue this might indicate it is beyond the openSUSE maintainer's specific domain.

[tkelman]

we don't ship it under the libcblas name. you might be using different versions of mxnet.jl on 0.4 vs 0.5?

[vchuravy]

MXNet.jl tries to use the BLAS installed with Julia, but we were not good enough at detecting when it was build with something that was not openblas.
@colbec let's take this discussion to the MXNet.jl repro so that we won't generate more traffic than necessary on this issue tracker.

[ronisbr]

Hi @colbec !

In openSUSE, we do not build julia against openblas. If you execute the runtests.jl as it is, it will fail. However, you can always install openblas and use update-alternative to make it the default libblas and liblapack of your system. It can be done using:

update-alternatives --config liblapack.so.3
update-alternatives --config libblas.so.3 

After changing to openblas, all the tests in runtest.jl of julia pass. Hence, maybe you can use MXNet.jl after this change also.

[colbec]

Hi @ronisbr , thanks for the suggestion. I did as you suggested and linked to the most obvious alternatives in update-alternative and rebooted and attempted a rebuild of MXNet on 0.4.6 but it failed as before not able to find clbas.h. It is definitely there in /usr/include/openblas/, strange that it cannot be found by MXNet.
But anyway it is all academic right now, MXNet builds and runs fine in Julia 0.5r3 and 0.6 so users can assess their own priority on this issue.

[tkelman]

@ronisbr opensuse should make openblas a mandatory dependency of julia. there's no reason to do otherwise, it makes julia look bad if basic linear algebra operations are slow and serial.

[ronisbr]

Hi @tkelman,

This is strange. For my simulations, in which there are a lot of operations with small matrices (3x3, 4x4), I see no gain at all using openblas. Actually, sometimes openblas is slower. Do you have any good explanation to that?

In terms of turning openblas as a mandatory dependency, I think I cannot do it. Because, even if openblas can get installed with julia, the user must choose it as the default implementation of the libraries using update-alternative.

[KristofferC]

The point of BLAS is not to be fast for small matrices but for large ones. For 2x2 and 3x3 Julia don't even call BLAS.

[ronisbr]

Thanks @KristofferC . That explains a lot :)

[tkelman]

set the openblas name directly, don't use the libblas update-alternatives alias. needless complexity there that we should just avoid.

[ronisbr]

But then there are some versions of openblas:

S | Nome                       | Resumo                                                           | Tipo  
--+----------------------------+------------------------------------------------------------------+-------
i | libopenblas_openmp0        | An optimized BLAS library based on GotoBLAS2, OpenMP version     | pacote
i | libopenblas_pthreads0      | An optimized BLAS library based on GotoBLAS2, pthreads version   | pacote
i | libopenblas_serial0        | An optimized BLAS library based on GotoBLAS2, serial version     | pacote
i | libopenblaso0              | An optimized BLAS library based on GotoBLAS2, OpenMP version     | pacote
i | libopenblasp0              | An optimized BLAS library based on GotoBLAS2, pthreads version   | pacote

Which one should I use? By using the update-alternatives approach, I can let the user decide which one is better.

[tkelman]

either the pthreads one or the openmp one (check which version the source build of julia defaults to). both should be user-controllable, and probably not much different in performance from one another. update-alternatives doesn't really work for julia if the threading api changes, since you'd need to rebuild the system image if the blas vendor or set_num_threads api functions need to change. just swapping the symlink isn't enough.

[ronisbr]

Thanks @tkelman! I will add openblas as a mandatory dependency and set:

LIBBLASNAME=libopenblas.so.0 \
LIBLAPACKNAME=libopenblas.so.0 \

Is it correct?

[tkelman]

I think so. That will be one of the parallel versions?

[ronisbr]

@tkelman sorry! That should be libopenblas_openmp.so.0.

[ronisbr]

Hi guys!

I updated julia to using openblas by default as suggested by @tkelman . The new version is already submitted to my home repo and should hit tumbleweed soon. Now, versioninfo() is:

julia> versioninfo()
Julia Version 0.5.0-rc4+0
Commit 9c76c3e* (2016-09-09 01:43 UTC)
Platform Info:
  System: Linux (x86_64-suse-linux)
  CPU: Intel(R) Xeon(R) CPU E5-2687W 0 @ 3.10GHz
  WORD_SIZE: 64
  BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas_openmp.so.0
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, sandybridge)

Notice that all tests in runtests.jl pass now without any modification.

Thanks for the tip @tkelman .

[tkelman]

much better, thanks! was the test failure against reference blas a numerical issue or missing function?

[ronisbr]

It was due to missing functions.

[tkelman]

Was it the cblas functions we use in a few places where the calling convention is ambiguous between different compilers? ref JuliaLang/LinearAlgebra.jl#137 - I thought most distros build the cblas API into libblas these days but maybe some separate them.

[ronisbr]

@tkelman here is the output of runtests.jl when julia was compiled and executed against libblas and liblapack:

# julia runtests.jl 
    From worker 7:       * linalg/special        in   4.15 seconds, maxrss  209.59 MB
    From worker 6:       * linalg/schur          in  15.64 seconds, maxrss  244.60 MB
    From worker 6:       * linalg/lapack        Error During Test
    From worker 6:    Test threw an exception of type ErrorException
    From worker 6:    Expression: (LAPACK.gglse!(A,c,B,d))[1] ≈ convert(Array{elty},[0.5,-0.5,1.5,0.5])
    From worker 6:    ccall: could not find function cblas_cdotc_sub in library liblapack.so.3
    From worker 6:     in dotc(::Int64, ::Ptr{Complex{Float32}}, ::Int64, ::Ptr{Complex{Float32}}, ::Int64) at ./linalg/blas.jl:264
    From worker 6:     in dotc(::SubArray{Complex{Float32},1,Array{Complex{Float32},1},Tuple{UnitRange{Int64}},true}, ::SubArray{Complex{Float32},1,Array{Complex{Float32},1},Tuple{UnitRange{Int64}},true}) at ./linalg/blas.jl:301
    From worker 6:     in gglse!(::Array{Complex{Float32},2}, ::Array{Complex{Float32},1}, ::Array{Complex{Float32},2}, ::Array{Complex{Float32},1}) at ./linalg/lapack.jl:1440
    From worker 6:     in macro expansion; at /usr/share/julia/test/linalg/lapack.jl:39 [inlined]
    From worker 6:     in anonymous at ./<missing>:?
    From worker 6:     in include_string(::String, ::String) at ./loading.jl:380
    From worker 6:     in include_from_node1(::String) at ./loading.jl:429
    From worker 6:     in macro expansion at ./util.jl:226 [inlined]
    From worker 6:     in runtests(::String) at /usr/share/julia/test/testdefs.jl:7
    From worker 6:     in (::Base.Serializer.__deserialized_types__.##16#24)(::String) at /usr/share/julia/test/runtests.jl:44
    From worker 6:     in (::Base.##625#627{Base.CallMsg{:call_fetch}})() at ./multi.jl:1421
    From worker 6:     in run_work_thunk(::Base.##625#627{Base.CallMsg{:call_fetch}}, ::Bool) at ./multi.jl:1001
    From worker 6:     in macro expansion at ./multi.jl:1421 [inlined]
    From worker 6:     in (::Base.##624#626{Base.CallMsg{:call_fetch},Base.MsgHeader,TCPSocket})() at ./event.jl:68
    From worker 3:       * linalg/qr            Error During Test
    From worker 3:    Test threw an exception of type ErrorException
    From worker 3:    Expression: abs(((qrfact(α))[:Q])[1,1]) ≈ one(eltyb)
    From worker 3:    ccall: could not find function cblas_cdotc_sub in library liblapack.so.3
    From worker 3:     in dotc(::Int64, ::Ptr{Complex{Float32}}, ::Int64, ::Ptr{Complex{Float32}}, ::Int64) at ./linalg/blas.jl:264
    From worker 3:     in dotc(::Array{Complex{Float32},1}, ::Array{Complex{Float32},1}) at ./linalg/blas.jl:301
    From worker 3:     in getindex(::Base.LinAlg.QRCompactWYQ{Complex{Float32},Array{Complex{Float32},2}}, ::Int64, ::Int64) at ./linalg/qr.jl:339
    From worker 3:     in macro expansion; at /usr/share/julia/test/linalg/qr.jl:38 [inlined]
    From worker 3:     in anonymous at ./<missing>:?
    From worker 3:     in include_string(::String, ::String) at ./loading.jl:380
    From worker 3:     in include_from_node1(::String) at ./loading.jl:429
    From worker 3:     in macro expansion at ./util.jl:226 [inlined]
    From worker 3:     in runtests(::String) at /usr/share/julia/test/testdefs.jl:7
    From worker 3:     in (::Base.Serializer.__deserialized_types__.##16#24)(::String) at /usr/share/julia/test/runtests.jl:44
    From worker 3:     in (::Base.##625#627{Base.CallMsg{:call_fetch}})() at ./multi.jl:1421
    From worker 3:     in run_work_thunk(::Base.##625#627{Base.CallMsg{:call_fetch}}, ::Bool) at ./multi.jl:1001
    From worker 3:     in macro expansion at ./multi.jl:1421 [inlined]
    From worker 3:     in (::Base.##624#626{Base.CallMsg{:call_fetch},Base.MsgHeader,TCPSocket})() at ./event.jl:68
    From worker 7:       * linalg/svd            in  23.26 seconds, maxrss  267.32 MB
    From worker 9:       * linalg/bunchkaufman   in  40.95 seconds, maxrss  253.79 MB
    From worker 8:       * linalg/eigen          in  45.45 seconds, maxrss  282.60 MB
    From worker 4:       * linalg/dense         Error During Test
    From worker 4:    Test threw an exception of type ErrorException
    From worker 4:    Expression: ((a[:,1:2] * x - b[:,1])' * (a[:,1:2] * x - b[:,1]))[1] ≈ convert(elty,2.546616541353384)
    From worker 4:    ccall: could not find function cblas_cdotc_sub in library liblapack.so.3
    From worker 4:     in dotc(::Int64, ::Ptr{Complex{Float32}}, ::Int64, ::Ptr{Complex{Float32}}, ::Int64) at ./linalg/blas.jl:264
    From worker 4:     in dotc(::Array{Complex{Float32},1}, ::Array{Complex{Float32},1}) at ./linalg/blas.jl:301
    From worker 4:     in Ac_mul_B(::Array{Complex{Float32},1}, ::Array{Complex{Float32},1}) at ./linalg/matmul.jl:73
    From worker 4:     in macro expansion; at /usr/share/julia/test/linalg/dense.jl:483 [inlined]
    From worker 4:     in anonymous at ./<missing>:?
    From worker 4:     in include_string(::String, ::String) at ./loading.jl:380
    From worker 4:     in include_from_node1(::String) at ./loading.jl:429
    From worker 4:     in macro expansion at ./util.jl:226 [inlined]
    From worker 4:     in runtests(::String) at /usr/share/julia/test/testdefs.jl:7
    From worker 4:     in (::Base.Serializer.__deserialized_types__.##16#24)(::String) at /usr/share/julia/test/runtests.jl:44
    From worker 4:     in (::Base.##625#627{Base.CallMsg{:call_fetch}})() at ./multi.jl:1421
    From worker 4:     in run_work_thunk(::Base.##625#627{Base.CallMsg{:call_fetch}}, ::Bool) at ./multi.jl:1001
    From worker 4:     in macro expansion at ./multi.jl:1421 [inlined]
    From worker 4:     in (::Base.##624#626{Base.CallMsg{:call_fetch},Base.MsgHeader,TCPSocket})() at ./event.jl:68
    From worker 8:       * linalg/givens         in   3.60 seconds, maxrss  286.91 MB
    From worker 9:       * linalg/pinv           in   9.08 seconds, maxrss  351.80 MB
    From worker 10:      * linalg/tridiag        in  35.23 seconds, maxrss  279.22 MB
    From worker 11:      * linalg/bidiag         in  37.75 seconds, maxrss  291.66 MB
    From worker 10:      * linalg/generic        in   5.99 seconds, maxrss  286.81 MB
    From worker 11:      * linalg/uniformscaling in   3.99 seconds, maxrss  303.10 MB
    From worker 11:      * linalg/hessenberg    Error During Test
    From worker 11:   Test threw an exception of type ErrorException
    From worker 11:   Expression: (H[:Q])[1,1] ≈ (full(H[:Q]))[1,1]
    From worker 11:   ccall: could not find function cblas_cdotc_sub in library liblapack.so.3
    From worker 11:    in dotc(::Int64, ::Ptr{Complex{Float32}}, ::Int64, ::Ptr{Complex{Float32}}, ::Int64) at ./linalg/blas.jl:264
    From worker 11:    in dotc(::Array{Complex{Float32},1}, ::Array{Complex{Float32},1}) at ./linalg/blas.jl:301
    From worker 11:    in getindex(::Base.LinAlg.HessenbergQ{Complex{Float32},Array{Complex{Float32},2}}, ::Int64, ::Int64) at ./linalg/hessenberg.jl:52
    From worker 11:    in macro expansion; at /usr/share/julia/test/linalg/hessenberg.jl:33 [inlined]
    From worker 11:    in anonymous at ./<missing>:?
    From worker 11:    in include_string(::String, ::String) at ./loading.jl:380
    From worker 11:    in include_from_node1(::String) at ./loading.jl:429
    From worker 11:    in macro expansion at ./util.jl:226 [inlined]
    From worker 11:    in runtests(::String) at /usr/share/julia/test/testdefs.jl:7
    From worker 11:    in (::Base.Serializer.__deserialized_types__.##16#24)(::String) at /usr/share/julia/test/runtests.jl:44
    From worker 11:    in (::Base.##625#627{Base.CallMsg{:call_fetch}})() at ./multi.jl:1421
    From worker 11:    in run_work_thunk(::Base.##625#627{Base.CallMsg{:call_fetch}}, ::Bool) at ./multi.jl:1001
    From worker 11:    in macro expansion at ./multi.jl:1421 [inlined]
    From worker 11:    in (::Base.##624#626{Base.CallMsg{:call_fetch},Base.MsgHeader,TCPSocket})() at ./event.jl:68
    From worker 10:      * linalg/lq            Error During Test
    From worker 10:   Test threw an exception of type ErrorException
    From worker 7:       * linalg/diagonal       in  48.16 seconds, maxrss  327.25 MB
    From worker 10:   Expression: eye(eltyb,n) * q ≈ convert(AbstractMatrix{tab},q)
    From worker 10:   ccall: could not find function cblas_cdotc_sub in library liblapack.so.3
    From worker 10:    in dotc(::Int64, ::Ptr{Complex{Float32}}, ::Int64, ::Ptr{Complex{Float32}}, ::Int64) at ./linalg/blas.jl:264
    From worker 10:    in dotc(::Array{Complex{Float32},1}, ::Array{Complex{Float32},1}) at ./linalg/blas.jl:301
    From worker 10:    in getindex(::Base.LinAlg.LQPackedQ{Complex{Float32},Array{Complex{Float32},2}}, ::Int64, ::Int64) at ./linalg/lq.jl:76
    From worker 10:    in _elementwise(::Base.#-, ::Type{Complex{Float32}}, ::Array{Complex{Float32},2}, ::Base.LinAlg.LQPackedQ{Complex{Float32},Array{Complex{Float32},2}}) at ./arraymath.jl:59
    From worker 10:    in #isapprox#10(::Float32, ::Int64, ::Base.LinAlg.#vecnorm, ::Function, ::Array{Complex{Float32},2}, ::Base.LinAlg.LQPackedQ{Complex{Float32},Array{Complex{Float32},2}}) at ./linalg/generic.jl:590
    From worker 10:    in isapprox(::Array{Complex{Float32},2}, ::Base.LinAlg.LQPackedQ{Complex{Float32},Array{Complex{Float32},2}}) at ./linalg/generic.jl:590
    From worker 10:    in eval_comparison(::Expr) at ./test.jl:169
    From worker 10:    in macro expansion; at /usr/share/julia/test/linalg/lq.jl:66 [inlined]
    From worker 10:    in anonymous at ./<missing>:?
    From worker 10:    in include_string(::String, ::String) at ./loading.jl:380
    From worker 10:    in include_from_node1(::String) at ./loading.jl:429
    From worker 10:    in macro expansion at ./util.jl:226 [inlined]
    From worker 10:    in runtests(::String) at /usr/share/julia/test/testdefs.jl:7
    From worker 10:    in (::Base.Serializer.__deserialized_types__.##16#24)(::String) at /usr/share/julia/test/runtests.jl:44
    From worker 10:    in (::Base.##625#627{Base.CallMsg{:call_fetch}})() at ./multi.jl:1421
    From worker 10:    in run_work_thunk(::Base.##625#627{Base.CallMsg{:call_fetch}}, ::Bool) at ./multi.jl:1001
    From worker 10:    in macro expansion at ./multi.jl:1421 [inlined]
    From worker 10:    in (::Base.##624#626{Base.CallMsg{:call_fetch},Base.MsgHeader,TCPSocket})() at ./event.jl:68
    From worker 12:      * linalg/symmetric      in  26.42 seconds, maxrss  267.53 MB
    From worker 5:       * linalg/matmul        Error During Test
WARNING: Method definition f(Tuple{Vararg{Int64, #N<:Any}}, AbstractArray{#T<:Any, #N<:Any}) in module Main at /usr/share/julia/test/core.jl:706 overwritten at /usr/share/julia/test/core.jl:712.
    From worker 5:    Test threw an exception of type ErrorException
    From worker 5:    Expression: dot(x,1:2,y,1:2) == convert(elty,12.5)
    From worker 5:    ccall: could not find function cblas_cdotc_sub in library liblapack.so.3
    From worker 5:     in dotc(::Int64, ::Ptr{Complex{Float32}}, ::Int64, ::Ptr{Complex{Float32}}, ::Int64) at ./linalg/blas.jl:264
    From worker 5:     in dot(::Array{Complex{Float32},1}, ::UnitRange{Int64}, ::Array{Complex{Float32},1}, ::UnitRange{Int64}) at ./linalg/matmul.jl:70
    From worker 5:     in macro expansion; at /usr/share/julia/test/linalg/matmul.jl:254 [inlined]
    From worker 5:     in anonymous at ./<missing>:?
    From worker 5:     in include_string(::String, ::String) at ./loading.jl:380
    From worker 5:     in include_from_node1(::String) at ./loading.jl:429
    From worker 5:     in macro expansion at ./util.jl:226 [inlined]
    From worker 5:     in runtests(::String) at /usr/share/julia/test/testdefs.jl:7
    From worker 5:     in (::Base.Serializer.__deserialized_types__.##16#24)(::String) at /usr/share/julia/test/runtests.jl:44
    From worker 5:     in (::Base.##625#627{Base.CallMsg{:call_fetch}})() at ./multi.jl:1421
    From worker 5:     in run_work_thunk(::Base.##625#627{Base.CallMsg{:call_fetch}}, ::Bool) at ./multi.jl:1001
    From worker 5:     in macro expansion at ./multi.jl:1421 [inlined]
    From worker 5:     in (::Base.##624#626{Base.CallMsg{:call_fetch},Base.MsgHeader,TCPSocket})() at ./event.jl:68
    From worker 14:      * inference             in   0.89 seconds, maxrss  209.84 MB
    From worker 14:      * keywordargs           in   1.03 seconds, maxrss  209.84 MB
WARNING: Method definition f() in module JLCall14301 at /usr/share/julia/test/core.jl:3529 overwritten at /usr/share/julia/test/core.jl:3539.
    From worker 9:       * linalg/lu             in  35.69 seconds, maxrss  351.80 MB
    From worker 9:       * char                  in   0.53 seconds, maxrss  351.80 MB
    From worker 8:       * linalg/cholesky       in  39.36 seconds, maxrss  326.02 MB
    From worker 8:       * triplequote           in   0.01 seconds, maxrss  326.02 MB
    From worker 8:       * unicode               in   2.75 seconds, maxrss  326.02 MB
    From worker 15:      * printf                in  10.94 seconds, maxrss  218.77 MB
    From worker 9:       * string                in   7.59 seconds, maxrss  353.98 MB
    From worker 13:      * linalg/arnoldi        in  25.77 seconds, maxrss  285.48 MB
    From worker 12:      * core                  in  18.98 seconds, maxrss  307.07 MB
    From worker 13:      * iobuffer              in   0.75 seconds, maxrss  286.83 MB
    From worker 12:      * staged                in   0.49 seconds, maxrss  309.10 MB
    From worker 9:       * hashing               in   3.99 seconds, maxrss  356.35 MB
    From worker 9:       * tuple                 in   1.19 seconds, maxrss  358.52 MB
    From worker 15:      * dict                  in   7.36 seconds, maxrss  249.50 MB
WARNING: could not attach metadata for @simd loop.
    From worker 9:       * reduce                in   3.01 seconds, maxrss  379.77 MB
    From worker 9:       * random                in   8.04 seconds, maxrss  402.71 MB
    From worker 15:      * reducedim             in  11.50 seconds, maxrss  272.47 MB
    From worker 15:      * intfuncs              in   0.82 seconds, maxrss  275.41 MB
    From worker 15:      * simdloop              in   1.09 seconds, maxrss  275.59 MB
    From worker 15:      * vecelement            in   1.25 seconds, maxrss  281.61 MB
    From worker 8:       * dates                 in  24.83 seconds, maxrss  389.29 MB
    From worker 12:      * arrayops              in  34.70 seconds, maxrss  516.56 MB
    From worker 14:      * numbers               in  55.24 seconds, maxrss  384.95 MB
    From worker 14:      * copy                  in   1.07 seconds, maxrss  392.72 MB
    From worker 13:      * offsetarray           in  45.27 seconds, maxrss  384.87 MB
    From worker 15:      * blas                 Error During Test
    From worker 15:   Test threw an exception of type ErrorException
    From worker 15:   Expression: BLAS.dotc(z1,z2) ≈ sum(conj(z1) .* z2)
    From worker 15:   ccall: could not find function cblas_cdotc_sub in library liblapack.so.3
    From worker 15:    in dotc(::Int64, ::Ptr{Complex{Float32}}, ::Int64, ::Ptr{Complex{Float32}}, ::Int64) at ./linalg/blas.jl:264
    From worker 15:    in dotc(::Array{Complex{Float32},1}, ::Array{Complex{Float32},1}) at ./linalg/blas.jl:301
    From worker 15:    in macro expansion; at /usr/share/julia/test/blas.jl:63 [inlined]
    From worker 15:    in anonymous at ./<missing>:?
    From worker 15:    in include_string(::String, ::String) at ./loading.jl:380
    From worker 15:    in include_from_node1(::String) at ./loading.jl:429
    From worker 15:    in macro expansion at ./util.jl:226 [inlined]
    From worker 15:    in runtests(::String) at /usr/share/julia/test/testdefs.jl:7
    From worker 15:    in (::Base.Serializer.__deserialized_types__.##16#24)(::String) at /usr/share/julia/test/runtests.jl:44
    From worker 15:    in (::Base.##625#627{Base.CallMsg{:call_fetch}})() at ./multi.jl:1421
    From worker 15:    in run_work_thunk(::Base.##625#627{Base.CallMsg{:call_fetch}}, ::Bool) at ./multi.jl:1001
    From worker 15:    in macro expansion at ./multi.jl:1421 [inlined]
    From worker 15:    in (::Base.##624#626{Base.CallMsg{:call_fetch},Base.MsgHeader,TCPSocket})() at ./event.jl:68
    From worker 13:      * fastmath              in   3.43 seconds, maxrss  387.34 MB
    From worker 16:      * operators             in   0.71 seconds, maxrss  209.84 MB
    From worker 16:      * path                  in   3.21 seconds, maxrss  209.84 MB
    From worker 16:      * ccall                 in   3.53 seconds, maxrss  213.98 MB
    From worker 9:       * abstractarray         in  46.31 seconds, maxrss  481.38 MB
    From worker 9:       * loading               in   0.05 seconds, maxrss  481.38 MB
    From worker 16:      * parse                 in   2.00 seconds, maxrss  218.14 MB
    From worker 16:      * bigfloat              in   0.28 seconds, maxrss  218.82 MB
    From worker 9:       * bigint                in   1.33 seconds, maxrss  482.89 MB
    From worker 9:       * statistics            in   7.31 seconds, maxrss  495.56 MB
    From worker 13:      * functional            in  20.24 seconds, maxrss  406.01 MB
    From worker 13:      * backtrace             in   0.46 seconds, maxrss  406.01 MB
    From worker 9:       * spawn                       [stdio passthrough ok]
    From worker 13:      * priorityqueue         in   0.99 seconds, maxrss  406.01 MB
    From worker 14:      * math                  in  33.47 seconds, maxrss  490.66 MB
    From worker 9:   in  17.57 seconds, maxrss  515.50 MB
WARNING: file permissions tests skipped due to running tests as root (not recommended)
WARNING: file permissions tests skipped due to running tests as root (not recommended)
    From worker 14:      * read                  in  12.32 seconds, maxrss  558.41 MB
    From worker 14:      * version               in   0.76 seconds, maxrss  558.61 MB
    From worker 14:      * resolve               in   2.76 seconds, maxrss  569.01 MB
    From worker 8:       * sparse               Error During Test
    From worker 8:    Test threw an exception of type ErrorException
    From worker 16:      * sorting               in  29.69 seconds, maxrss  277.07 MB
    From worker 8:    Expression: dot(x,x) ≈ dot(xf,xf)
    From worker 8:    ccall: could not find function cblas_zdotc_sub in library liblapack.so.3
    From worker 8:     in dotc(::Int64, ::Ptr{Complex{Float64}}, ::Int64, ::Ptr{Complex{Float64}}, ::Int64) at ./linalg/blas.jl:264
    From worker 8:     in dotc(::Array{Complex{Float64},1}, ::Array{Complex{Float64},1}) at ./linalg/blas.jl:301
    From worker 8:     in dot(::Array{Complex{Float64},1}, ::Array{Complex{Float64},1}) at ./linalg/generic.jl:302
    From worker 8:     in include_string(::String, ::String) at ./loading.jl:380
    From worker 8:     in include_from_node1(::String) at ./loading.jl:429
    From worker 8:     in include_string(::String, ::String) at ./loading.jl:380
    From worker 8:     in include_from_node1(::String) at ./loading.jl:429
    From worker 8:     in macro expansion at ./util.jl:226 [inlined]
    From worker 8:     in runtests(::String) at /usr/share/julia/test/testdefs.jl:7
    From worker 8:     in (::Base.Serializer.__deserialized_types__.##16#24)(::String) at /usr/share/julia/test/runtests.jl:44
    From worker 8:     in (::Base.##625#627{Base.CallMsg{:call_fetch}})() at ./multi.jl:1421
    From worker 8:     in run_work_thunk(::Base.##625#627{Base.CallMsg{:call_fetch}}, ::Bool) at ./multi.jl:1001
    From worker 8:     in macro expansion at ./multi.jl:1421 [inlined]
    From worker 8:     in (::Base.##624#626{Base.CallMsg{:call_fetch},Base.MsgHeader,TCPSocket})() at ./event.jl:68
    From worker 16:      * mpfr                  in   2.49 seconds, maxrss  281.52 MB
    From worker 14:      * pollfd                in   3.38 seconds, maxrss  569.01 MB
    From worker 9:       * mmap                  in   8.84 seconds, maxrss  516.06 MB
    From worker 9:       * floatapprox           in   0.33 seconds, maxrss  516.06 MB
    From worker 17:      * socket                in   2.68 seconds, maxrss  209.84 MB
    From worker 14:      * complex               in   4.72 seconds, maxrss  569.01 MB
    From worker 14:      * regex                 in   0.34 seconds, maxrss  569.01 MB
    From worker 14:      * float16               in   0.69 seconds, maxrss  569.01 MB
    From worker 14:      * combinatorics         in   0.28 seconds, maxrss  569.01 MB
    From worker 14:      * sysinfo               in   0.42 seconds, maxrss  569.01 MB
    From worker 9:       * datafmt               in   4.78 seconds, maxrss  516.73 MB
    From worker 14:      * env                   in   0.42 seconds, maxrss  569.01 MB
    From worker 17:      * reflection            in   3.66 seconds, maxrss  231.17 MB
    From worker 17:      * mod2pi                in   0.26 seconds, maxrss  232.26 MB
    From worker 9:       * rounding              in   0.53 seconds, maxrss  516.79 MB
    From worker 17:      * euler                 in   0.90 seconds, maxrss  244.30 MB
    From worker 17:      * lineedit              in   1.98 seconds, maxrss  247.08 MB
    From worker 13:      * file                  in  35.35 seconds, maxrss  406.01 MB
    From worker 17:      * replcompletions       in   2.48 seconds, maxrss  249.98 MB
    From worker 17:      * replutil              in   3.17 seconds, maxrss  255.94 MB
    From worker 13:      * repl                  in   3.61 seconds, maxrss  406.01 MB
    From worker 17:      * sets                  in   2.30 seconds, maxrss  262.80 MB
    From worker 17:      * goto                  in   0.03 seconds, maxrss  262.86 MB
    From worker 17:      * llvmcall              in   0.09 seconds, maxrss  262.88 MB
    From worker 16:      * broadcast             in  19.60 seconds, maxrss  336.16 MB
    From worker 9:       * show                  in  12.23 seconds, maxrss  553.02 MB
    From worker 9:       * meta                  in   0.30 seconds, maxrss  558.68 MB
    From worker 17:      * grisu                 in   1.73 seconds, maxrss  265.91 MB
    From worker 9:       * stacktraces           in   0.82 seconds, maxrss  558.82 MB
    From worker 13:      * test                  in   5.70 seconds, maxrss  413.98 MB
    From worker 17:      * profile               in   2.84 seconds, maxrss  279.74 MB
    From worker 13:      * docs                  in   2.41 seconds, maxrss  425.63 MB
    From worker 13:      * base64                in   0.18 seconds, maxrss  425.65 MB
    From worker 14:      * ranges                in  17.55 seconds, maxrss  572.30 MB
    From worker 17:      * markdown              in   2.93 seconds, maxrss  281.61 MB
Warning: threaded loop executed in order
    From worker 16:      * nullable              in   7.21 seconds, maxrss  342.84 MB
    From worker 16:      * enums                 in   1.03 seconds, maxrss  342.84 MB
    From worker 17:      * threads               in   2.23 seconds, maxrss  292.20 MB
    From worker 17:      * i18n                  in   0.01 seconds, maxrss  292.21 MB
    From worker 13:      * serialize             in   3.79 seconds, maxrss  427.52 MB
    From worker 17:      * workspace             in   0.50 seconds, maxrss  292.21 MB
    From worker 13:      * libdl                 in   0.41 seconds, maxrss  430.66 MB
    From worker 17:      * int                   in   2.16 seconds, maxrss  305.72 MB
    From worker 17:      * intset                in   0.41 seconds, maxrss  307.69 MB
    From worker 9:       * libgit2               in  10.51 seconds, maxrss  597.08 MB
    From worker 2:       * linalg/triangular     in 222.75 seconds, maxrss  672.13 MB
    From worker 2:       * inline                in   0.23 seconds, maxrss  672.24 MB
    From worker 17:      * floatfuncs            in   2.25 seconds, maxrss  311.55 MB
    From worker 17:      * error                 in   0.61 seconds, maxrss  311.57 MB
WARNING: Method definition ambig(Any, Integer) in module Main at /usr/share/julia/test/ambiguous.jl:7 overwritten at /usr/share/julia/test/ambiguous.jl:79.
    From worker 17:      * ambiguous            Skipping Base.<|
    From worker 17: Skipping Base.>:
    From worker 17: Skipping Base.active_repl
    From worker 17: Skipping Base.active_repl_backend
    From worker 17:  in   0.59 seconds, maxrss  312.59 MB
    From worker 17:      * cartesian             in   0.01 seconds, maxrss  312.59 MB
    From worker 2:       * boundscheck           in   1.63 seconds, maxrss  672.25 MB
    From worker 14:      * misc                  in  10.92 seconds, maxrss  637.77 MB
    From worker 13:      * checked               in   8.00 seconds, maxrss  445.83 MB
    From worker 2:       * dsp                   in   7.73 seconds, maxrss  693.61 MB
    From worker 17:      * fft                   in  16.88 seconds, maxrss  375.48 MB
    From worker 12:      * bitarray              in 110.31 seconds, maxrss  516.56 MB
    From worker 9:       * parallel              in  63.55 seconds, maxrss  597.08 MB
    From worker 7:       * subarray              in 235.21 seconds, maxrss  793.24 MB
    From worker 16:      * cmdlineargs           in 101.96 seconds, maxrss  366.08 MB
Exception running test linalg/lapack :
On worker 6:
LoadError: There was an error during testing
 in record at ./test.jl:397
 in do_test at ./test.jl:281
 in macro expansion; at /usr/share/julia/test/linalg/lapack.jl:39 [inlined]
 in anonymous at ./<missing>:?
 in include_string at ./loading.jl:380
 in include_from_node1 at ./loading.jl:429
 in macro expansion at ./util.jl:226 [inlined]
 in runtests at /usr/share/julia/test/testdefs.jl:7
 in JuliaLang/julia#16 at /usr/share/julia/test/runtests.jl:44
 in JuliaLang/julia#625 at ./multi.jl:1421
 in run_work_thunk at ./multi.jl:1001
 in macro expansion at ./multi.jl:1421 [inlined]
 in JuliaLang/julia#624 at ./event.jl:68
while loading /usr/share/julia/test/linalg/lapack.jl, in expression starting on line 33
Exception running test linalg/qr :
On worker 3:
LoadError: There was an error during testing
 in record at ./test.jl:397
 in do_test at ./test.jl:281
 in macro expansion; at /usr/share/julia/test/linalg/qr.jl:38 [inlined]
 in anonymous at ./<missing>:?
 in include_string at ./loading.jl:380
 in include_from_node1 at ./loading.jl:429
 in macro expansion at ./util.jl:226 [inlined]
 in runtests at /usr/share/julia/test/testdefs.jl:7
 in JuliaLang/julia#16 at /usr/share/julia/test/runtests.jl:44
 in JuliaLang/julia#625 at ./multi.jl:1421
 in run_work_thunk at ./multi.jl:1001
 in macro expansion at ./multi.jl:1421 [inlined]
 in JuliaLang/julia#624 at ./event.jl:68
while loading /usr/share/julia/test/linalg/qr.jl, in expression starting on line 23
Exception running test linalg/dense :
On worker 4:
LoadError: There was an error during testing
 in record at ./test.jl:397
 in do_test at ./test.jl:281
 in macro expansion; at /usr/share/julia/test/linalg/dense.jl:483 [inlined]
 in anonymous at ./<missing>:?
 in include_string at ./loading.jl:380
 in include_from_node1 at ./loading.jl:429
 in macro expansion at ./util.jl:226 [inlined]
 in runtests at /usr/share/julia/test/testdefs.jl:7
 in JuliaLang/julia#16 at /usr/share/julia/test/runtests.jl:44
 in JuliaLang/julia#625 at ./multi.jl:1421
 in run_work_thunk at ./multi.jl:1001
 in macro expansion at ./multi.jl:1421 [inlined]
 in JuliaLang/julia#624 at ./event.jl:68
while loading /usr/share/julia/test/linalg/dense.jl, in expression starting on line 477
Exception running test linalg/hessenberg :
On worker 11:
LoadError: There was an error during testing
 in record at ./test.jl:397
 in do_test at ./test.jl:281
 in macro expansion; at /usr/share/julia/test/linalg/hessenberg.jl:33 [inlined]
 in anonymous at ./<missing>:?
 in include_string at ./loading.jl:380
 in include_from_node1 at ./loading.jl:429
 in macro expansion at ./util.jl:226 [inlined]
 in runtests at /usr/share/julia/test/testdefs.jl:7
 in JuliaLang/julia#16 at /usr/share/julia/test/runtests.jl:44
 in JuliaLang/julia#625 at ./multi.jl:1421
 in run_work_thunk at ./multi.jl:1001
 in macro expansion at ./multi.jl:1421 [inlined]
 in JuliaLang/julia#624 at ./event.jl:68
while loading /usr/share/julia/test/linalg/hessenberg.jl, in expression starting on line 8
Exception running test linalg/lq :
On worker 10:
LoadError: There was an error during testing
 in record at ./test.jl:397
 in do_test at ./test.jl:281
 in macro expansion; at /usr/share/julia/test/linalg/lq.jl:66 [inlined]
 in anonymous at ./<missing>:?
 in include_string at ./loading.jl:380
 in include_from_node1 at ./loading.jl:429
 in macro expansion at ./util.jl:226 [inlined]
 in runtests at /usr/share/julia/test/testdefs.jl:7
 in JuliaLang/julia#16 at /usr/share/julia/test/runtests.jl:44
 in JuliaLang/julia#625 at ./multi.jl:1421
 in run_work_thunk at ./multi.jl:1001
 in macro expansion at ./multi.jl:1421 [inlined]
 in JuliaLang/julia#624 at ./event.jl:68
while loading /usr/share/julia/test/linalg/lq.jl, in expression starting on line 24
Exception running test linalg/matmul :
On worker 5:
LoadError: There was an error during testing
 in record at ./test.jl:397
 in do_test at ./test.jl:281
 in macro expansion; at /usr/share/julia/test/linalg/matmul.jl:254 [inlined]
 in anonymous at ./<missing>:?
 in include_string at ./loading.jl:380
 in include_from_node1 at ./loading.jl:429
 in macro expansion at ./util.jl:226 [inlined]
 in runtests at /usr/share/julia/test/testdefs.jl:7
 in JuliaLang/julia#16 at /usr/share/julia/test/runtests.jl:44
 in JuliaLang/julia#625 at ./multi.jl:1421
 in run_work_thunk at ./multi.jl:1001
 in macro expansion at ./multi.jl:1421 [inlined]
 in JuliaLang/julia#624 at ./event.jl:68
while loading /usr/share/julia/test/linalg/matmul.jl, in expression starting on line 248
Exception running test blas :
On worker 15:
LoadError: There was an error during testing
 in record at ./test.jl:397
 in do_test at ./test.jl:281
 in macro expansion; at /usr/share/julia/test/blas.jl:63 [inlined]
 in anonymous at ./<missing>:?
 in include_string at ./loading.jl:380
 in include_from_node1 at ./loading.jl:429
 in macro expansion at ./util.jl:226 [inlined]
 in runtests at /usr/share/julia/test/testdefs.jl:7
 in JuliaLang/julia#16 at /usr/share/julia/test/runtests.jl:44
 in JuliaLang/julia#625 at ./multi.jl:1421
 in run_work_thunk at ./multi.jl:1001
 in macro expansion at ./multi.jl:1421 [inlined]
 in JuliaLang/julia#624 at ./event.jl:68
while loading /usr/share/julia/test/blas.jl, in expression starting on line 38
Exception running test sparse :
On worker 8:
LoadError: LoadError: There was an error during testing
 in record at ./test.jl:397
 in do_test at ./test.jl:281
 in include_string at ./loading.jl:380
 in include_from_node1 at ./loading.jl:429
 in include_string at ./loading.jl:380
 in include_from_node1 at ./loading.jl:429
 in macro expansion at ./util.jl:226 [inlined]
 in runtests at /usr/share/julia/test/testdefs.jl:7
 in JuliaLang/julia#16 at /usr/share/julia/test/runtests.jl:44
 in JuliaLang/julia#625 at ./multi.jl:1421
 in run_work_thunk at ./multi.jl:1001
 in macro expansion at ./multi.jl:1421 [inlined]
 in JuliaLang/julia#624 at ./event.jl:68
while loading /usr/share/julia/test/sparsedir/sparsevector.jl, in expression starting on line 723
while loading /usr/share/julia/test/sparse.jl, in expression starting on line 4
ERROR: LoadError: Some tests exited with errors.
 in (::##11#19)() at /usr/share/julia/test/runtests.jl:72
 in cd(::##11#19, ::String) at ./file.jl:59
 in include_from_node1(::String) at ./loading.jl:426
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /usr/share/julia/test/runtests.jl, in expression starting on line 26

[tkelman]

thanks for the info @ronisbr (logs that long might be a little better in a gist next time though), that is indeed JuliaLang/LinearAlgebra.jl#137

[colbec]

I note that Yast in Leap 42.1 has a package cblas-devel which appears to offer headers for cblas. It's not installed by default.

[tkelman]

We don't need the headers (MXNet.jl might), but we are currently assuming that cblas symbols for cblas_cdotc_sub and a few others are built in the same shared library as the conventional Fortran BLAS API which isn't the case occasionally. We should avoid the reference BLAS due to its poor performance anyway, but there might be higher-performing BLAS implementations in the future that don't put cblas symbols in the same library so JuliaLang/LinearAlgebra.jl#137 is still worth fixing sooner or later.

[ronisbr]

Hi @tkelman!

Julia 0.5.0~rc4 was accepted and now Tumbleweed users after the update will use julia with libopenblas_openmp0. Thanks for the advices 👍