AArch64: Fails to build when OpenSSL is enabled #5018

knn-k · 2019-03-07T08:51:00Z

Build of AArch64 runtime fails if you specify --with-openssl=fetched in running configure.
The build environment of AArch64 runtime expects cross-compilation on x86 at this point. The build process compiles libcrypto.so targeting x86 instead of AArch64.

The text was updated successfully, but these errors were encountered:

pshipton · 2019-03-07T16:08:42Z

This is really an issue with the extensions, like https://github.com/ibmruntimes/openj9-openjdk-jdk8 and https://github.com/ibmruntimes/openj9-openjdk-jdk11

@andrew-m-leonard @groeges

groeges · 2019-03-08T08:32:10Z

@knn-k What version of OpenJDK were you trying to build? jdk11 or jdk12?
And are you saying that without the --with-openssl=fetched the AArch64 build works?

knn-k · 2019-03-08T08:47:53Z

@groeges It is JDK11.
You can build and run the AArch64 Linux runtime if you don't use the --with-openssl=... option.
I have not tried --with-openssl=system nor --with-openssl=path_to_library.
The build instruction is in PR #4696 (not merged yet).

groeges · 2019-03-08T09:42:37Z

There is a change going in to only build OpenSSL if we are bundling the OpenSSL library with the JDK.
Are we expecting to bundle OpenSSL with the AArch64 Linux runtime, or will OpenSSL be available on the system? If we are not bundling then this change may resolve the issue, but we should still try and fix if we in case someone builds it and tries to bundle the OpenSSL library.

Tried to follow your updated instructions but got this error when running the docker build command

ADD failed: failed to GET http://ftp.us.debian.org/debian/pool/main/c/cups/libcups2-dev_2.2.1-8%2bdeb9u2_arm64.deb with status 404 Not Found: <html>
<head><title>404 Not Found</title></head>

Is this a know issue?

groeges · 2019-03-08T09:48:00Z

Updated the Dockerfile to use

    http://ftp.us.debian.org/debian/pool/main/c/cups/libcups2-dev_2.2.1-8%2bdeb9u3_arm64.deb         \
    http://ftp.us.debian.org/debian/pool/main/c/cups/libcupsimage2-dev_2.2.1-8%2bdeb9u3_arm64.deb    \

as the deb9u2 versions didn't seem to exist. Docker image now gets built correctly.

knn-k · 2019-03-08T10:00:05Z

I am trying to build using --with-openssl=path_to_library now.

I guess we don't need to bundle the OpenSSL library with the AArch64 runtime.

knn-k · 2019-03-08T10:45:58Z

The AArch64 Linux runtime was built successfully in the cross-compiling environment using --with-openssl=path_to_library. I see libjncrypto.so in the JDK image, and it links libcrypto.so.

What would be the easiest way to see if it works or not?

groeges · 2019-03-08T10:53:58Z

You could try compiling this small java test:

import java.util.Random;
import java.security.spec.AlgorithmParameterSpec;

import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;

public class CryptoTest {
        public static void main(String[] args) {
                try {
            Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
            Random r = new Random(10);
            byte[] skey_bytes = new byte[16];
            r.nextBytes(skey_bytes);
            SecretKeySpec skey = new SecretKeySpec(skey_bytes, "AES");
            cipher.init(Cipher.ENCRYPT_MODE, skey);
                        System.out.println("Crypto test COMPLETED");
                } catch (Exception e) {
                        System.out.println("Crypto test FAILED");
                        e.printStackTrace();
                }
        }
}

and then run with

java -Djdk.nativeCryptoTrace=true CryptoTest

which should give the following output which states that the native crypto library was loaded

MessageDigest load - using Native crypto library.
CipherCore Load - using native crypto library.
Crypto test COMPLETED

groeges · 2019-03-08T10:58:40Z

Tried adding code to the openssl.gmk to use the correct OpenSSL configure options to compile with linux-aarch64 but that causes the build of OpenSSL to fail with an error:

gcc  -I. -Icrypto/include -Iinclude -Icrypto -fPIC -pthread -Wa,--noexecstack -Wall -O3 -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1.1\"" -DNDEBUG  -c -o crypto/aes/aesv8-armx.o crypto/aes/aesv8-armx.S
In file included from crypto/aes/aesv8-armx.S:1:0:
crypto/arm_arch.h:55:6: error: #error "unsupported ARM architecture"
 #    error "unsupported ARM architecture"
      ^~~~~
Makefile:857: recipe for target 'crypto/aes/aesv8-armx.o' failed

It seems __ARM_ARCH is not defined from gcc and hence causes the failure.

knn-k · 2019-03-11T01:50:46Z

My build with --with-openssl=path_to_library generates the following output:

$ jdk/bin/java -Xint -Djdk.nativeCryptoTrace=true CryptoTest
MessageDigest load - Native crypto library disabled.
Warning: Native crypto library load failed. Using Java crypto implementation
Crypto test COMPLETED

It is the same as the output from a build without the --with-openssl= option.
I haven't investigated the cause yet.

groeges · 2019-03-11T09:17:45Z

If you are not bundling the libcrypto library then that message means that it could not find the library on the system. Do you know if the system has OpenSSL installed and where the libcrypto library is?

knn-k · 2019-03-18T09:30:32Z

I have been busy working on other tasks.
Today I caught up with the latest openj9-openjdk-jdk11 extension, built the aarch64 runtime with --with-openssl=path_to_library, and found the output from the test is different from that of last week:

$ jdk/bin/java -Xint -Djdk.nativeCryptoTrace=true CryptoTest
Warning: Native crypto library load failed. Using Java crypto implementation
Warning: Native crypto library load failed. Using Java crypto implementation
Crypto test COMPLETED

libjncrypto.so in my build used to refer libcrypto.so.1.1 last week, but it doesn't in today's build:

$ readelf -d jdk/lib/libjncrypto.so

Dynamic section at offset 0x5c58 contains 29 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libdl.so.2]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x0000000000000001 (NEEDED) Shared library: [ld-linux-aarch64.so.1]
0x000000000000000e (SONAME) Library soname: [libjncrypto.so]
0x000000000000000f (RPATH) Library rpath: [$ORIGIN]

groeges · 2019-03-19T14:43:11Z

With the latest code the libcrypto.so is not statically loaded but is now dynamically loaded so that is probably why the readily command doesn't show the libcrypto.so file. As it is dynamically loads the library, libcrypto.so needs to be available on the libpath on the system, or bundled as part of the jdk.

knn-k · 2019-03-20T01:38:20Z

Thank you for the information.
I found libcrypto.so.1.1 was not placed in the right location, and I see the following error after fixing it, using the same JDK build:

$ jdk/bin/java -Xint -Djdk.nativeCryptoTrace=true CryptoTest
Inconsistency detected by ld.so: dl-open.c: 689: _dl_open: Assertion `_dl_debug_initialize (0, args.nsid)->r_state == RT_CONSISTENT' failed!

I am going to investigate it.

knn-k · 2019-03-22T02:36:55Z

It turned out that there was mismatch between libcrypto.so.1.1 I used and the OS version.
I switched to recent version of Debian aarch64, and my JDK build with --with-openssl=path_to_library worked as expected:

$ jdk/bin/java -Xint -Djdk.nativeCryptoTrace=true CryptoTest
MessageDigest load - using Native crypto library.
CipherCore Load - using native crypto library.
Crypto test COMPLETED

I will add comment to the building instructions on building with OpenSSL.

This commit adds comments for AArch64 Linux build with OpenSSL to the Build_Instructions_V11. Fixes: eclipse-openj9#5018 Signed-off-by: knn-k <konno@jp.ibm.com>

groeges · 2019-03-22T09:53:32Z

Excellent. Glad this now all works correctly.

sxa · 2019-04-08T16:27:25Z

@knn-k I've had it working ok with --with-openssl=system on one of my aarch64 machines so that options should be good too.

knn-k · 2019-04-09T02:02:23Z

@sxa555
Yes, I also confirmed that --with-openssl=system works as expected with native build of the runtime on AArch64 Linux. #5394 adds instructions for native build, and mentions --with-openssl=system.

knn-k mentioned this issue Mar 7, 2019

AArch64: Update build instructions #4696

Merged

pshipton added the userRaised label Mar 7, 2019

knn-k added a commit to knn-k/openj9 that referenced this issue Mar 22, 2019

AArch64: Add comments for building with OpenSSL

93d08cf

This commit adds comments for AArch64 Linux build with OpenSSL to the Build_Instructions_V11. Fixes: eclipse-openj9#5018 Signed-off-by: knn-k <konno@jp.ibm.com>

knn-k mentioned this issue Mar 22, 2019

AArch64: Add comments for building with OpenSSL #5206

Merged

0xdaryl closed this as completed in #5206 Mar 23, 2019

0xdaryl added the arch:aarch64 label Jun 27, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AArch64: Fails to build when OpenSSL is enabled #5018

AArch64: Fails to build when OpenSSL is enabled #5018

knn-k commented Mar 7, 2019

pshipton commented Mar 7, 2019

groeges commented Mar 8, 2019 •

edited

Loading

knn-k commented Mar 8, 2019 •

edited

Loading

groeges commented Mar 8, 2019

groeges commented Mar 8, 2019

knn-k commented Mar 8, 2019

knn-k commented Mar 8, 2019

groeges commented Mar 8, 2019

groeges commented Mar 8, 2019 •

edited

Loading

knn-k commented Mar 11, 2019

groeges commented Mar 11, 2019 •

edited

Loading

knn-k commented Mar 18, 2019

groeges commented Mar 19, 2019 •

edited

Loading

knn-k commented Mar 20, 2019 •

edited

Loading

knn-k commented Mar 22, 2019

groeges commented Mar 22, 2019

sxa commented Apr 8, 2019

knn-k commented Apr 9, 2019

AArch64: Fails to build when OpenSSL is enabled #5018

AArch64: Fails to build when OpenSSL is enabled #5018

Comments

knn-k commented Mar 7, 2019

pshipton commented Mar 7, 2019

groeges commented Mar 8, 2019 • edited Loading

knn-k commented Mar 8, 2019 • edited Loading

groeges commented Mar 8, 2019

groeges commented Mar 8, 2019

knn-k commented Mar 8, 2019

knn-k commented Mar 8, 2019

groeges commented Mar 8, 2019

groeges commented Mar 8, 2019 • edited Loading

knn-k commented Mar 11, 2019

groeges commented Mar 11, 2019 • edited Loading

knn-k commented Mar 18, 2019

groeges commented Mar 19, 2019 • edited Loading

knn-k commented Mar 20, 2019 • edited Loading

knn-k commented Mar 22, 2019

groeges commented Mar 22, 2019

sxa commented Apr 8, 2019

knn-k commented Apr 9, 2019

groeges commented Mar 8, 2019 •

edited

Loading

knn-k commented Mar 8, 2019 •

edited

Loading

groeges commented Mar 8, 2019 •

edited

Loading

groeges commented Mar 11, 2019 •

edited

Loading

groeges commented Mar 19, 2019 •

edited

Loading

knn-k commented Mar 20, 2019 •

edited

Loading