TensorFlow library not loading with GraalVM Native Image #67
Labels
bug
Something isn't working
Comments
|
Thanks for reporting the issue @murphye |
|
@murphye I noticed static initialization block is not executed in GraalVM Native Image. Both DJL and JavaCPP rely on static initialization code. I'm not familiar with GraalVM, would you please help on:
|
|
I tried to add |
|
@frankfliu What classes (with packages) contain the static initialization block(s)? Can we offer alternative that doesn't do this?
|
|
@stu1130 Can you turn on debug output to get additional error output? Whatever the equivalent is for Maven |
Update: Native debug support will be in the next version of GraalVM CE |
|
In searching through JavaCPP repo, here are static initialization blocks. For my error message, it stems from Only Calling |
|
@MarkAtwood I add and |
|
@stu1130 Unfortunately your error message isn't very helpful. I opened a GitHub issue on that matter as it may be related to the Quarkus Gradle plugin: quarkusio/quarkus#9306 Can you put your current version of the code/config in GitHub so I can link to the exact commit version? That would be helpful. |
|
@stu1130 @frankfliu I believe I have enough information to play around with I believe the end goal should be to create a Quarkus extension for DJL that would provide a preconfigured DJL config for the developer. |
|
Here is one more class using static initializer, it comes from TF-java dependencies: |
|
Here is the format you need to follow: This is painful for the moment, but a better way can potentially be implemented for a Quarkus extension to handle this. For now, let's go with |
|
@murphye Let us know if you encounter other issues when using |
Lokiiiiii
pushed a commit
to Lokiiiiii/djl
that referenced
this issue
Oct 10, 2023
* [serving] Refactor ModelDefinition class * [serving] Add async model loading on startup * [serving] Start HTTP listener while model is loading
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
I am working on an implementation of the covid19-detection example code, but using Quarkus to serve the model and also support GraalVM Native Image.
The project is located here:
https://github.com/murphye/djl-demo/tree/master/covid19-detection-quarkus
The application runs fine on the JVM, but when running in native mode, the TensorFlow libraries are not being loaded (i.e.
System.loadLibrary).Important: I cannot find a reference in the DJL code for
System.loadLibraryand I do not understand how the TensorFlow libraries are actually loaded. If I better understood how the mechanism worked, I could better diagnose it. It does seem to be related to Bytedeco which I am not familiar with.Here is the code that I am running to demonstrate the issue:
Error Message
Here is the output of the error when running this code. The TF libraries are downloaded and placed in
/Users/ermurphy/.tensorflow/cache/2.1.0-a-SNAPSHOT-cpu-osx-x86_64correctly.Expected Behavior
Running in GraalVM Native Image executable, the libaries should be loaded and usable through JNI bridge. I have proven this in the past with this PoC:
https://github.com/murphye/quarkus-tensorflow-inception/blob/master/src/main/java/io/quarkus/tensorflow/LoadTensorFlow.java
Next Steps
I need some guidance on how TensorFlow is loaded in DJL if it's not using
System.loadLibraryas shown here:https://github.com/murphye/quarkus-tensorflow-inception/blob/master/src/main/java/io/quarkus/tensorflow/LoadTensorFlow.java#L98
How else does the TensorFlow library get loaded, and how can I further diagnose the issue when running in Native mode?
The text was updated successfully, but these errors were encountered: