Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native Compilation #99

Merged
merged 1 commit into from
Oct 9, 2024
Merged

Native Compilation #99

merged 1 commit into from
Oct 9, 2024

Conversation

melloware
Copy link
Contributor

@melloware melloware commented Oct 7, 2024

Fix #94
Fix #22

OK made a ton of progress to the point where I could load the native image and run http://localhost:8080/jasperreports/csv and get an error.

A few things.

  1. In Native mode you can't compile JRXML because there is not Java available "no Javac found to compile JRXML". However if you add Java to the native image you still get errors because of the dynamic nature of Compilation so you can't use JRXML in native you have to use .jasper compiled files which is Jasper's recommendation anyway.

  2. I found Camel Xalan support and added it. I don't love adding this but they already have Xalan Native stuff in Camel. https://github.com/apache/camel-quarkus/blob/main/extensions-support/xalan/deployment/src/main/java/org/apache/camel/quarkus/support/xalan/deployment/XalanNativeImageProcessor.java

  3. Now the real problem when loading a .jasper compiled file is this...

2024-10-07 19:53:02,781 DEBUG [net.sf.jas.eng.uti.JRClassLoader] (executor-thread-1) loadClass: CustomersReport_9ade706fa183f7d5d529332635ecd19b367b8e715502ba8d758a8729c8ce4fa7
2024-10-07 19:53:02,781 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-1) HTTP Request to /jasperreports/csv failed, error id: 6b2a6c6b-b319-41fe-81d4-10b673eb051c-1: com.oracle.svm.core.jdk.UnsupportedFeatureError: No classes have been predefined during the image build to load from bytecodes at runtime.
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.util.VMError.unsupportedFeature(VMError.java:121)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.hub.PredefinedClassesSupport.throwNoBytecodeClasses(PredefinedClassesSupport.java:76)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.hub.PredefinedClassesSupport.loadClass(PredefinedClassesSupport.java:130)
        at java.base@21.0.4/java.lang.ClassLoader.defineClass(ClassLoader.java:280)
        at net.sf.jasperreports.engine.util.JRClassLoader.loadClass(JRClassLoader.java:349)
        at net.sf.jasperreports.engine.util.CompiledClassesLoader.loadCompiledClass(CompiledClassesLoader.java:87)
        at net.sf.jasperreports.engine.util.JRClassLoader.loadClassFromBytes(JRClassLoader.java:290)
        at net.sf.jasperreports.engine.design.JRAbstractJavaCompiler.loadClass(JRAbstractJavaCompiler.java:168)
        at net.sf.jasperreports.engine.design.JRAbstractJavaCompiler.loadEvaluator(JRAbstractJavaCompiler.java:117)
        at net.sf.jasperreports.engine.design.JRAbstractCompiler.createEvaluator(JRAbstractCompiler.java:459)
        at net.sf.jasperreports.engine.design.JRAbstractCompiler.loadEvaluator(JRAbstractCompiler.java:427)
        at net.sf.jasperreports.engine.JasperCompileManager.getEvaluator(JasperCompileManager.java:374)
        at net.sf.jasperreports.engine.fill.JRFillDataset.createCalculator(JRFillDataset.java:526)
        at net.sf.jasperreports.engine.fill.BaseReportFiller.<init>(BaseReportFiller.java:170)
        at net.sf.jasperreports.engine.fill.JRBaseFiller.<init>(JRBaseFiller.java:273)
        at net.sf.jasperreports.engine.fill.JRVerticalFiller.<init>(JRVerticalFiller.java:82)
        at net.sf.jasperreports.engine.fill.JRFiller.createBandReportFiller(JRFiller.java:252)
        at net.sf.jasperreports.engine.fill.JRFiller.createReportFiller(JRFiller.java:272)
        at net.sf.jasperreports.engine.fill.JRFiller.fill(JRFiller.java:211)
        at io.quarkiverse.jasperreports.it.JasperReportsResource.fill(JasperReportsResource.java:130)

Jasper uses the classloader to load the .jasper file which is ByteCode which is NOT allowed in GraalVM com.oracle.svm.core.jdk.UnsupportedFeatureError: No classes have been predefined during the image build to load from bytecodes at runtime.. @gastaldi i don't know if there is a clever way to tell GraalVM to load the .jasper files?

@nderwin
Copy link
Contributor

nderwin commented Oct 7, 2024

@melloware Naive question: would ClassLoader.getResourceAsStream(...) work in native mode? If so, maybe try writing a net.sf.jasperreports.repo.StreamRepositoryService as a wrapper around that for loading the .jasper files?

@melloware
Copy link
Contributor Author

I am not a Jasper expert but this is how you load a .jasper and their underlying code does some magic...

 mainReport = (JasperReport) JRLoader.loadObject(JRLoader.getLocationInputStream(TEST_REPORT_NAME + ".jasper"));

Which does this...

public static InputStream getLocationInputStream(String location) throws JRException//FIXME deprecate this?
	{
		InputStream is = null;
		
		is = getResourceInputStream(location);
		
		if (is == null)
		{
			is = getFileInputStream(location);
		}
		
		if (is == null)
		{
			is = getURLInputStream(location);
		}
		
		return is;
	}

So that loads the bytes from getResourceAsStream already.

@nderwin
Copy link
Contributor

nderwin commented Oct 7, 2024

I wonder if the work I'm doing for #72 would help - by compiling the .jrxml files into .jasper files and adding them as generated resources; just spit-balling here, I would also not consider myself a Jasper expert.

@melloware
Copy link
Contributor Author

@nderwin yep i think we have to find a way to add all .jasper compiled into the native image somehow. What makes it tough is the compiled JASPER files have these weird class names like CustomersReport_9ade706fa183f7d5d529332635ecd19b367b8e715502ba8d758a8729c8ce4fa7 and then we have to figure out a way to load it without using the loadBytes method as that is what the GraalVM has blocked as invalid.

@gastaldi
Copy link
Member

gastaldi commented Oct 7, 2024

Back in my days I used a Maven plugin to compile the .jrxml to a .jasper file (the WAR file only contained the binary).

I believe that's the same approach here, we need a build step that takes a .jrxml file and outputs the binary compilation that is included as a native image resource. I don't see a use case requiring native images to support ad-hoc compilation of reports.

@gastaldi
Copy link
Member

gastaldi commented Oct 8, 2024

2. I found Camel Xalan support and added it. I don't love adding this but they already have Xalan Native stuff in Camel.

Are you sure it doesn't bring any camel specific classes? Perhaps copying this processor to this project would be a better choice

@melloware
Copy link
Contributor Author

I wasn't sure but that is on my list to copy it. That was my gut feeling too to copy it.

@gastaldi
Copy link
Member

gastaldi commented Oct 8, 2024

I wasn't sure but that is on my list to copy it. That was my gut feeling too to copy it.

Great minds think alike! 🤝

@melloware
Copy link
Contributor Author

From what I am reading to load Jasper files at runtime in native mode with load bytes we have to do this: https://www.graalvm.org/latest/reference-manual/native-image/metadata/#defining-classes-at-run-time

Basically somehow generate this hash config at build time and include it in the deployment. Then when classes are loaded with those identical bytes then GraalVM swaps them in...I think.

@gastaldi
Copy link
Member

gastaldi commented Oct 8, 2024

Just a hunch, but shouldn't a GeneratedClassBuildItem handle that if it's a Java class?

@melloware
Copy link
Contributor Author

OK @gastaldi i think i might need to ask the Native experts on Zulip. Here is what I found out....

  1. The class not being loaded is actually inside the JasperReport itself it looks like it stores subclasses.

image

  1. That is OK so I go inside and I grab the classname and its bytes and register it with GeneratedClassBuildItem
try {
                String jasperFilePath = file.toFile().getAbsolutePath();
                JasperReport report = (JasperReport) JRLoader.loadObject(JRLoader.getLocationInputStream(jasperFilePath));
                JRReportCompileData reportData = (JRReportCompileData) report.getCompileData();
                ReportExpressionEvaluationData mainData = (ReportExpressionEvaluationData) reportData
                        .getMainDatasetCompileData();
                String reportDataClass = mainData.getCompileName();
                if (StringUtils.isNotBlank(reportDataClass)) {
                    byte[] bytes = (byte[]) mainData.getCompileData();
                    Log.infof("Report Data Class: %s Size: %d", reportDataClass, bytes.length);
                    additionalClasses.produce(new GeneratedClassBuildItem(false, reportDataClass, bytes));
                }
            } catch (JRException e) {
                Log.error("Error loading report file class.", e);
            }
  1. I can tell its loaded properly because I generate this log line.
JasperReportsProcessor$1] C:\dev\quarkus\quarkus-jasperreports\integration-tests\src\main\resources\CustomersReport.jasper
JasperReportsProcessor$1] C:\dev\quarkus\quarkus-jasperreports\integration-tests\src\main\resources\OrdersReport.jasper
JasperReportsProcessor] Report Data Class: CustomersReport_9ade706fa183f7d5d529332635ecd19b367b8e715502ba8d758a8729c8ce4fa7 Size: 2056

So I loaded the right class and bytes in Native mode.

  1. When I run the built Native image the stack trace is the same even though that class was loaded.
__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2024-10-08 13:39:05,012 INFO  [io.quarkus] (main) JasperReports IT 999-SNAPSHOT native (powered by Quarkus 3.15.1) started in 0.017s. Listening on: http://0.0.0.0:8080
2024-10-08 13:39:05,012 INFO  [io.quarkus] (main) Profile prod activated.
2024-10-08 13:39:05,012 INFO  [io.quarkus] (main) Installed features: [awt, cdi, jasperreports, poi, rest, smallrye-context-propagation, smallrye-openapi, vertx]
2024-10-08 13:39:10,243 DEBUG [net.sf.jas.eng.fil.BaseReportFiller] (executor-thread-1) Fill 1: created for CustomersReport
2024-10-08 13:39:10,244 DEBUG [net.sf.jas.eng.uti.CompiledClassesLoader] (executor-thread-1) loading compiled class CustomersReport_9ade706fa183f7d5d529332635ecd19b367b8e715502ba8d758a8729c8ce4fa7
2024-10-08 13:39:10,244 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-1) HTTP Request to /jasperreports/csv failed, error id: eb7659a2-e8bd-47bd-a620-a9a3b0377637-1: com.oracle.svm.core.jdk.UnsupportedFeatureError: No classes have been predefined during the image build to load from bytecodes at runtime.
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.util.VMError.unsupportedFeature(VMError.java:121)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.hub.PredefinedClassesSupport.throwNoBytecodeClasses(PredefinedClassesSupport.java:76)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.hub.PredefinedClassesSupport.loadClass(PredefinedClassesSupport.java:130)
        at java.base@21.0.4/java.lang.ClassLoader.defineClass(ClassLoader.java:280)
        at net.sf.jasperreports.engine.util.JRClassLoader.loadClass(JRClassLoader.java:341)
        at net.sf.jasperreports.engine.util.CompiledClassesLoader.loadCompiledClass(CompiledClassesLoader.java:87)
        at net.sf.jasperreports.engine.util.JRClassLoader.loadClassFromBytes(JRClassLoader.java:286)
        at net.sf.jasperreports.engine.design.JRAbstractJavaCompiler.loadClass(JRAbstractJavaCompiler.java:168)
        at net.sf.jasperreports.engine.design.JRAbstractJavaCompiler.loadEvaluator(JRAbstractJavaCompiler.java:117)
        at net.sf.jasperreports.engine.design.JRAbstractCompiler.createEvaluator(JRAbstractCompiler.java:459)
        at net.sf.jasperreports.engine.design.JRAbstractCompiler.loadEvaluator(JRAbstractCompiler.java:427)
        at net.sf.jasperreports.engine.JasperCompileManager.getEvaluator(JasperCompileManager.java:374)
        at net.sf.jasperreports.engine.fill.JRFillDataset.createCalculator(JRFillDataset.java:526)
        at net.sf.jasperreports.engine.fill.BaseReportFiller.<init>(BaseReportFiller.java:170)
        at net.sf.jasperreports.engine.fill.JRBaseFiller.<init>(JRBaseFiller.java:273)
        at net.sf.jasperreports.engine.fill.JRVerticalFiller.<init>(JRVerticalFiller.java:82)
        at net.sf.jasperreports.engine.fill.JRFiller.createBandReportFiller(JRFiller.java:252)
        at net.sf.jasperreports.engine.fill.JRFiller.createReportFiller(JRFiller.java:272)
        at net.sf.jasperreports.engine.fill.JRFiller.fill(JRFiller.java:211)
        at io.quarkiverse.jasperreports.it.JasperReportsResource.fill(JasperReportsResource.java:130)
        at io.quarkiverse.jasperreports.it.JasperReportsResource.csv(JasperReportsResource.java:168)
        at io.quarkiverse.jasperreports.it.JasperReportsResource_ClientProxy.csv(Unknown Source)
        at io.quarkiverse.jasperreports.it.JasperReportsResource$quarkusrestinvoker$csv_fcf335e3100284bd0c0f8a8d7925231362b8ffa2.invoke(Unknown Source)
        at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
        at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:141)
        at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147)
        at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:635)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
        at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base@21.0.4/java.lang.Thread.runWith(Thread.java:1596)
        at java.base@21.0.4/java.lang.Thread.run(Thread.java:1583)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:896)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:872)

2024-10-08 13:43:48,930 INFO  [io.quarkus] (Shutdown thread) JasperReports IT stopped in 0.003s

@gastaldi
Copy link
Member

gastaldi commented Oct 8, 2024

@zakkak any hints here?

@melloware
Copy link
Contributor Author

I made progress here. Stuck on the next issue but got a lot further.

@melloware melloware force-pushed the native-work branch 2 times, most recently from b1f9183 to eb89d88 Compare October 8, 2024 20:02
@nderwin
Copy link
Contributor

nderwin commented Oct 8, 2024

@melloware Are you seeing .jasper files being generated in the target directory?

@melloware
Copy link
Contributor Author

Not yet you are still working on that. I am loading the Jasper files I stored in sec main resources right now for my testing.

I think I will have this all working tomorrow!

@zakkak
Copy link

zakkak commented Oct 8, 2024

@zakkak any hints here?

I need to have a better look (and possibly reproduce the issue) to give better feedback but so far I can say the following:

  1. GeneratedClassBuildItem will generate a .class file in the jar file. So I guess it could be leveraged to load the bytecode from the classpath, but it won't help with JRClassLoader.loadClassFromBytes or similar approaches. So if you go that way you will probablt need to do some substitutions to load the class from the classpath instead.

  2. Basically somehow generate this hash config at build time and include it in the deployment. Then when classes are loaded with those identical bytes then GraalVM swaps them in...I think.

    Right, that's my understanding as well. Are these .jasper files different per build or are they always the same? In the second case you can use the native-image agent to generate this configuration for you and then added it to the META-INF folder of the extension.

    You can have a look at https://quarkus.io/guides/native-reference#native-image-agent-integration for how to achieve this without manually running the agent (not sure what will be easier in this case).

@melloware
Copy link
Contributor Author

I got it working @zakkak i look through all the jasper file and pull out the bytecode and load it with Genrated BuildIten. Then I GraalVM substituted the JRClassloader the method that takes bytes and load the class forName since I know I have already preloaded it. So far it's all working.

@zakkak
Copy link

zakkak commented Oct 8, 2024

Great work @melloware!

@melloware melloware marked this pull request as ready for review October 9, 2024 14:51
@melloware melloware requested a review from a team as a code owner October 9, 2024 14:51
@melloware
Copy link
Contributor Author

@gastaldi Holy sh$%t i got it all working and tests passing in Native Mode!

@melloware melloware changed the title Native work in progress Native Compilation Oct 9, 2024
@melloware
Copy link
Contributor Author

I am going to commit this so i can continue working on new PR's with this baseline since it all works. @gastaldi we can always refactor later.

@melloware melloware merged commit a859eca into main Oct 9, 2024
1 check passed
@melloware melloware deleted the native-work branch October 9, 2024 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Native Image Investigation GraalVM compatibility
4 participants