The generated runnable jar is not a fat jar (dependencies not included)

[salmonb]

Thanks for making this plugin.

I'm submitting a…

• bug report
• feature request
• other

Short description of the issue/suggestion:

The generated application crashes. After investigation, it's because the runnable start generated by the plugin is not a fat jar (standalone application), it doesn't include the dependencies required by the application.

Steps to reproduce the issue/enhancement:

1. Create an application with a dependency
2. Run the plugin
3. Try to run the generated runnable jar

What is the expected behavior?

The generated runnable jar should run as a standalone application.
It should be a fat jar with all dependencies required by the application included.

What is the current behavior?

The runnable jar crashes with a java.lang.ClassNotFoundException because it doesn't find the class required in the dependency.

Do you have outputs, screenshots, demos or samples which demonstrate the problem or enhancement?

It's a private project... If my description is not good enough, please let me know and I will try to create a simple reproducer.

What is the motivation / use case for changing the behavior?

Just make it work :)

Please tell us about your environment:

• JavaPackager version: 1.6.6
• OS version: macOS, Linux, Windows (I tested on different machines, same problem)
• JDK version: 18
• Build tool:
  • Maven
  • Gradle

Other information (e.g. related issues, suggestions how to fix, links for us to have context)

Thank you!

[fvarrui]

Hi @salmonb!

JavaPackager doesn't generate FAT JARs ... only a runnable JAR with your code. You can set copyDependencies property to true, and all libraries will be copied to your app folder. And if you want a FAT JAR, you have to create it by your own (e.g. using another Maven plugin) and set runnableJar property with your FAT JAR path and copyDependencies=false.

Here you can find an example.

I hope it helps!

[salmonb]

Hi @fvarrui

Thanks for your quick reply.

I'm happy with a thin jar and copied dependencies, but it doesn't work in my case.

copyDependencies was already true (as it is true by default), and I can see indeed that the dependencies have been copied inside the app, but even with the thin jar and those dependencies copied, I'm still getting the java.lang.ClassNotFoundException when starting the installed app.

If I generate a fat jar myself and give it to javapackager through the runnableJar parameter, the installed application starts as expected. But I would prefer to keep it simple and make the standard javapackager way working.

I finally made a reproducer here: https://github.com/salmonb/javapackager-issue
You can have a look at the README to reproduce the problem.

Thanks

[salmonb]

I forgot to tell (and I realise now that it's probably the cause of the problem) that my application is a Java >= 9 app, so it uses the Java module system.

It looks like the runnable Jar generated by javapackager is not compatible with the module system.

A quick search and I found that the MANIFEST file should have a section like this for Java 9 apps:

• META-INF
  • versions
    • 9
      • module-info.class

So the solution is probably to add this section when javapackager is called in a module where there is a module-info.class.

[fvarrui]

I forgot to tell (and I realise now that it's probably the cause of the problem) that my application is a Java >= 9 app, so it uses the Java module system.

It looks like the runnable Jar generated by javapackager is not compatible with the module system.

JavaPackager runs apps in legacy mode (non modular way) ... this should not be a problem, as if you run a Java app using "classpath" instead of "modulepath", all module-infos are ignored at runtime by the JVM (tlhis is for backward compatibility).

ASAP, I'll have a look into your project (https://github.com/salmonb/javapackager-issue) ... but I haven't seen where you specify the main class ???

A quick search and I found that the MANIFEST file should have a section like this for Java 9 apps:

• META-INF

  • versions

    • 9

      • module-info.class

So the solution is probably to add this section when javapackager is called in a module where there is a module-info.class.

Sorry, I didn't understand this point.

[salmonb]

I was referring to https://stackoverflow.com/questions/43606502/can-i-require-java-9-module-via-manifest-mf (my quick search) but maybe that's not the point...

My main class is actually defined in the parent pom (which is on a Maven repository and not directly in the repository). When I call your plugin in the xxx-openjfx module, it takes the default configuration which is defined in that parent pom.

When I try to run the generated runnableJar beside the libs folder, it should work, isn't it? But I'm still getting the java.lang.ClassNotFoundException even though the libs folder contains the jar where the main class is contained. That's why I was thinking about a problem with the MANIFEST maybe. But I let you investigate further. Please let me know if you need more info.

[fvarrui]

Ok, this is the META-INF/MANIFEST.MF file in webfx-example-0.1.0-SNAPSHOT-runnable.jar, the JAR embedded in the EXE file:

Manifest-Version: 1.0
Created-By: Apache Maven 3.8.4
Built-By: fvarrui
Build-Jdk: 17.0.1
Class-Path: libs/webfx-example-application-0.1.0-SNAPSHOT.jar libs/webfx
 -kit-openjfx-0.1.0-20220525.141711-23.jar libs/javafx-base-18.0.1.jar l
 ibs/javafx-base-18.0.1-win.jar libs/javafx-controls-18.0.1.jar libs/jav
 afx-controls-18.0.1-win.jar libs/javafx-graphics-18.0.1.jar libs/javafx
 -graphics-18.0.1-win.jar libs/webfx-kit-javafxgraphics-peers-0.1.0-2022
 0525.141711-53.jar libs/webfx-kit-javafxgraphics-peers-base-0.1.0-20220
 525.141711-53.jar libs/webfx-kit-util-0.1.0-20220525.141711-53.jar libs
 /webfx-kit-launcher-0.1.0-20220525.141711-53.jar libs/webfx-platform-cl
 ient-uischeduler-0.1.0-20220426.222906-52.jar libs/webfx-platform-share
 d-util-0.1.0-20220426.222906-54.jar libs/webfx-platform-java-boot-impl-
 0.1.0-20220426.222906-22.jar libs/webfx-platform-shared-boot-0.1.0-2022
 0426.222906-22.jar libs/webfx-platform-java-scheduler-impl-0.1.0-202204
 26.222906-52.jar libs/webfx-platform-shared-log-0.1.0-20220426.222906-5
 4.jar libs/webfx-platform-shared-scheduler-0.1.0-20220426.222906-53.jar
  libs/webfx-platform-java-shutdown-impl-0.1.0-20220426.222906-52.jar li
 bs/webfx-platform-shared-shutdown-0.1.0-20220426.222906-54.jar libs/web
 fx-platform-shared-log-impl-simple-0.1.0-20220426.222906-53.jar
Main-Class: dev.webfx.platform.shared.services.boot.ApplicationBooter

And this is the content of this JAR:

Main class dev.webfx.platform.shared.services.boot.ApplicationBooter doesn't exis inside this JAR (as it's inside webfx-platform-shared-boot-0.1.0-20220426.222906-22.jar), so, this shouldn't be the runnable JAR needed by JavaPackager.

[fvarrui]

What if you add a require statement for the module webfx.platform.shared.boot in webfx-example-application-openjfx's module-info:

// File managed by WebFX (DO NOT EDIT MANUALLY)

module webfx.example.application.openjfx {

    // Direct dependencies modules
    requires webfx.example.application;
    requires webfx.kit.openjfx;
    requires webfx.platform.java.boot.impl;
    requires webfx.platform.java.scheduler.impl;
    requires webfx.platform.java.shutdown.impl;
    requires webfx.platform.shared.log.impl.simple;
    requires webfx.platform.shared.boot; // <----------------------------

}

then create the class org.example.webfxexample.openjfk.Main (e.g.) in the webfx-example-application-openjfx project, which invokes ApplicationBooter.main():

package org.example.webfxexample.openjfk;

import dev.webfx.platform.shared.services.boot.ApplicationBooter;

public class Main {

	public static void main(String[] args) {
		ApplicationBooter.main(args);
	}

}

and finally set this new class as mainClass for JavaPackager.

[salmonb]

I know it's a bit unusual to not have the main class in the jar, but this is because I'm using a little framework (I'm actually the author of it) with a plugin architecture (my app is started through a Java Service). So the entry point is not directly my app, but that framework, which is located in the dependencies. It's still a quite standard Java application IMO.

Your suggestion would probably work, but many files are automatically generated as you probably noticed with the DO NOT EDIT MANUALLY comments, and the framework is maintaining these files. If there is no other solution, I would prefer in the end to generate the fat jar of my app and pass it to JavaPackager rather than modifying the sources.

Do you know if that limitation (not being able to run a thin jar if the main class is not inside) is coming from Java or JavaPackager?

[salmonb]

I made some further investigation and it appears that it is possible to make a thin jar with a main class outside of it (but in the dependencies), so this is not a limitation coming from Java.

I have maybe another explanation of what goes wrong: the list of jars listed in META-INF/MANIFEST.MF doesn't match the jars in the libs folder. For example, the jar containing the main class is located in libs/webfx-platform-shared-boot-0.1.0-SNAPSHOT.jar but in the MANIFEST, it is listed as libs/webfx-platform-shared-boot-0.1.0-20220426.222906-22.jar. The name doesn't match, that's why it doesn't find the main class. Same problem with other SNAPSHOT jars. So it looks like there is something wrong when copying the dependencies with SNAPSHOT artifacts.

Could you please have a look at this?

[salmonb]

According to https://stackoverflow.com/questions/41982167/maven-jar-plugin-wrong-class-path-entry-for-snapshot-dependency, the problem is coming from the Maven Jar plugin (which generates the MANIFEST file), and can be resolved by setting the configuration parameter useUniqueVersions to false.

Could you please test if setting useUniqueVersions to false within your plugin when calling the Jar plugin solves that Class-Path problem in the MANIFEST file, and make my app running?

[fvarrui]

Hi @salmonb!
I've just created a new branch related to this issue: issue-204 branch. I've applied your suggested change to the plugin (setUniqueVersions=false) ... I'm already testing, so I'll let you know when I've made some progress.

[fvarrui]

Its seems that your suggestion fixed the problem!! 👏 👏 I'm not releasing SNAPSHOT versions to Maven Central, so you have to package and install manually the plugin in your local Maven repo (JavaPackager version in branch issue-204: 1.6.7-SNAPSHOT).

There's a problem analyzing dependencies with jdeps ... it's a problem in the plugin and I'm working on it. A workaround is setting customizedJre=false. I did it this way to package your sample app.

[salmonb]

Hi @fvarrui

That's great news, thank you!

I will use your SNAPSHOT version for the time being.

As you noticed, there is a warning message from JavaFX because the application is not launched with the Java Platform Module System.

I managed to start it with the thin jar and dependencies in the JPMS way like this (and the warning was removed):

java --module-path libs --module webfx.platform.shared.boot/dev.webfx.platform.shared.services.boot.ApplicationBooter  

Do you think you can provide an option in your plugin to start modularised application in this way?

[fvarrui]

Yes, I plan to add JPMS support in JavaPackager, but I've kept the legacy mode (--class-path) for simplicity as it works for all Java versions

[salmonb]

Great that you plan the JPMS support, and I think you can achieve this without making any change on the generated jars, so they work both in legacy mode or JPMS mode. It will just be the command line to start the application that will differ between the 2 modes.

Starting the application in legacy mode is done wiith

java -jar webfx-example-0.1.0-SNAPSHOT-runnable.jar

and in JPMS mode with

java --module-path libs --module webfx.platform.shared.boot/dev.webfx.platform.shared.services.boot.ApplicationBooter 

So for people wanting the final executable to start in JPMS mode, you would need to offer an additional parameter in your plugin to pass the starting module and main class (webfx.platform.shared.boot/dev.webfx.platform.shared.services.boot.ApplicationBooter in my case)

[fvarrui]

Hi @salmonb,
Thanks!

It's a bit more complicated, since I have to adapt the generation of the different native executables, those of each platform, as they are the ones that actually execute the JVM. e.g.: launch4j for Windows or universalJavaApplicagtionStub for MacOS only accept the "--class-path" (-cp) argument, not supporting "--module-path" nor "--module" arguments.

Any help would be appreciated! 😃

[salmonb]

I see, and so you don't have a direct control on how the application is started...

Maybe while these third-party tools are not yet ready for JPMS, you can do like this project does: https://xy2401.com/local-docs/java/jetty.9.4.24.v20191120/startup-jpms.html
It looks like they have a legacy mode application that starts the final application in JPMS mode.

Could that work if your plugin generates such an intermediate application starter that finally starts the final application in JPMS mode with the parameter we would pass to your plugin?

[fvarrui]

I see, and so you don't have a direct control on how the application is started...

Yes!! That's the problem

Maybe while these third-party tools are not yet ready for JPMS, you can do like this project does: https://xy2401.com/local-docs/java/jetty.9.4.24.v20191120/startup-jpms.html It looks like they have a legacy mode application that starts the final application in JPMS mode.

Could that work if your plugin generates such an intermediate application starter that finally starts the final application in JPMS mode with the parameter we would pass to your plugin?

Yes, in fact, I think it's possible. Maybe this way all native launchers could always call the same startup JAR, which will launch a new JVM instance like you said. But I'm not sure how this will affect the way the system shows the started process (e.g. in Task Manager). This was one of the reasons why you have several launchers for Windows: launch4j, winrun4j, why (new one in 1.6.7), ... if we do so, with an intermmediate launcher, "java" process will be shown in the Task Manager, not the EXE.

I have to think a bit more about this, but thanks so much for your proposal!

[fvarrui]

Hi @salmonb!

I think I have good news... I've been trying to run a modular Java application in different ways and finally realized that maybe we can pass --module-path as a VM argument to Launch4j and --module=module.name/main.class as the main class argument. The key is that we can use --module=xxx (1 argument) instead of --module xxx (2 arguments), since both are fully valid arguments. At least I think it might work fine for Linux and Mac startup scripts, and hopefully it will work for Windows as well.

In theory it should work since Launch4j should be nothing more than a wrapper for Java. I mean, my understanding is that all it does is invoke java passing it the supplied parameters. If so, it should work fine.

I'll tell you something when I come to some empirical conclusion 😃

[fvarrui]

Hi @salmonb!
I've managed to run a generated Launch4j EXE using Java modules just setting some JavaPackager properties (without changing anything):

<winConfig>
    <wrapJar>false</wrapJar>
</winConfig>
<vmArgs>								
    <vmArg>--module-path ${project.name}-${project.version}-runnable.jar;libs/commons-io-2.7.jar</vmArg>
</vmArgs>
<mainClass>--module=hello.world.maven/io.github.fvarrui.helloworld.Main</mainClass>

It's working fine, but JAR cannot be wrapped into the EXE ... of course, since this is possible, it can be automated internally in JavaPackager, so module-path argument can be calculated (there's no need to specify it) and mainClass can be adapted adding --module={module.name}/ to the specified mainClass, so the process could be transparent to the user.

I'm thinking that also it's possible to automatically detect if the runnable JAR is modular or not, and if so then apply last changes internally to properties. Even it's possible to automatically modularise all non-modular dependencies, transforming them into explicit modules (including module-info.class file), what would be useful when using jlink.

[salmonb]

hi @fvarrui, I was on holidays...

Thank you so much for all the progress you made in the meantime, very much appreciated 😃

Great idea to automate the process and make it transparent to the user.
Do you think you will have time to implement this in version 1.6.7?

[fvarrui]

Hi @salmonb!
I'm going to release version 1.6.7 in the next few days, because I've already waited a long time to do it.
I think that JPMS support deserves a minor vesion update: 1.7.0.

[fvarrui]

I'm also going to open a new issue requesting to add JPMS support to myself 😄 ... so, you can follow the progress.

I think we can close this issue.

[fvarrui]

Branch issue-204 merged into devel.

[salmonb]

hi @fvarrui,

Yes sure you can close this issue now. Let me know when you have created the new issue for the JPMS support.

And no problem to wait version 1.7.0 to have it.

Thanks for all your work on this project 👍

[fvarrui]

JavaPackager v1.6.7 released to Maven Central. See changes here.