Public static Methods not found? #43

manticore-projects · 2022-05-28T14:13:00Z

Greetings,

after setting up the Gradle Build, I have tried to compile a simple Java Class and it fails promptly:

Execution failed for task ':wasm'.
> java.util.NoSuchElementException
        at com.manticore.jsqlformatter.JSQLFormatter.appendDelete(JSQLFormatter.java:522)

which refers to a private static method using an Enum separation :

  private static void appendDelete(StringBuilder builder, Delete delete, int indent) {
    int i = 0;

    appendKeyWord(builder, outputFormat, "DELETE", "", " ");

    OracleHint oracleHint = delete.getOracleHint();
    if (oracleHint != null) appendHint(builder, outputFormat, oracleHint.toString(), "", " ");

    List<Table> tables = delete.getTables();

    if (tables != null && tables.size() > 0) {
      int j = 0;
      for (Table table : tables) {
        switch (separation) {                    // This is line 522 where JWebAssembly fails
/*--> */ case AFTER:                            // separation is defined on Class Level as: 
            appendObjectName(                    // private static Separation separation = Separation.BEFORE;
                builder,
                outputFormat,
                table.getFullyQualifiedName(),
                "",
                j < tables.size() - 1 ? ", " : " ");
            break;
          case BEFORE:
          default:
            appendObjectName(
                builder, outputFormat, table.getFullyQualifiedName(), j > 0 ? ", " : "", " ");
            break;
        }

        j++;
      }
    }
    ....

What exactly could be the problem here please?

The text was updated successfully, but these errors were encountered:

Horcrux7 · 2022-05-28T14:57:09Z

Do you have more lines of the stacktrace? With the Gradle command line option --stacktrace you get the stacktrace if there is an error.
With which Java compiler was it compiled?
Which compiler version do you use? I have rewritten some things yesterday in the BranchManager. Because the Gradle cache it can be tricky to update it.
I think it is an exception on the BranchManager which calculate the GOTO in the Java byte code back to programming structure which is valid in WASM. A very complex and fragile part of the compiler.

I think the best for reproducing is if you send the *.class file. Then all conditions like Java version, etc can be ignored.

manticore-projects · 2022-05-28T15:04:59Z

Greetings.

Thank you for prompt response.
I have the Gradle Stacktrace output attached.

JVM is Jetbrains 11 (OpenJDK 11).
gradle.zip

I have also the JAR file attached, it stems from JSQLFormatter.
jsqlformatter-0.1.12.zip

> Task :JSQLFormatter:wasm FAILED
	compiler: jwebassembly-master-SNAPSHOT.jar
Parsing error with org/apache/felix/bundleplugin/AntPlugin.class in file:/home/are/.gradle/caches/modules-2/files-2.1/org.apache.felix/maven-bundle-plugin/5.1.5/ca60c68ce4ba07ca6a230542dd293d65bc7c360b/maven-bundle-plugin-5.1.5.jar
java.io.IOException: Unknown annotation value type pool type: 101
	at de.inetsoftware.classparser.Annotations.readElementValue(Annotations.java:91)
	at de.inetsoftware.classparser.Annotations.read(Annotations.java:51)
	at de.inetsoftware.classparser.ClassFile.getAnnotation(ClassFile.java:209)
	at de.inetsoftware.jwebassembly.module.ModuleGenerator.prepare(ModuleGenerator.java:182)
	at de.inetsoftware.jwebassembly.module.ModuleGenerator.scanLibraries(ModuleGenerator.java:156)
	at de.inetsoftware.jwebassembly.module.ModuleGenerator.<init>(ModuleGenerator.java:116)
	at de.inetsoftware.jwebassembly.JWebAssembly.compile(JWebAssembly.java:359)
	at de.inetsoftware.jwebassembly.JWebAssembly.compileToBinary(JWebAssembly.java:340)
	at de.inetsoftware.jwebassembly.JWebAssembly.compileToBinary(JWebAssembly.java:308)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)

manticore-projects · 2022-05-28T15:09:28Z

Which compiler version do you use? I have rewritten some things yesterday in the BranchManager. Because the Gradle cache it can be tricky to update it.

I have deleted the Gradle Cache and run Gradle with --no-build-cache.
JDK-8 would fail:

java.lang.UnsupportedClassVersionError: de/inetsoftware/jwebassembly/gradle/WasmPlugin has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

JDK-11 works, but throws:

java.io.IOException: Unknown annotation value type pool type: 101
	at de.inetsoftware.classparser.Annotations.readElementValue(Annotations.java:91)
	at de.inetsoftware.classparser.Annotations.read(Annotations.java:51)
...

Horcrux7 · 2022-05-28T15:32:45Z

This is the Kotlin error from #41. 101 is a small "e" and was fixed in ad93c2c

at de.inetsoftware.classparser.Annotations.readElementValue(Annotations.java:91)

The exception is now in line 114. This means you does not use the latest version: https://github.com/i-net-software/JWebAssembly/blob/master/src/de/inetsoftware/classparser/Annotations.java

You can try:

compilerVersion = 'com.github.i-net-software:jwebassembly:master-97f9fb39a7'

This is the ID from the last commit: https://jitpack.io/#i-net-software/jwebassembly/97f9fb39a7

manticore-projects · 2022-05-28T16:33:07Z

I am sorry, but after messing around I feel like a lot of more work needs to go into deployment first:

The correct Gradle dependencies seem to be:

    implementation 'com.github.i-net-software:JWebAssembly:97f9fb39a7'
    implementation 'com.github.i-net-software:jwebassembly-api:master-SNAPSHOT'

Spelling seems to matter and https://jitpack.io/#i-net-software/jwebassembly/97f9fb39a7 explains it wrong.

I pulled the source from GIT and deployed it to my Local Maven Repo. Then removed the references to https://jitpack.io.

I have thought this would make the latest GIT commit available, but no chance:

a) Local Repo shows jwebassembly-compiler-0.4.jar and jwebassembly-compiler-0.4.pom

b) while the dependency asks for JWebAssembly or jwebassembly-api --> how does this all work together please?

c) compilerVersion = 'com.github.i-net-software:jwebassembly:master-97f9fb39a7' does not work, but only complains about:

are@archlinux ~/d/s/J/JSQLFormatter (main) [1]> gradle --no-build-cache wasm
> Task :wasm FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':wasm'.
> Could not resolve all files for configuration ':wasmCompiler'.
   > Could not find com.github.i-net-software:jwebassembly:master-97f9fb39a7.
     Searched in the following locations:
       - file:/home/are/.m2/repository/com/github/i-net-software/jwebassembly/master-97f9fb39a7/jwebassembly-master-97f9fb39a7.pom
       - https://oss.sonatype.org/content/groups/public/com/github/i-net-software/jwebassembly/master-97f9fb39a7/jwebassembly-master-97f9fb39a7.pom
       - https://repo.maven.apache.org/maven2/com/github/i-net-software/jwebassembly/master-97f9fb39a7/jwebassembly-master-97f9fb39a7.pom
       - https://jitpack.io/com/github/i-net-software/jwebassembly/master-97f9fb39a7/jwebassembly-master-97f9fb39a7.pom

manticore-projects · 2022-05-28T16:39:46Z

After a lot of trial'n error, I got access to that particular commit build.
Gradle Build file:

dependencies {
    implementation 'com.github.i-net-software:JWebAssembly:97f9fb39a7'
    implementation 'com.github.i-net-software:jwebassembly-api:master-SNAPSHOT'
}

wasm {
    compilerVersion = 'com.github.i-net-software:jwebassembly:97f9fb39a7'
}

Details matter!

Unfortunately, this did not solve or improve anything. I still get exactly the same error message:

Execution failed for task ':wasm'.
> java.util.NoSuchElementException
        at com.manticore.jsqlformatter.JSQLFormatter.appendDelete(JSQLFormatter.java:522)

* Try:
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':wasm'.
        at de.inetsoftware.jwebassembly.gradle.WasmCompiler.compile(WasmCompiler.java:213)
        at de.inetsoftware.jwebassembly.gradle.WasmTask$1.execute(WasmTask.java:225)
        at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator.execute(NormalizingCopyActionDecorator.java:61)
        at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator.execute(DuplicateHandlingCopyActionDecorator.java:47)
        at org.gradle.api.internal.file.copy.CopyActionExecuter.execute(CopyActionExecuter.java:47)
        at org.gradle.api.tasks.AbstractCopyTask.copy(AbstractCopyTask.java:149)
        at jdk.internal.reflect.GeneratedMethodAccessor385.invoke(Unknown Source)

Horcrux7 · 2022-05-28T17:36:36Z

If you use a local maven repository then use compilerVersion = '+'.
jwebassembly-api is the native API replacment and the DOM API
jwebassembly-compiler is the compiler which read the byte code form Java, your code and the libraries and generate the wasm output
jwebassembly-compiler is saved on JWebAssembly project on Github on historian reasons.
jitpack does not use maven names. It use the Github project name. This result in de.inetsoftware:jwebassembly-compiler:+ --> com.github.i-net-software:jwebassembly-api:master-SNAPSHOT
if you set a numeric compiler version then the Gradle plugin use de.inetsoftware:jwebassembly-compiler:{number}
You are right. The compiler version with commit number and master was wrong. My mistake.

Execution failed for task ':wasm'.
> java.util.NoSuchElementException
        at com.manticore.jsqlformatter.JSQLFormatter.appendDelete(JSQLFormatter.java:522)

I need the file com.manticore.jsqlformatter.JSQLFormatter.class that produce the problem for analyzing. Occur this with a release version of the library?

Horcrux7 · 2022-05-28T18:07:34Z

I can reproduce the problem. I will look into.

manticore-projects · 2022-05-28T18:21:43Z

Thanks a lot!

I would like to offer two things:

I will push tomorrow a branch of JSQLFormatter with WebAssembly Output.
This might serve as a full fledged test project, which has everything -- except an UI. In my understanding, it would serve well.
I would like to support you with Unit Tests for your project.
All the information above is valuable and helpful. I would like to offer a Sphinx Website Template, where this can be gathered.

Horcrux7 · 2022-05-28T20:30:21Z

I does not understand how this should work if the compiler fail currently.
If you can reduce the problem to the max without extra classes then this can be helpful. I think it is a combination of IF, LOOP, SWITCH and ENUM.
Documentation is a large problem because things change frequently on beginning. And pull request are not possible for the wiki. But I can save all markdown texte that I receive.

Horcrux7 · 2022-05-29T09:07:51Z

The original problem is fixed but there is a problem with net.sf.jsqlparser.parser.CCJSqlParser.Statement() and net.sf.jsqlparser.parser.CCJSqlParser.JsonFunction(). Where can I find the source code for it? It is not in the library.

manticore-projects · 2022-05-29T10:41:56Z

Good Morning and compliments of the day.

Thank you again for your help and effort.
JSQLFormatter uses JSQLParser, which is a (self containing) Java Library:

dependencies {
    implementation 'com.github.jsqlparser:jsqlparser:4.5-SNAPSHOT'
    implementation 'com.github.i-net-software:JWebAssembly:master-SNAPSHOT'
    implementation 'com.github.i-net-software:jwebassembly-api:master-SNAPSHOT'
}

Will the JWebAssembly Gradle Plugin detect such dependencies automatically or will I need to create a FAT/Ueber JAR containing all the Classes?

manticore-projects · 2022-05-29T11:04:38Z

The original problem is fixed

I am sad, that I fail to get this stuff running:

I deleted all Gradle cache (this should not be necessar though)
I pulled from you GIT repo and deployed to Local Maven Repo
I tried the '+' Version for the JWebAssembly Compiler as well as the latest Commit ID f393802418

I still get the same error on Row 522 as shown above.

That said, I have created a JWebAssembly branch of my project: https://github.com/manticore-projects/jsqlformatter/tree/JWebAssembly
Maybe you would like to clone it and try to assemble/wasm with Gradle by yourself.

Horcrux7 · 2022-05-29T11:40:45Z

JSQLFormatter uses JSQLParser, which is a (self containing) Java Library:

The problem with the JSqlParser project is that I can't find the sources of CCJSqlParser, Only CCJSqlParserUtils. https://github.com/JSQLParser/JSqlParser/tree/master/src/main/java/net/sf/jsqlparser/parser
Then it must be go without the sources, which make it a little difficult.

Will the JWebAssembly Gradle Plugin detect such dependencies automatically or will I need to create a FAT/Ueber JAR containing all the Classes?

Yes, but you need only the dependency of the jwebassembly-api. The compiler is loaded from the wasm task self. The compiler version is declared via the property compilerVersion

I still get the same error on Row 522 as shown above.

I have no idea. Can it be that you have set the dependency and not the compilerVersion?

If set the Gradle command line flag --debug and delete the output that the wasm task is not UPTODATE then you can search for wasmCompiler. In the follow lines you can see which compiler jar is used.

manticore-projects · 2022-05-29T12:21:54Z

JSQLFormatter uses JSQLParser, which is a (self containing) Java Library:

The problem with the JSqlParser project is that I can't find the sources of CCJSqlParser, Only CCJSqlParserUtils. https://github.com/JSQLParser/JSqlParser/tree/master/src/main/java/net/sf/jsqlparser/parser Then it must be go without the sources, which make it a little difficult.

That's because it is a Parser Generator, meaning the Sources are generated only during the build.
I can work around that (when my main problem with the JWebAssembly Compiler is solved).

I still get the same error on Row 522 as shown above.

I have no idea. Can it be that you have set the dependency and not the compilerVersion?

I have used master-SNAPSHOT everywhere possible.
Maybe you can quickly clone the Branch and try? https://github.com/manticore-projects/jsqlformatter/tree/JWebAssembly

Horcrux7 · 2022-05-29T14:46:58Z

I have analyze the error and found the cause. It could be the cause of all the problems. At least of several problems. The cause is a SWITCH with calculated/conditional switch value like:

switch ((this.jj_ntk == -1) ? jj_ntk_f() : this.jj_ntk) {

The Java bytecode give only the code position where the switch start with a value from stack. Wasm required the code position on which the calculation of the expression starts. The current JUnit tests was to simple.

I need a new idea how I can calculate/detect the right code position and then implements it. This can take some time.

…tion of the SWITCH value does not have to be determined. #43

… This eliminates the need to calculate the end of the SWITCH structure. #43

Horcrux7 · 2022-06-06T18:50:46Z

After fixing many bugs with SWICHES, I give up for now. I have no idea how this can be fixed. It looks like the code is generated at Java byte code level. I don't see how I can represent this in Java Syntax or WebAssembly. It seems that there are back jumps from inside a SWITCH in CCJSqlParser.JsonFunction(). A back jump can only be represent with a loop in Java and WebAssembly. But there is no end of the loop. This result in overlapping structures. This is not possible with Java Syntax and WebAssembly.

But the analyze has help to make some things clearer in JWebAssembly. For example I use blocks with input parameter for the SWITCHES.

manticore-projects · 2022-06-08T07:32:49Z

Greetings.

Thanks a lot for all your effort and I am glad to have provided a solid use case test.
On the Switches: I understand that this is a kind of special code, because its auto-generated by Java CC. Although its perfectly valid Java Code and we do have the source. Also, the Java Byte Code can be de-compiled (e.g. in IntelliJ).

So I do wonder: Would it not be easier to transpile Java Source Code instead of Java Byte Code (and to use an additional De-Compiler when only Byte Code was available)?

Java Source Code can be parsed with Java CC, all what was need was to de-parse/transpile it into WebAssembly. At least this is my oversimplified understanding.

Horcrux7 · 2022-06-08T09:25:22Z

The decompilers that I have try because I does not have the source code have not work. There was an output but this output was not matching the byte code.

The byte code has a structure like:

Loopstart
   switchstart
      case x:
         loopend:
switchend

This is not valid in Java or in WebAssembly.

So I do wonder: Would it not be easier to transpile Java Source Code instead of Java Byte Code (and to use an additional De-Compiler when only Byte Code was available)?

The TeaVM use the concept of transpiling.

Java Source Code can be parsed with Java CC, all what was need was to de-parse/transpile it into WebAssembly. At least this is my oversimplified understanding.

I does not have the Java source code of CCJSqlParser. If you have the source code then make it available and I will look into it again. With a source code with which I can reproduce the problem then I can create simple unit test.

manticore-projects · 2022-06-08T09:32:42Z

Again, thanks a lot.
The source code is auto-generated during the build:

Step 1: Java CC builds Java Source Code from Grammar
Step 2: Java Source Code is compiled into Java Byte Code

I have the Autobuilt Source Code attached, but you can also Git Pull the Branch I created for your tests. Please let me know any additional help/support I can render for you.
generated.zip

Horcrux7 · 2022-06-12T19:12:56Z

Thanks for the sources. This will help.

…oblems with not splitable instruction like DUP (Java) or TEE (Wasm) #43

…op. #43

Horcrux7 · 2022-06-19T15:21:37Z

With the library version that I try the compiler error are solved. Please try it.

It ended now with a dependency to native method because

it use the Java logger API
the logger try to load a ResourceBundle
this add a dependency to URL
which result in a dependency to ZipFile

Of course ZipFile will not work because WebAssembly can't work with files. You can see the parsing order of methods if you set the debug flag.

If you have any suggestion or patch this is welcome.

manticore-projects · 2022-06-20T01:52:29Z

Good Morning,

awesome work and much appreciated. I will try as soon as possible.
On the Logger/ResourceBundle:

I think, a lot of Java libraries will call the Java Logger API
It will be cumbersome, to remove such calls from the Java Source
So why not blacklist such calls during the translation? E.g. by maintaining a List of Java API calls, which will be ignored, maybe based on Package/Class and/or Method Signature of the Method.

Big THANK YOU and Dankeschoen again!

Horcrux7 · 2022-06-20T18:19:33Z

Yes.
Yes.
This is what the JWebAssembly-API do. Take a look into the different replacements classes: https://github.com/i-net-software/JWebAssembly-API/tree/master/src/de/inetsoftware/jwebassembly/api/java
The needed work is to find good replacement points.

…eplaced, referenced native methods. #43

Horcrux7 added the bug label May 28, 2022

Horcrux7 added a commit that referenced this issue May 29, 2022

fix BranchNode order for some complex constructs. #43

f393802

manticore-projects mentioned this issue May 29, 2022

Share/deploy the Auto Generated Sources on top of the Actual Sources JSQLParser/JSqlParser#1546

Closed

Horcrux7 added a commit that referenced this issue Jun 5, 2022

Use a BLOCK with input parameters for SWITCHES so that the start posi…

c392afc

…tion of the SWITCH value does not have to be determined. #43

Horcrux7 added a commit that referenced this issue Jun 6, 2022

Use the normal break handling for the GOTO in the CASEs of an SWITCH.…

442e0c8

… This eliminates the need to calculate the end of the SWITCH structure. #43

Horcrux7 added a commit that referenced this issue Jun 6, 2022

fix handling of subopertions in a SWITCH and add more asserts. #43

a46cbe2

Horcrux7 added a commit that referenced this issue Jun 16, 2022

fix while(true) inside a while(true) loop. #43

062e2ae

Horcrux7 added a commit that referenced this issue Jun 19, 2022

use for IF conditions also a block with input parameter to prevent pr…

47c28f7

…oblems with not splitable instruction like DUP (Java) or TEE (Wasm) #43

Horcrux7 added a commit that referenced this issue Jun 19, 2022

expand a loop if there is a SWICH structure that point outside the lo…

9d1c9ee

…op. #43

Horcrux7 added a commit that referenced this issue Jun 26, 2022

Add the compiler flag "IgnoreNative" to generate stubs for rall not r…

9da6d8e

…eplaced, referenced native methods. #43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Public static Methods not found? #43

Public static Methods not found? #43

manticore-projects commented May 28, 2022

Horcrux7 commented May 28, 2022

manticore-projects commented May 28, 2022 •

edited

Loading

manticore-projects commented May 28, 2022

Horcrux7 commented May 28, 2022

manticore-projects commented May 28, 2022

manticore-projects commented May 28, 2022

Horcrux7 commented May 28, 2022

Horcrux7 commented May 28, 2022 •

edited

Loading

manticore-projects commented May 28, 2022 •

edited

Loading

Horcrux7 commented May 28, 2022

Horcrux7 commented May 29, 2022

manticore-projects commented May 29, 2022

manticore-projects commented May 29, 2022

Horcrux7 commented May 29, 2022

manticore-projects commented May 29, 2022

Horcrux7 commented May 29, 2022

Horcrux7 commented Jun 6, 2022

manticore-projects commented Jun 8, 2022

Horcrux7 commented Jun 8, 2022

manticore-projects commented Jun 8, 2022

Horcrux7 commented Jun 12, 2022

Horcrux7 commented Jun 19, 2022

manticore-projects commented Jun 20, 2022 •

edited

Loading

Horcrux7 commented Jun 20, 2022

Public static Methods not found? #43

Public static Methods not found? #43

Comments

manticore-projects commented May 28, 2022

Horcrux7 commented May 28, 2022

manticore-projects commented May 28, 2022 • edited Loading

manticore-projects commented May 28, 2022

Horcrux7 commented May 28, 2022

manticore-projects commented May 28, 2022

manticore-projects commented May 28, 2022

Horcrux7 commented May 28, 2022

Horcrux7 commented May 28, 2022 • edited Loading

manticore-projects commented May 28, 2022 • edited Loading

Horcrux7 commented May 28, 2022

Horcrux7 commented May 29, 2022

manticore-projects commented May 29, 2022

manticore-projects commented May 29, 2022

Horcrux7 commented May 29, 2022

manticore-projects commented May 29, 2022

Horcrux7 commented May 29, 2022

Horcrux7 commented Jun 6, 2022

manticore-projects commented Jun 8, 2022

Horcrux7 commented Jun 8, 2022

manticore-projects commented Jun 8, 2022

Horcrux7 commented Jun 12, 2022

Horcrux7 commented Jun 19, 2022

manticore-projects commented Jun 20, 2022 • edited Loading

Horcrux7 commented Jun 20, 2022

manticore-projects commented May 28, 2022 •

edited

Loading

Horcrux7 commented May 28, 2022 •

edited

Loading

manticore-projects commented May 28, 2022 •

edited

Loading

manticore-projects commented Jun 20, 2022 •

edited

Loading