Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NPE while executing plugin on unchanged proto files #104

Open
vicmosin opened this issue May 12, 2021 · 4 comments
Open

NPE while executing plugin on unchanged proto files #104

vicmosin opened this issue May 12, 2021 · 4 comments

Comments

@vicmosin
Copy link

  1. mvn clean
  2. mvn generate-sources -> generates initially
  3. mvn generate-sources -> fails with
Caused by: java.lang.NullPointerException: Cannot read the array length because "<local6>" is null
    at com.github.os72.protocjar.Protoc.extractStdTypes (Protoc.java:372)
    at com.github.os72.protocjar.maven.ProtocJarMojo.performProtoCompilation (ProtocJarMojo.java:397)
    at com.github.os72.protocjar.maven.ProtocJarMojo.execute (ProtocJarMojo.java:370)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)

Config to reproduce:

         <plugin>
                <groupId>com.github.os72</groupId>
                <artifactId>protoc-jar-maven-plugin</artifactId>
                <version>3.11.4</version>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>run</goal>
                        </goals>
                        <configuration>
                            <protocArtifact>com.google.protobuf:protoc:3.12.0</protocArtifact>
                            <includeStdTypes>true</includeStdTypes>
                            <inputDirectories>
                                <include>../proto</include>
                            </inputDirectories>
                            <outputTargets>
                                <outputTarget>
                                    <type>java</type>
                                </outputTarget>
                                <outputTarget>
                                    <type>grpc-java</type>
                                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}</pluginArtifact>
                                </outputTarget>
                            </outputTargets>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

I assume that happens only when optimizeCodegen=true, setting it to false works (at least as workaround)

@serdaryesilmurat
Copy link

Is there any progress on this issue?

Yes, optimizeCodegen=true seems to be a workaround, but I don't think it should be accepted as a solution.

@os72
Copy link
Owner

os72 commented Dec 5, 2021

Not sure it'll move forward without some support for the project

@Qleoz12
Copy link

Qleoz12 commented Jan 20, 2023

hi, im facing the same error.... some solutions? or where do you put optimizeCodegen=true option?

@commonquail
Copy link

This is #121, #60, #117.

The error manifests because of an interplay between <includeStdTypes>, <protocArtifact>, and <protocVersion>.

  1. <includeStdTypes> triggers a protocol major version lookup
  2. but that lookup only succeeds if <protocVersion> is set and otherwise sets local6=null
  3. but <protocArtifact> usually takes the place of <protocVersion> because of Downloading protoc doesn't respect settings chosen via mvn -s #139 (Fails behind proxy / download mechanism #61).

Fortunately <protocArtifact> supersedes <protocVersion> for protoc resolution so all three settings can be safely specified.

Alternatively, it may be that <includeMavenTypes> can safely and adequately substitute <includeStdTypes> and thereby obviate <protocVersion>. I have not tried this.

dongjoon-hyun added a commit to apache/spark that referenced this issue Oct 28, 2024
### What changes were proposed in this pull request?

This PR aims to fix `protobuf` module `Maven` compilation failure due to the following `protocol-jar-maven-plugin` bug.
- os72/protoc-jar-maven-plugin#104

Note that this happens only at `Maven` compilation.

### Why are the changes needed?

**BEFORE**
```
$ build/mvn -DskipTests clean install -pl connector/protobuf -e -am
...
[INFO] Spark Protobuf ..................................... FAILURE [  7.295 s]
...
[ERROR] Failed to execute goal com.github.os72:protoc-jar-maven-plugin:3.11.4:run (default) on project spark-protobuf_2.13: Execution default of goal com.github.os72:protoc-jar-maven-plugin:3.11.4:run failed:
Cannot read the array length because "<local6>" is null
```

**AFTER**
```
$ build/mvn -DskipTests clean install -pl connector/protobuf -e -am
...
[INFO] Reactor Summary for Spark Project Parent POM 4.0.0-SNAPSHOT:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [  2.601 s]
[INFO] Spark Project Tags ................................. SUCCESS [  5.234 s]
[INFO] Spark Project Sketch ............................... SUCCESS [  5.278 s]
[INFO] Spark Project Common Utils ......................... SUCCESS [ 14.695 s]
[INFO] Spark Project Local DB ............................. SUCCESS [  6.258 s]
[INFO] Spark Project Networking ........................... SUCCESS [  9.384 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [  7.761 s]
[INFO] Spark Project Variant .............................. SUCCESS [  3.304 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [  8.677 s]
[INFO] Spark Project Connect Shims ........................ SUCCESS [  2.690 s]
[INFO] Spark Project Launcher ............................. SUCCESS [  5.618 s]
[INFO] Spark Project Core ................................. SUCCESS [01:11 min]
[INFO] Spark Project SQL API .............................. SUCCESS [ 21.759 s]
[INFO] Spark Project Catalyst ............................. SUCCESS [01:30 min]
[INFO] Spark Project SQL .................................. SUCCESS [01:53 min]
[INFO] Spark Protobuf ..................................... SUCCESS [ 18.285 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  06:26 min
[INFO] Finished at: 2024-10-27T21:01:12-07:00
[INFO] ------------------------------------------------------------------------
```

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manually run the following command like I did in the above.
```
$ build/mvn -DskipTests clean install -pl connector/protobuf -e -am
```

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #48673 from dongjoon-hyun/SPARK-50143.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants