gRPC: Cannot seem to use the protobuf-maven-plugin instead of quarkus-maven

[tmulle]

Describe the bug

I originally wanted to use the built in grpc compiler built into Quarkus, but I'm having trouble getting it to handle external imports so I can reference my protos from an external jar which are nested in a folder off the root.

ie. protobuf/Foo.proto

So, I saw that we can use the protobuf-maven-plugin to configure more options.

Following the link in the docs at https://quarkus.io/guides/grpc-getting-started#protobuf-maven-plugin I configured everything as mentioned.

I'm am using the maven-dependency plugin to pull our standard proto files store in Nexus and then we expand them into the target folder so we can include them with the protobuf-maven-plugin.

We do this in another non-quarkus project and it works fine, so I wanted to use this mechanism.

But, when I run the mvn clean compile a few things happen:

1. It appears the protobuf-maven-plugin attempts to run but doesn't find any protos at src/main/proto which is the default configuration according to the docs at: https://www.xolstice.org/protobuf-maven-plugin/compile-mojo.html
   which defines the sourceRoot as src/main/proto. I do have proto files in the src/main/proto folder.

2. It then appears that the default quarkus grpc compiler is still trying to run and is failing because of the import issues I am trying to fix.

Is there a way to turn off the default grpc compiling when using the other plugin? I didn't see anything in the docs.

Output

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: aarch_64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 13.5
[INFO] os.detected.version.major: 13
[INFO] os.detected.version.minor: 5
[INFO] os.detected.classifier: osx-aarch_64
[INFO] 
[INFO] --------------------< org.acme:quarkus-bcconnector >--------------------
[INFO] Building quarkus-bcconnector 1.0.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- dependency:3.6.0:unpack (unpack) @ quarkus-bcconnector ---
[INFO] Configured Artifact: com.acme:protobufs-raw:11.28.0-SNAPSHOT:jar
[INFO] 
[INFO] --- protobuf:0.6.1:compile (compile) @ quarkus-bcconnector ---
[INFO] No proto files to compile.
[INFO] 
[INFO] --- protobuf:0.6.1:compile-custom (compile) @ quarkus-bcconnector ---
[INFO] No proto files to compile.
[INFO] 
[INFO] --- resources:3.3.1:resources (default-resources) @ quarkus-bcconnector ---
[INFO] Copying 2 resources from src/main/resources to target/classes
[INFO] 
[INFO] --- quarkus:3.4.1:generate-code (default) @ quarkus-bcconnector ---
Message.proto: File not found.
services.proto:9:1: Import "Message.proto" was not found or had errors.
services.proto:37:21: "BCMessage" is not defined.
services.proto:37:41: "BCMessage" is not defined.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.713 s
[INFO] Finished at: 2023-09-21T12:30:41-04:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:3.4.1:generate-code (default) on project quarkus-bcconnector: Quarkus code generation phase has failed: InvocationTargetException: Failed to generate Java classes from proto files: [/Users/tmulle/NetBeansProjects/quarkus-bcconnector/src/main/proto/services.proto, /Users/tmulle/NetBeansProjects/quarkus-bcconnector/src/main/proto/hello.proto] to /Users/tmulle/NetBeansProjects/quarkus-bcconnector/target/generated-sources/grpc with command /Users/tmulle/NetBeansProjects/quarkus-bcconnector/target/com.google.protobuf-protoc-osx-aarch_64-exe -I=/Users/tmulle/NetBeansProjects/quarkus-bcconnector/target/protoc-dependencies/2d160609fb74c459975eca766b93d1dc5316867f -I=/Users/tmulle/NetBeansProjects/quarkus-bcconnector/src/main/proto --plugin=protoc-gen-grpc=/Users/tmulle/NetBeansProjects/quarkus-bcconnector/target/io.grpc-protoc-gen-grpc-java-osx-aarch_64-exe --plugin=protoc-gen-q-grpc=/Users/tmulle/NetBeansProjects/quarkus-bcconnector/target/quarkus-grpc2792988370329882637.sh --q-grpc_out=/Users/tmulle/NetBeansProjects/quarkus-bcconnector/target/generated-sources/grpc --grpc_out=/Users/tmulle/NetBeansProjects/quarkus-bcconnector/target/generated-sources/grpc --java_out=/Users/tmulle/NetBeansProjects/quarkus-bcconnector/target/generated-sources/grpc /Users/tmulle/NetBeansProjects/quarkus-bcconnector/src/main/proto/services.proto /Users/tmulle/NetBeansProjects/quarkus-bcconnector/src/main/proto/hello.proto -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Expected behavior

1. If I specify the protobuf-maven-plugin then it should take precedence over the built in grpc compile in quarkus maven

Actual behavior

No response

How to Reproduce?

No response

Output of uname -a or ver

No response

Output of java -version

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

3.4.1

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

No response

[quarkus-bot]

/cc @alesj (grpc), @cescoffier (grpc), @quarkusio/devtools (maven)

[alesj]

Any way to reproduce this?

[tmulle]

So I am currently documenting the two ways I'm trying to get compiling to work.

1. The standard quarkus grpc compiler
2. The protobuf-maven-plugin

This current ticket is for the protobuf-maven-plugin and I've documented what I'm trying to do at:
https://github.com/tmulle/quarkus-grpc-import-test/tree/protobuf-maven-plugin-test

It is a branch of the quarkus-grpc-import-test which is referenced in ticket #36078 in which I'm trying to use the standadrd quarkus grpc/protoc compiler
to handle external protos in a dependency with subfolder.

[cescoffier]

We do not recommend using the gRPC maven plugin. If you really want to do it, follow the instructions from https://quarkus.io/guides/grpc-getting-started#protobuf-maven-plugin, and make sure your proto files are NOT in src/main/proto.

The import issue looks like a bug we need to fix.

[tmulle]

@cescoffier ok, thanks for the update.

There is a slight issue when still trying to use the plugin as you have documented.

1. Using this configuration below, causes the plugin to always say "No protos to compile" even though I put them in the correct location.

2. Commenting out the includes parameters, causes the plugin to find the protos in the project, but no longer can find my base protos we expanded.

3. I have to comment out the generate-code and generate-test in the quarkus-maven plugin to stop it from trying to compile the proto/grpc even though I've specified the plugin.

Anyway, I've filed a bug with the git hub project for that plugin and asked on Stack Overflow for help. The plugin is no longer maintained so I don't know if it will get fixed. I can't imagine I'm the only person to hit this issue.

As far as the import issue being fixed that would be great so we can generate gRPC code with our external dependencies as mentioned.

<plugin>
                <groupId>org.xolstice.maven.plugins</groupId>
                <artifactId>protobuf-maven-plugin</artifactId>      
                <version>0.6.1</version>
                <configuration>
                    <protocArtifact>com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier}</protocArtifact> 
                    <pluginId>grpc-java</pluginId>
                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}</pluginArtifact>
                    <protocPlugins>
                        <protocPlugin>
                            <id>quarkus-grpc-protoc-plugin</id>
                            <groupId>io.quarkus</groupId>
                            <artifactId>quarkus-grpc-protoc-plugin</artifactId>
                            <version>3.4.1</version>
                            <mainClass>io.quarkus.grpc.protoc.plugin.MutinyGrpcGenerator</mainClass>
                        </protocPlugin>
                    </protocPlugins>
                    <includes>
                        <include>${project.build.directory}/protobuf/base/protobuf/*.proto</include>
                    </includes>
                    <protoSourceRoot>${project.basedir}/src/main/protobuf</protoSourceRoot>
                    <checkStaleness>false</checkStaleness>
                </configuration>
                <executions>
                    <execution>
                        <id>compile</id>
                        <goals>
                            <goal>compile</goal>
                            <goal>compile-custom</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>test-compile</id>
                        <goals>
                            <goal>test-compile</goal>
                            <goal>test-compile-custom</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

[tmulle]

Ok, turns out it was my fault and misread the docs for the protobuf-maven-plugin.

I was using includes when I should've been using:

<additionalProtoPathElements>
                        <additionProtoPathElement>${project.build.directory}/protos/protobuf</additionProtoPathElement>
                    </additionalProtoPathElements>

I found it by cloning the repo for the plugin and stepping through the debugger and realized the error.

[tmulle]

I was able to get my code building with the protobuf-maven-plugin finally.

I did have to add a dependency on:

 <dependency>
            <groupId>javax.annotation</groupId>
            <artifactId>javax.annotation-api</artifactId>
        </dependency>

because I would get compile errors about missing @javax.annotation.Generated.

Looking forward to the official quarlus gRPC build method though.

[cescoffier]

So, yes, the Maven plugin and protoc are still using the old javax @Generated annotation. In Quarkus, we replace that annotation as we wanted to move away from the javax namespace (Jakarta stuff).