ISSUE-8445: Building Pulsar (incl. docker images) from source with Java 9+ (Java 11, Java 15, etc.) causes silent NoSuchMethodErrors at runtime

[sijie]

Original Issue: apache#8445

---

Building Pulsar (incl. docker images) from source with Java 11 causes silent NoSuchMethodErrors at runtime

To Reproduce
Steps to reproduce the behavior:

Use Java 11.
Then, compile v2.6.2-candidate-1 docker images locally:

git clone --depth 1 -b v2.6.2-candidate-1 https://github.com/apache/pulsar pulsar-2.6
cd pulsar-2.6
mvn -DskipTests clean install
./docker/build.sh

Start pulsar in docker

docker run -p 6650:6650 pulsar:2.6.2 bin/pulsar standalone --no-functions-worker -nss

Compile & run this Pulsar code with required libraries:

import org.apache.pulsar.client.api.*;

public class PulsarJava11Problem {
    public static void main(String[] args) throws PulsarClientException {
        PulsarClient pulsarClient = PulsarClient.builder().serviceUrl("pulsar://localhost:6650").build();
        Producer<String> producer = pulsarClient.newProducer(Schema.STRING).topic("mytopic").create();
    }
}

The producer never gets created and it times out.

What happens on a server is a java.lang.NoSuchMethodError: java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;

This happens at https://github.com/apache/pulsar/blob/097108a7abae750a239abb606d2a31442319e741/pulsar-common/src/main/java/org/apache/pulsar/common/schema/LongSchemaVersion.java#L44 .

java.lang.NoSuchMethodError: java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
	at org.apache.pulsar.common.schema.LongSchemaVersion.bytes(LongSchemaVersion.java:44)
	at org.apache.pulsar.common.protocol.Commands.newProducerSuccessCommand(Commands.java:377)
	at org.apache.pulsar.broker.service.PulsarCommandSenderImpl.sendProducerSuccessResponse(PulsarCommandSenderImpl.java:95)
	at org.apache.pulsar.broker.service.ServerCnx.lambda$handleProducer$19(ServerCnx.java:1137)
	at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670)
	at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:646)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
	at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670)
	at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:646)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
	at org.apache.pulsar.broker.service.schema.BookkeeperSchemaStorage.lambda$createSchemaLocator$28(BookkeeperSchemaStorage.java:504)
	at org.apache.bookkeeper.util.ZkUtils$1.processResult(ZkUtils.java:80)
	at org.apache.bookkeeper.zookeeper.ZooKeeperClient$10$1.processResult(ZooKeeperClient.java:737)
	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:636)
	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510)

This exception never gets printed to the log, this stacktrace was captured using the debugger.

Expected behavior

Compiling with an unsupported Java version should result in an error and the NoSuchMethodError that happens in Pulsar shouldn't get "swallowed".

It should be possible to compile the code with Java 9+ without running into issues when running with Java 8.

There's an explanation of the issue with ByteBuffer at jetty/jetty.project#3244 . A solution is also described:

The solution is to cast the ByteBuffer to Buffer when calling those methods:

((Buffer)byteBuffer).position(0);