Skip to content

Understanding the Problem

Jump to bottom
Elliotte Rusty Harold edited this page Jan 23, 2019 · 8 revisions

tldr; some of the libraries GCP publishes cannot be used together in the same project.

At worst, library A requires version 1 of library X and will not work with version 2 of library X. At the same time library B requires version 2 of library X and will not work with version 1. Thus no other project can use both A and B.

A slightly more common variant is that library A requires version 1 of library X but will work with version 2. Library B requires version 2 of library X and will not work with version 1. However the build path is configured such that version 1 is selected instead of version 2.

Is this a real problem? Yes. Most of the time A and B work perfectly well together, but not always; and when a problem like this does arise it's typically quite hard to diagnose and fix.

Here is one of the simplest examples we've found.

Start with this pom.xml. It has exactly two dependencies, com.google.api-client:google-api-client:1.27.0 and io.grpc:grpc-core:1.17.1. Both are official, supported GCP products. At the time of this writing these are the latest, most up-to-date versions of these artifacts.

<project xmlns="http://maven.apache.org/POM/4.0.0" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.google.cloud.tools.opensource</groupId>
  <artifactId>no-such-method-error-example</artifactId>
  <version>1.0-SNAPSHOT</version>

  <name>no-such-method-error-example</name>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
  </properties>

  <dependencies>
    <dependency>
      <groupId>com.google.api-client</groupId>
      <artifactId>google-api-client</artifactId>
      <version>1.27.0</version>
    </dependency>
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-core</artifactId>
      <version>1.17.1</version>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>1.6.0</version>
        <configuration>
          <mainClass>io.grpc.internal.App</mainClass>
        </configuration>
      </plugin>
    </plugins>
  </build>

</project>

You also need a single .java source file:

package io.grpc.internal;

import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import java.util.Map;
import java.util.Random;

public class App {

  public static void main(String[] args) {
    Map<String, Object> choice = ImmutableMap.of("clientLanguage", ImmutableList.of("en"));
    DnsNameResolver.maybeChooseServiceConfig(choice, new Random(), "localhost");
  }
}

(You can find this project ready to run in the source repo.)

The project compiles without error. Now let's run it:

$ mvn exec:java
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building no-such-method-error-example 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ no-such-method-error-example ---
[WARNING] 
java.lang.NoSuchMethodError: com.google.common.base.Verify.verify(ZLjava/lang/String;Ljava/lang/Object;)V
	at io.grpc.internal.DnsNameResolver.maybeChooseServiceConfig(DnsNameResolver.java:514)
	at io.grpc.internal.App.main(App.java:31)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
	at java.lang.Thread.run(Thread.java:748)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.800 s
[INFO] Finished at: 2019-01-16T14:34:56-05:00
[INFO] Final Memory: 9M/155M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project no-such-method-error-example: An exception occured while executing the Java class. com.google.common.base.Verify.verify(ZLjava/lang/String;Ljava/lang/Object;)V -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

What happened? In brief, io.grpc:grpc-core:1.17.1 needs a particular overloaded verify method that is present in Guava 26.0. However, com.google.api-client:google-api-client:1.27.0 instead pulls in Guava 20.0 which does not have this method. This is not discovered until runtime when something follows a code path that tries to call the missing method and fails.

Is this example too simple?

Indeed this is deliberately as small an example as we could contrive to make the problem clear. There are two features that make it look perhaps too much of a contrived problem:

  1. We put it in the io.grpc.internal package instead of a user package. This is because DnsNameResolver.maybeChooseServiceConfig is package protected. However that code is not dead. It is indirectly reachable from user code. However, it's easier to see and understand what's happening if we invoke the method directly.

  2. This program does not actually use com.google.api-client:google-api-client:1.27.0. You could simply eliminate this dependency and the problem goes away. However we can easily imagine a program that uses both com.google.api-client:google-api-client:1.27.0 and io.grpc:grpc-core:1.17.1. It would simply be slightly larger than the one we have here.

How can we fix this?

There are several ways. Instead of removing com.google.api-client:google-api-client:1.27.0 we could rewrite the pom.xml to put this dependency second:

  <dependencies>
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-core</artifactId>
      <version>1.17.1</version>
    </dependency>
    <dependency>
      <groupId>com.google.api-client</groupId>
      <artifactId>google-api-client</artifactId>
      <version>1.27.0</version>
    </dependency>
  </dependencies>

Then the version of Guava referenced by io.grpc:grpc-core:1.17.1 is chosen instead of com.google.api-client:google-api-client:1.27.0.

This fixes this simple case. However there are more complex cases where there is no acceptable ordering of the dependencies. Also, it's quite hard to find or understand why this happens.

A slightly more obvious solution is to choose a version of the conflicting dependency explicitly:

  <dependencies>
    <dependency>
      <groupId>com.google.api-client</groupId>
      <artifactId>google-api-client</artifactId>
      <version>1.27.0</version>
    </dependency>
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-core</artifactId>
      <version>1.17.1</version>
    </dependency>
    <dependency>
      <groupId>com.google.guava</groupId>
      <artifactId>guava</artifactId>
      <version>26.0-android</version>
    </dependency>
  </dependencies>

Does Gradle solve this?

Yes, in this case. Gradle uses a different dependency mediation algorithm that always chooses the most recent version of any conflicting dependency. However, there are other, more complex cases where Maven's algorithm succeeds and Gradle's fails.

BOM

The solution we are pursuing is a combination of tooling, documentation, and code changes in the underlying GCP libraries. Most importantly we are developing a bill of materials (BOM) that covers the entire GCP orbit. The BOM will provide mutually compatible versions of all GCP libraries.

Where no such N-way mutually compatible versions exist, we will update the relevant libraries so that compatible versions do exist.

requireUpperBoundsDeps

The Maven enforcer plugin can optionally fail the build if the dependency mediation algorithm selects any version other than the latest one found in the tree. In the example shown here, that does change the problem from a runtime error to a compile time error:

[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps failed with message:
Failed while enforcing RequireUpperBoundDeps. The error(s) are [
Require upper bound dependencies error for com.google.guava:guava:20.0 paths to dependency are:
+-com.google.cloud.tools.opensource:no-such-method-error-example:1.0-SNAPSHOT
  +-com.google.api-client:google-api-client:1.27.0
    +-com.google.guava:guava:20.0
and
+-com.google.cloud.tools.opensource:no-such-method-error-example:1.0-SNAPSHOT
  +-io.grpc:grpc-core:1.17.1
    +-com.google.guava:guava:26.0-android
and
+-com.google.cloud.tools.opensource:no-such-method-error-example:1.0-SNAPSHOT
  +-com.google.api-client:google-api-client:1.27.0
    +-com.google.oauth-client:google-oauth-client:1.27.0
      +-com.google.guava:guava:20.0
and
+-com.google.cloud.tools.opensource:no-such-method-error-example:1.0-SNAPSHOT
  +-com.google.api-client:google-api-client:1.27.0
    +-com.google.oauth-client:google-oauth-client:1.27.0
      +-com.google.http-client:google-http-client:1.27.0
        +-com.google.guava:guava:20.0
]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.119 s
[INFO] Finished at: 2019-01-23T15:03:07-05:00
[INFO] Final Memory: 9M/159M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce (enforce) on project no-such-method-error-example: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
[ERROR]

However:

  1. This requires end users to specifically add this enforcer rule to their builds.

  2. Upper bounds isn't always correct. Sometimes a project may work with a less recent version and fail with the most recent version. It all depends on what kinds of changes are made in between releases.

Clone this wiki locally