GCloud dependency conflicts

[bjwatson]

Steps to Reproduce

1. virtualenv venv
2. source venv/bin/activate
3. pip install -e git+https://github.com/googleapis/artman#egg=remote
4. start_conductor.py

Expected Result

The conductor successfully starts.

Actual Result

The following dependency conflict occurs:

Traceback (most recent call last):
  File "/usr/local/google/home/brianwatson/src/gapi-dev/googleapis/venv/bin/start_conductor.py", line 4, in <module>
    __import__('pkg_resources').require('googleapis-artman==0.1.0')
  File "/usr/local/buildtools/current/sitecustomize/sitecustomize.py", line 181, in SetupPathsAndImport
    return real_import(name, globals, locals, fromlist, level)
  File "/usr/local/google/home/brianwatson/src/gapi-dev/googleapis/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2985, in <module>
    @_call_aside
  File "/usr/local/google/home/brianwatson/src/gapi-dev/googleapis/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2971, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/google/home/brianwatson/src/gapi-dev/googleapis/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2998, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/google/home/brianwatson/src/gapi-dev/googleapis/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 662, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/local/google/home/brianwatson/src/gapi-dev/googleapis/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 675, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/local/google/home/brianwatson/src/gapi-dev/googleapis/venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 859, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (grpc-google-logging-v2 0.9.3 (/usr/local/google/home/brianwatson/src/gapi-dev/googleapis/venv/lib/python2.7/site-packages), Requirement.parse('grpc-google-logging-v2<0.9.0,>=0.8.1'), set(['gax-google-logging-v2']))

Analysis

This is ultimately because the last release of gcloud (0.18.1) does not set upper bounds on their gax-google-* and grpc-google-* dependencies. Due to the rename of gax-google-* to gapic-google-*, its latest version is only 0.8.x. This is incompatible with the latest 0.9.x versions of grpc-google-*.

FYI @tseaver and @dhermes, if you could push a 0.18.2 version of gcloud that sets an upper bound of <0.9.0 on the gax-google-* and grpc-google-* dependencies, then this issue will be resolved.

The other fix is to migrate artman to google-cloud 0.19.x (the rename of gcloud), which I began in #101 and is captured as a feature request in #103.

[geigerj]

Note: workaround is to manually manage the installed version with pip

(venv) $ pip uninstall {conflicting package}
(venv) $ pip install {conflicting package}=={non-conflicting version}

[dhermes]

This is a known issue but google-cloud-python has now been released (version 0.19.0)

I am AFK right now so can't find the issue but @geigerj mentioned in our bug tracker that 0.18.2 wouldn't be necessary in light of 0.19.0

[bjwatson]

@dhermes I think we're realizing in hindsight that this small 0.18.2 release would help the transition that was created by us doing two rename simultaneously: gcloud -> google-cloud and gax-google-* -> gapic-google-*.

[dhermes]

Since this is "our" issue I opened #2385 as well (for record-keeping). This release seems to be more trouble than it's worth.

[bjwatson]

Thanks, Danny. @geigerj or I will test that this bug is no longer occurring, and then close this issue.

[bjwatson]

This is all due to lacking upper bounds on dependencies in our code as well as yours. We just need to tighten this up going forward, so that we're enforcing semver (including the variant we agreed to for 0.x).

[bjwatson]

@dhermes Could I trouble you to release gcloud 0.18.3 with the google-gax upper-bound tightened to 0.13dev? I don't think that gax-google-* 0.8.x is compatible with google-gax 0.13.0, due to moving away from the gRPC beta methods in the latter (the former did not do this until gapic-google-* 0.9.x).

[dhermes]

@bjwatson Does it matter? We already have 0.19.0 released. This is a lot of churn for 0.x versions

[bjwatson]

It's because our self-service instructions are broken for our partners due to artman still depending on gcloud. There's more work we need to do to port it to google-cloud: #101

It seems fastest to just fix the upper bounds in gcloud 0.18.x, so that we can keep focusing on churning out APIs before the end of the quarter.

[dhermes]

1. OK doing it now
2. Please fix your self-service instructions instead of punting back to us

---

Pushed the tag, canceled the build since it won't push to PyPI anyway and Travis is being slow these days. The only relevant commit makes the upper bound tighter. Manually uploaded to PyPI with twine.

[bjwatson]

@dhermes Thanks for pushing the new release.

Just for the record, the problem wasn't with our instructions. It was with upgrading our tool's dependency from gcloud 0.18.x to google-cloud 0.19.x, which apparently is not trivial (#101). We'll need to do it eventually anyway, but I was thinking that fixing the dependency upper bounds in gcloud 0.18.x was a more expedient solution. Sorry it required two passes.

[bjwatson]

Confirmed that this issue is now fixed.