bundle update foo can not find compatible versions

[marcandre]

I may be missing something, but my understanding is that if bundle install works fine, then a subsequent bundle update foo should never fail (assuming no changes to the Gemfile). At worst it should resolve foo to whatever version is already in the Gemfile.lock, no?

I've attached my Gemfile & Gemfile.lock. To reproduce my issue, bundle && bundle update ember-rails. For some reason it's trying to upgrade Rails to v5 and (rightfully) failing:

$ bundle -v
Bundler version 1.13.5
$ bundle
[...]
Bundle complete! 193 Gemfile dependencies, 389 gems now installed [...]
$ bundle update ember-rails
[...]Resolving dependencies.............................................................
Bundler could not find compatible versions for gem "activesupport":
  In Gemfile:
    active_model_serializers (~> 0.8.0) was resolved to 0.8.3, which depends on
      activemodel (>= 3.0) was resolved to 5.0.0.1, which depends on
        activesupport (= 5.0.0.1)

    jquery-rails was resolved to 1.0.1, which depends on
      railties (~> 3.0) was resolved to 3.0.0, which depends on
        activesupport (= 3.0.0)
Bundler could not find compatible versions for gem "rack":
  In Gemfile:
    rack (~> 1.6)

    jquery-rails was resolved to 4.0.1, which depends on
      railties (>= 4.2.0.beta) was resolved to 5.0.0.1, which depends on
        actionpack (= 5.0.0.1) was resolved to 5.0.0.1, which depends on
          rack (~> 2.0)
Bundler could not find compatible versions for gem "rails":
  In snapshot (Gemfile.lock):
    rails (= 4.2.7)

  In Gemfile:
    rails (~> 4.2.7)

    jquery-rails was resolved to 0.1.1, which depends on
      rails (~> 3.0.0.rc)

Gemfile_with_lock.zip

PS: I've made no attempt to minimize my Gemfile to get that effect, let me know if I should.

[segiddins]

I can reproduce this, but have no idea why it's happening. A minimal reproduction would definitely help pinpoint what exactly in bundler is causing the issue

[segiddins]

Worth noting this is a regression from Bundler 1.11.2

[segiddins]

I have a sneaky fix locally for this

[marcandre]

I'm not sure what the PR does exactly, but I fear it doesn't really fix this.

With this particular Gemfile, the correct fix would upgrade ember-rails to '0.18.5'; newer versions require an upgrade to ember-data-source which is locked. I arrived to this conclusion by manually trying different requirements and doing bundle.

Is your patch arriving to the same result, or is it keeping it at '0.16.4'?

[segiddins]

Updates ember-rails to 0.18.5

diff --git a/Gemfile.lock b/Gemfile.lock
index a7faded..d2f04f2 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -134,7 +134,7 @@ GEM
     babel-transpiler (0.7.0)
       babel-source (>= 4.0, < 6)
       execjs (~> 2.0)
-    barber (0.9.2)
+    barber (0.11.2)
       ember-source (>= 1.0, < 3)
       execjs (>= 1.2, < 3)
     bcrypt-ruby (3.0.1)
@@ -284,13 +284,14 @@ GEM
     email_validator (1.1.0)
     ember-data-source (1.0.0.beta.16.1)
       ember-source (~> 1.8)
-    ember-rails (0.16.4)
+    ember-handlebars-template (0.7.4)
+      barber (>= 0.11.0)
+      sprockets (>= 3.3, < 4)
+    ember-rails (0.18.5)
       active_model_serializers
-      barber (>= 0.6.0)
       ember-data-source (>= 1.0.0.beta.5)
+      ember-handlebars-template (>= 0.1.1, < 1.0)
       ember-source (>= 1.1.0)
-      execjs (>= 1.2)
-      handlebars-source (> 1.0.0, < 3)
       jquery-rails (>= 1.0.17)
       railties (>= 3.1)
     ember-source (1.9.1)
@@ -299,7 +300,7 @@ GEM
     erubis (2.7.0)
     eventmachine (1.2.0.1)
     excon (0.51.0)
-    execjs (2.6.0)
+    execjs (2.7.0)
     exifr (1.2.4)
     eye (0.8.1)
       celluloid (~> 0.17.3)
@@ -581,7 +582,7 @@ GEM
     jmespath (1.3.1)
     journey (1.0.4)
     jquery-migrate-rails (1.2.1)
-    jquery-rails (4.1.1)
+    jquery-rails (4.2.1)
       rails-dom-testing (>= 1, < 3)
       railties (>= 4.2.0)
       thor (>= 0.14, < 2.0)

[marcandre]

Oh, great!

I note that it updaded jquery-rails and execjs needlessly (currently locked versions are ok), but it is already much better than failing to upgrade 😄. Might be a different bug altogether with the new conservative upgrading, I can open another issue afterwards.

[segiddins]

Nope, since those aren't explicitly included in the Gemfile, bundler is free to update them. Take a look at the conservative update flags if you'd prefer bundler not do that

[marcandre]

I'm quite surprised that explicit inclusion in Gemfile makes a difference, I don't quite see why that's the case and would lean to disagree with that.
In any case, execjs isn't listed in the Gemfile, but jquery-rails is...

[chrismo]

I can try to explain the explicit inclusion in Gemfile (I need to sorta write this up for related work anyway, so this was a good excuse):

If a given Gemfile and lockfile haven't changed, they've been pulled fresh from git, then bundle install of course should just install the same versions as listed in the lockfile.

If a change is made to the Gemfile, say a newer version of gem foo, then bundle install is run, then that gem is unlocked and free to move. Bundler also needs to decide what to do about all of foo's dependencies.

If you only asked for gem 'foo' in your Gemfile and foo depends on bar and qux - when you want foo to move you shouldn't have to also tell Bundler to update its dependencies, you probably don't care or want to care about its dependencies, just get on with it. So Bundler will treat it all as a unit and unlock them all and allow them all to move.

The problem comes when other gems you request in your Gemfile might also depend on bar or qux. Now Bundler has a dilemma: you want foo to move, and foo needs bar and qux, but you still have tam in your Gemfile, and it's not moving, and it also depends on qux.

If Bundler chooses to favor foo by allowing qux to move, it could break tam altogether. So Bundler chooses to let foo and bar be unlocked and move, but keeps qux locked. This could also lead to a problem, but it will at least report a VersionConflict involving foo, the changed gem, instead of tam, the gem you probably didn't know also depended on qux.

So, this leads us to the rule: "All of foo's dependencies will also be unlocked and be free to move IF they also aren't dependencies of a different declared gem in the Gemfile."

But what if one of foo's dependencies is itself a declared dependency in the Gemfile? For example, your Gemfile is gem 'foo' and gem 'bar'. Well, really it's essentially the same rule as before, just that bar isn't a dependency of another declared gem, it IS a declared gem.

This is (hopefully) what this section on Conservative Updates in bundle install docs describes.

Now to bring this back around to this ticket: bundle update has no such conservative update option for shared dependencies, though the declared gem version of the rule still holds.

If you think bundle update should have a similar conservative update option for shared dependencies, it's almost done: #4980.

[miks]

If I understand correctly, problem mentioned here is fixed with 1.14.4.
But I'm still getting weird errors like this:

    releaf was resolved to 1.0.8, which depends on
      releaf-core (= 1.0.8) was resolved to 1.0.8, which depends on
        railties (~> 4.2.7.1) was resolved to 4.2.7.1, which depends on
          actionpack (= 4.2.7.1) was resolved to 4.2.7.1, which depends on
            activesupport (= 5.0.1)

    releaf was resolved to 1.0.8, which depends on
      releaf-core (= 1.0.8) was resolved to 1.0.8, which depends on
        railties (~> 4.2.7.1) was resolved to 4.2.7.1, which depends on
          activesupport (= 4.2.7.1)

Where in fact actionpack 4.2.7.1 depends on activesupport 4.2.7.1 (https://rubygems.org/gems/actionpack/versions/4.2.7.1)

bundler 1.13.6 is working correctly. I got this error on travis-ci where latest (1.14.4) version is used.

[segiddins]

@miks please open a new issue with a gemfile we can use to debug, thanks!