Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v0.12.0 regression in nested dependencies #599

Closed
aaronhurt opened this issue Sep 6, 2016 · 13 comments
Closed

v0.12.0 regression in nested dependencies #599

aaronhurt opened this issue Sep 6, 2016 · 13 comments
Labels
bug
Milestone
0.12.2

Comments

@aaronhurt
Copy link

When running glide update with version 0.11.1 we see the following:

ahurt1:ena-policy ahurt$ glide -version
glide version 0.11.1
ahurt1:ena-policy ahurt$ glide up
[INFO]  Downloading dependencies. Please wait...
[INFO]  --> Fetching updates for bitbucket.xxx.xxx/go/ena-session.
[INFO]  --> Fetching updates for github.com/renstrom/shortuuid.
[INFO]  --> Fetching updates for github.com/klauspost/crc32.
[INFO]  --> Fetching updates for github.com/satori/go.uuid.
[INFO]  --> Fetching updates for github.com/klauspost/compress.
[INFO]  --> Fetching updates for github.com/go-sql-driver/mysql.
[INFO]  --> Fetching updates for github.com/lann/builder.
[INFO]  --> Fetching updates for bitbucket.xxx.xxx/go/ena-common.
[INFO]  --> Fetching updates for github.com/lann/ps.
[INFO]  --> Fetching updates for github.com/jmoiron/sqlx.
[INFO]  --> Fetching updates for github.com/emicklei/go-restful.
[INFO]  --> Fetching updates for github.com/BurntSushi/toml.
[INFO]  --> Fetching updates for bitbucket.xxx.xxx/go/ena-community.
[INFO]  --> Fetching updates for gopkg.in/yaml.v2.
[INFO]  --> Fetching updates for github.com/couchbase/gocb.
[INFO]  --> Fetching updates for github.com/Masterminds/squirrel.
[INFO]  --> Fetching updates for github.com/klauspost/pgzip.
[INFO]  --> Fetching updates for github.com/klauspost/cpuid.
[INFO]  --> Fetching updates for bitbucket.xxx.xxx/go/ena-consul.
[INFO]  --> Fetching updates for github.com/hashicorp/go-cleanhttp.
[INFO]  --> Fetching updates for github.com/hashicorp/serf.
[INFO]  --> Setting version for bitbucket.xxx.xxx/go/ena-community to master.
[INFO]  Resolving imports
[INFO]  Fetching github.com/Sirupsen/logrus into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  --> Detected semantic version. Setting version for github.com/Sirupsen/logrus to v0.10.0.
[INFO]  Fetching github.com/rifflock/lfshook into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  --> Detected semantic version. Setting version for github.com/rifflock/lfshook to v1.0.
[INFO]  Fetching github.com/Masterminds/glide/cfg into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  --> Detected semantic version. Setting version for github.com/Masterminds/glide to v0.12.1.
[INFO]  Fetching github.com/mitchellh/cli into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  Fetching github.com/hashicorp/consul/api into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  --> Detected semantic version. Setting version for github.com/hashicorp/consul to v0.6.4.
[INFO]  Found Godeps.json file in vendor/github.com/hashicorp/consul
[INFO]  --> Parsing Godeps metadata...
[INFO]  --> Setting version for github.com/hashicorp/go-cleanhttp to 875fb671b3ddc66f8e2f0acc33829c8cb989a38d.
[INFO]  --> Setting version for github.com/hashicorp/serf to e4ec8cc423bbe20d26584b96efbeb9102e16d05f.
[INFO]  Fetching golang.org/x/sys/unix into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  --> Setting version for golang.org/x/sys to 20457ee8ea8546920d3f4e19e405da45250dc5a5.
[INFO]  Fetching github.com/Masterminds/vcs into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  --> Detected semantic version. Setting version for github.com/Masterminds/vcs to v1.8.0.
[INFO]  Fetching github.com/armon/go-radix into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  --> Setting version for github.com/armon/go-radix to 4239b77079c7b5d1243b7b4736304ce8ddb6f0f2.
[INFO]  Fetching github.com/bgentry/speakeasy into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  --> Setting version for github.com/bgentry/speakeasy to 36e9cfdd690967f4f690c6edcc9ffacd006014a0.
[INFO]  Fetching github.com/mattn/go-isatty into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  --> Setting version for github.com/mattn/go-isatty to 56b76bdf51f7708750eac80fa38b952bb9f32639.
[INFO]  Fetching github.com/stretchr/testify/suite into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  Fetching github.com/davecgh/go-spew/spew into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  Fetching github.com/pmezard/go-difflib/difflib into /Users/ahurt/Projects/go/src/bitbucket.xxx.xxx/ws/ena-policy/vendor
[INFO]  Downloading dependencies. Please wait...
[INFO]  Setting references for remaining imports
[INFO]  Project relies on 31 dependencies.

With 0.12.0+ we see something quite different:

ahurt1:ena-policy ahurt$ glide up
[INFO]  Downloading dependencies. Please wait...
[INFO]  --> Fetching github.com/klauspost/cpuid.
[INFO]  --> Fetching gopkg.in/yaml.v2.
[INFO]  --> Fetching github.com/lann/ps.
[INFO]  --> Fetching github.com/renstrom/shortuuid.
[INFO]  --> Fetching github.com/hashicorp/go-cleanhttp.
[INFO]  --> Fetching github.com/jmoiron/sqlx.
[INFO]  --> Fetching github.com/klauspost/crc32.
[INFO]  --> Fetching github.com/klauspost/compress.
[INFO]  --> Fetching github.com/satori/go.uuid.
[INFO]  --> Fetching github.com/lann/builder.
[INFO]  --> Fetching bitbucket.xxx.xxx/go/ena-community.
[INFO]  --> Fetching bitbucket.xxx.xxx/go/ena-common.
[INFO]  --> Fetching github.com/BurntSushi/toml.
[INFO]  --> Fetching bitbucket.xxx.xxx/go/ena-session.
[INFO]  --> Fetching bitbucket.xxx.xxx/go/ena-consul.
[INFO]  --> Fetching github.com/Masterminds/squirrel.
[INFO]  --> Fetching github.com/couchbase/gocb.
[INFO]  --> Fetching github.com/klauspost/pgzip.
[INFO]  --> Fetching github.com/emicklei/go-restful.
[INFO]  --> Fetching github.com/go-sql-driver/mysql.
[INFO]  --> Fetching github.com/hashicorp/serf.
[INFO]  --> Setting version for bitbucket.xxx.xxx/go/ena-community to master.
[INFO]  Resolving imports
[INFO]  --> Fetching github.com/Sirupsen/logrus.
[INFO]  --> Fetching github.com/rifflock/lfshook.
[INFO]  --> Fetching github.com/Masterminds/glide.
[INFO]  --> Fetching github.com/mitchellh/cli.
[INFO]  --> Fetching github.com/hashicorp/consul.
[INFO]  --> Detected semantic version. Setting version for github.com/hashicorp/consul to v0.6.4.
[INFO]  --> Fetching golang.org/x/sys.
[INFO]  --> Fetching github.com/Masterminds/vcs.
[INFO]  --> Fetching github.com/armon/go-radix.
[INFO]  --> Fetching github.com/bgentry/speakeasy.
[INFO]  --> Fetching github.com/mattn/go-isatty.
[INFO]  --> Fetching github.com/stretchr/testify.
[INFO]  --> Fetching github.com/davecgh/go-spew.
[INFO]  --> Fetching github.com/pmezard/go-difflib.
[INFO]  Downloading dependencies. Please wait...
[INFO]  Setting references for remaining imports
[INFO]  Exporting resolved dependencies...
[INFO]  --> Exporting bitbucket.xxx.xxx/go/ena-community
[INFO]  --> Exporting bitbucket.xxx.xxx/go/ena-session
[INFO]  --> Exporting github.com/Masterminds/squirrel
[INFO]  --> Exporting github.com/hashicorp/go-cleanhttp
[INFO]  --> Exporting github.com/go-sql-driver/mysql
[INFO]  --> Exporting github.com/klauspost/pgzip
[INFO]  --> Exporting github.com/couchbase/gocb
[INFO]  --> Exporting github.com/hashicorp/serf
[INFO]  --> Exporting github.com/Sirupsen/logrus
[INFO]  --> Exporting github.com/klauspost/crc32
[INFO]  --> Exporting github.com/rifflock/lfshook
[INFO]  --> Exporting github.com/emicklei/go-restful
[INFO]  --> Exporting github.com/satori/go.uuid
[INFO]  --> Exporting github.com/hashicorp/consul
[INFO]  --> Exporting github.com/Masterminds/vcs
[INFO]  --> Exporting github.com/jmoiron/sqlx
[INFO]  --> Exporting github.com/armon/go-radix
[INFO]  --> Exporting github.com/klauspost/compress
[INFO]  --> Exporting github.com/BurntSushi/toml
[INFO]  --> Exporting github.com/bgentry/speakeasy
[INFO]  --> Exporting github.com/mattn/go-isatty
[INFO]  --> Exporting github.com/renstrom/shortuuid
[INFO]  --> Exporting github.com/lann/ps
[INFO]  --> Exporting github.com/stretchr/testify
[INFO]  --> Exporting github.com/mitchellh/cli
[INFO]  --> Exporting github.com/klauspost/cpuid
[INFO]  --> Exporting github.com/pmezard/go-difflib
[INFO]  --> Exporting bitbucket.xxx.xxx/go/ena-consul
[INFO]  --> Exporting golang.org/x/sys
[INFO]  --> Exporting github.com/lann/builder
[INFO]  --> Exporting github.com/davecgh/go-spew
[INFO]  --> Exporting bitbucket.xxx.xxx/go/ena-common
[INFO]  --> Exporting github.com/Masterminds/glide
[INFO]  --> Exporting gopkg.in/yaml.v2
[INFO]  Replacing existing vendor dependencies
[INFO]  Project relies on 31 dependencies.

The exact same glide.yaml is being used and does not explicitly pin the version of any packages other than ena-community. The versions are pinned by included packages that have their own glide.yaml files checked into the repository. The 0.11.1 code properly finds and handles the nested dependencies whereas the 0.12.0+ does not.
@mattfarina
Copy link
Member

@leprechau thanks for filing this. I'll dig into this ASAP.

@mattfarina
Copy link
Member

@leprechau do you have an example glide.yaml file you can share?

@aaronhurt
Copy link
Author

aaronhurt commented Sep 7, 2016 •
edited
Loading

@mattfarina yes ...

The glide.yaml from the ena-policy application where we are running glide up

package: bitbucket.xxx.xxx/ws/ena-policy
import:
  - package: bitbucket.xxx.xxx/go/ena-community
    repo: ssh://git@bitbucket.xxx.xxx:7999/go/ena-community.git
    vcs: git
    version: master
  - package: bitbucket.xxx.xxx/go/ena-common
    repo: ssh://git@bitbucket.xxx.xxx:7999/go/ena-common.git
    vcs: git
  - package: bitbucket.xxx.xxx/go/ena-consul
    repo: ssh://git@bitbucket.xxx.xxx:7999/go/ena-consul.git
    vcs: git
  - package: bitbucket.xxx.xxx/go/ena-session
    repo: ssh://git@bitbucket.xxx.xxx:7999/go/ena-session.git
    vcs: git
  - package: github.com/couchbase/gocb
  - package: github.com/Masterminds/squirrel
  - package: github.com/klauspost/pgzip
  - package: github.com/klauspost/cpuid
  - package: github.com/go-sql-driver/mysql
  - package: github.com/klauspost/compress
  - package: github.com/jmoiron/sqlx
  - package: github.com/renstrom/shortuuid
  - package: github.com/emicklei/go-restful
  - package: github.com/klauspost/crc32
  - package: github.com/satori/go.uuid
  - package: github.com/lann/builder
  - package: github.com/lann/ps
  - package: github.com/BurntSushi/toml
  - package: gopkg.in/yaml.v2
  - package: github.com/hashicorp/go-cleanhttp
  - package: github.com/hashicorp/serf/coordinate

This is the glide.yaml from the ena-common package that is imported by ena-policy

package: bitbucket.xxx.xxx/go/ena-common
import:
  - package: github.com/Masterminds/glide
    version: =v0.12.1
    subpackages:
    - cfg
    vcs: git
  - package: github.com/emicklei/go-restful
  - package: github.com/mitchellh/cli
  - package: github.com/Sirupsen/logrus
    version: =v0.10.0
  - package: github.com/rifflock/lfshook
    version: =v1.0

@mattfarina
Copy link
Member

@leprechau I created an example project at https://github.com/mattfarina/go-test2. With this I can't reproduce the problem. What I see is...

[INFO]  Downloading dependencies. Please wait...
[INFO]  --> Fetching updates for github.com/Masterminds/vcs.
[INFO]  --> Fetching updates for github.com/mattfarina/go-test.
[INFO]  --> Detected semantic version. Setting version for github.com/Masterminds/vcs to 1.7.0.
[INFO]  Resolving imports
[INFO]  --> Fetching updates for github.com/Masterminds/semver.
[INFO]  --> Detected semantic version. Setting version for github.com/Masterminds/semver to v1.1.1.
[INFO]  Downloading dependencies. Please wait...
[INFO]  Setting references for remaining imports
[INFO]  Exporting resolved dependencies...
[INFO]  --> Exporting github.com/Masterminds/vcs
[INFO]  --> Exporting github.com/mattfarina/go-test
[INFO]  --> Exporting github.com/Masterminds/semver
[INFO]  Replacing existing vendor dependencies
[INFO]  Project relies on 3 dependencies.

What am I missing? Can you show me how to tweak this to reproduce the problem?

@aaronhurt
Copy link
Author

aaronhurt commented Sep 8, 2016 •
edited
Loading

@mattfarina It took me a bit to reproduce but I have done it with another test repo set. The issue only seems to appear when you are importing subpackages of a target repository.

I used this repository: https://github.com/leprechau/glide-test1

These are the results here ...

charlie:glide-test1 ahurt$ glide -version
glide version 0.11.1
charlie:glide-test1 ahurt$ glide up
[INFO]  Downloading dependencies. Please wait...
[INFO]  --> Fetching updates for github.com/leprechau/glide-test2.
[INFO]  Resolving imports
[INFO]  Fetching github.com/Masterminds/glide/cfg into /Users/ahurt/Projects/go/src/github.com/leprechau/glide-test1/vendor
[INFO]  --> Detected semantic version. Setting version for github.com/Masterminds/glide to v0.12.1.
[INFO]  Fetching github.com/couchbase/gocb into /Users/ahurt/Projects/go/src/github.com/leprechau/glide-test1/vendor
[INFO]  --> Detected semantic version. Setting version for github.com/couchbase/gocb to v1.0.0.
[INFO]  Fetching github.com/Masterminds/vcs into /Users/ahurt/Projects/go/src/github.com/leprechau/glide-test1/vendor
[INFO]  --> Detected semantic version. Setting version for github.com/Masterminds/vcs to v1.8.0.
[INFO]  Fetching gopkg.in/yaml.v2 into /Users/ahurt/Projects/go/src/github.com/leprechau/glide-test1/vendor
[INFO]  Downloading dependencies. Please wait...
[INFO]  Setting references for remaining imports
[INFO]  Project relies on 5 dependencies.

charlie:glide-test1 ahurt$ glide -version
glide version 0.12.1
charlie:glide-test1 ahurt$ glide up
[INFO]  Downloading dependencies. Please wait...
[INFO]  --> Fetching updates for github.com/leprechau/glide-test2.
[INFO]  Resolving imports
[INFO]  --> Fetching updates for github.com/Masterminds/glide.
[INFO]  --> Fetching updates for github.com/couchbase/gocb.
[INFO]  --> Fetching updates for github.com/Masterminds/vcs.
[INFO]  --> Fetching updates for gopkg.in/yaml.v2.
[INFO]  Downloading dependencies. Please wait...
[INFO]  Setting references for remaining imports
[INFO]  Exporting resolved dependencies...
[INFO]  --> Exporting github.com/leprechau/glide-test2
[INFO]  --> Exporting github.com/Masterminds/vcs
[INFO]  --> Exporting github.com/couchbase/gocb
[INFO]  --> Exporting github.com/Masterminds/glide
[INFO]  --> Exporting gopkg.in/yaml.v2
[INFO]  Replacing existing vendor dependencies
[INFO]  Project relies on 5 dependencies.
charlie:glide-test1 ahurt$

@mattfarina
Copy link
Member

@leprechau I think I see what's going on. Several versions of Glide ago we made a change. Glide looks at the imports in the source and only follows those paths. That means if you have a nested package in a dependency that is never references in the import tree Glide skips fetching the dependencies for that since it's not needed.

There is a flag, --all-dependencies that will walk the everything. Though, I just discovered a bug in that. Working on that now.

@aaronhurt
Copy link
Author

@mattfarina This just broke for us in the 0.12.x release cycle and worked in previous versions. When designing applications with importable sub packages it's very handy to just have one glide.yaml in the root of the repository to manage all dependencies of the entire project.

@mattfarina
Copy link
Member

@leprechau in the example you shared, github.com/leprechau/glide-test2/foo and github.com/leprechau/glide-test2/bar are not imported. So, their dependencies are not fetched.

If I was in the directory for glide-test1 and ran:

$ glide up
$ go run main.go

This will work because Glide will get all the imports referenced (needed) by the code.

If github.com/leprechau/glide-test2 importedgithub.com/leprechau/glide-test2/barthen the dependencies ofbar would be fetched.

Using Glide 0.11.1 I just got the following...

$ glide up
[INFO]  Downloading dependencies. Please wait...
[INFO]  --> Fetching updates for github.com/leprechau/glide-test2.
[INFO]  Resolving imports
[INFO]  Downloading dependencies. Please wait...
[INFO]  Setting references for remaining imports
[INFO]  Versions did not change. Skipping glide.lock update.
[INFO]  Project relies on 1 dependencies.

To walk all the directory trees as opposed to the imports use the --all-dependencies flag.

$ glide up --all-dependencies

Please note #601. I plan to release 0.12.2 with fixes either Friday or Monday.

@aaronhurt
Copy link
Author

aaronhurt commented Sep 8, 2016 •
edited
Loading

@mattfarina Sorry, you're right I hadn't pushed my last example from where I had reproduced this. The code I had locally in glide-test1 was actually importing the two sub packages. Sorry for not pushing that last night.

import (
        "fmt"
        _ "github.com/leprechau/glide-test2/bar"
        _ "github.com/leprechau/glide-test2/foo"
)

It's updated in my example repository now. Sorry for the confusion.

@mattfarina
Copy link
Member

@leprechau "I think" I see your problem. Working on it before the next release. Thanks for your patience in explaining it.

@aaronhurt
Copy link
Author

@mattfarina Thank you and thank you for your work on glide. It's definitely improved our Go development process and become a crucial piece of our stack.

@mattfarina
Copy link
Member

PR is in #603. If this works for you then I think we're ready to roll this in and do a 0.12.2 release.

@mattfarina mattfarina added the bug label Sep 9, 2016
@mattfarina mattfarina added this to the 0.12.2 milestone Sep 9, 2016
@aaronhurt
Copy link
Author

aaronhurt commented Sep 9, 2016 •
edited
Loading

I'm still getting the same behavior on my test repo with the using the latest master with #603 applied.

Aarons-MacBook-Pro:glide-test1 ahurt$ rm -rf vendor/
Aarons-MacBook-Pro:glide-test1 ahurt$ /Users/ahurt/Projects/go/src/github.com/myENA/glide/glide -version
glide version 0.12.0-dev
Aarons-MacBook-Pro:glide-test1 ahurt$ /Users/ahurt/Projects/go/src/github.com/myENA/glide/glide cc
[INFO]  Glide cache has been cleared.
Aarons-MacBook-Pro:glide-test1 ahurt$ /Users/ahurt/Projects/go/src/github.com/myENA/glide/glide up
[INFO]  Downloading dependencies. Please wait...
[INFO]  --> Fetching github.com/leprechau/glide-test2.
[INFO]  Resolving imports
[INFO]  --> Fetching github.com/Masterminds/glide.
[INFO]  --> Fetching github.com/couchbase/gocb.
[INFO]  --> Fetching github.com/Masterminds/vcs.
[INFO]  --> Fetching gopkg.in/yaml.v2.
[INFO]  Downloading dependencies. Please wait...
[INFO]  Setting references for remaining imports
[INFO]  Exporting resolved dependencies...
[INFO]  --> Exporting github.com/leprechau/glide-test2
[INFO]  --> Exporting github.com/Masterminds/glide
[INFO]  --> Exporting github.com/couchbase/gocb
[INFO]  --> Exporting github.com/Masterminds/vcs
[INFO]  --> Exporting gopkg.in/yaml.v2
[INFO]  Replacing existing vendor dependencies
[INFO]  Versions did not change. Skipping glide.lock update.
[INFO]  Project relies on 5 dependencies.
Aarons-MacBook-Pro:glide-test1 ahurt$

mattfarina added a commit that referenced this issue Sep 12, 2016
@mattfarina
Merge pull request #603 from Masterminds/fix/nested-config-reading
3844a21 
Fixed #599: In some cases not importing dependencies config
mattfarina added a commit that referenced this issue Sep 12, 2016
@mattfarina
Fixed #599: In some cases not importing dependencies config
34f0775 
This was happening when a sub-package imported before the top level
or if the top level for the project not imported at all.
@ebaron ebaron mentioned this issue Dec 1, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
0.12.2
Development

No branches or pull requests
2 participants
@mattfarina @aaronhurt