-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bundler 2: [Prerelease] Add version resolver #3319
Merged
Merged
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
55460c7
Bundler 2: Add version resolver native helper
feelepxyz ffa5288
Fix required ruby version monkey patch
feelepxyz 02e27c6
Remove bundler 2 specs from update checker and file updater
feelepxyz eff6206
Add bundler 2 specific tets
feelepxyz 6a68837
Merge branch 'main' into feelepxyz/bundler2-version-resolver
jurre File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,140 @@ | ||
module Functions | ||
class VersionResolver | ||
GEM_NOT_FOUND_ERROR_REGEX = /locked to (?<name>[^\s]+) \(/.freeze | ||
|
||
attr_reader :dependency_name, :dependency_requirements, | ||
:gemfile_name, :lockfile_name | ||
|
||
def initialize(dependency_name:, dependency_requirements:, | ||
gemfile_name:, lockfile_name:) | ||
@dependency_name = dependency_name | ||
@dependency_requirements = dependency_requirements | ||
@gemfile_name = gemfile_name | ||
@lockfile_name = lockfile_name | ||
end | ||
|
||
def version_details | ||
dep = dependency_from_definition | ||
|
||
# If the dependency wasn't found in the definition, but *is* | ||
# included in a gemspec, it's because the Gemfile didn't import | ||
# the gemspec. This is unusual, but the correct behaviour if/when | ||
# it happens is to behave as if the repo was gemspec-only. | ||
if dep.nil? && dependency_requirements.any? | ||
return "latest" | ||
end | ||
|
||
# Otherwise, if the dependency wasn't found it's because it is a | ||
# subdependency that was removed when attempting to update it. | ||
return nil if dep.nil? | ||
|
||
# If the dependency is Bundler itself then we can't trust the | ||
# version that has been returned (it's the version Dependabot is | ||
# running on, rather than the true latest resolvable version). | ||
return nil if dep.name == "bundler" | ||
|
||
details = { | ||
version: dep.version, | ||
ruby_version: ruby_version, | ||
fetcher: fetcher_class(dep) | ||
} | ||
if dep.source.instance_of?(::Bundler::Source::Git) | ||
details[:commit_sha] = dep.source.revision | ||
end | ||
details | ||
end | ||
|
||
private | ||
|
||
# rubocop:disable Metrics/PerceivedComplexity | ||
def dependency_from_definition(unlock_subdependencies: true) | ||
dependencies_to_unlock = [dependency_name] | ||
dependencies_to_unlock += subdependencies if unlock_subdependencies | ||
begin | ||
definition = build_definition(dependencies_to_unlock) | ||
definition.resolve_remotely! | ||
rescue ::Bundler::GemNotFound => e | ||
unlock_yanked_gem(dependencies_to_unlock, e) && retry | ||
rescue ::Bundler::HTTPError => e | ||
# Retry network errors | ||
# Note: in_a_native_bundler_context will also retry `Bundler::HTTPError` errors | ||
# up to three times meaning we'll end up retrying this error up to six times | ||
# TODO: Could we get rid of this retry logic and only rely on | ||
# SharedBundlerHelpers.in_a_native_bundler_context | ||
attempt ||= 1 | ||
attempt += 1 | ||
raise if attempt > 3 || !e.message.include?("Network error") | ||
|
||
retry | ||
end | ||
|
||
dep = definition.resolve.find { |d| d.name == dependency_name } | ||
return dep if dep | ||
return if dependency_requirements.any? || !unlock_subdependencies | ||
|
||
# If no definition was found and we're updating a sub-dependency, | ||
# try again but without unlocking any other sub-dependencies | ||
dependency_from_definition(unlock_subdependencies: false) | ||
end | ||
# rubocop:enable Metrics/PerceivedComplexity | ||
|
||
def subdependencies | ||
# If there's no lockfile we don't need to worry about | ||
# subdependencies | ||
return [] unless lockfile | ||
|
||
all_deps = ::Bundler::LockfileParser.new(lockfile). | ||
specs.map(&:name).map(&:to_s).uniq | ||
top_level = build_definition([]).dependencies. | ||
map(&:name).map(&:to_s) | ||
|
||
all_deps - top_level | ||
end | ||
|
||
def build_definition(dependencies_to_unlock) | ||
# Note: we lock shared dependencies to avoid any top-level | ||
# dependencies getting unlocked (which would happen if they were | ||
# also subdependencies of the dependency being unlocked) | ||
::Bundler::Definition.build( | ||
gemfile_name, | ||
lockfile_name, | ||
gems: dependencies_to_unlock, | ||
lock_shared_dependencies: true | ||
) | ||
end | ||
|
||
def unlock_yanked_gem(dependencies_to_unlock, error) | ||
raise unless error.message.match?(GEM_NOT_FOUND_ERROR_REGEX) | ||
|
||
gem_name = error.message.match(GEM_NOT_FOUND_ERROR_REGEX). | ||
named_captures["name"] | ||
raise if dependencies_to_unlock.include?(gem_name) | ||
|
||
dependencies_to_unlock << gem_name | ||
end | ||
|
||
def lockfile | ||
return @lockfile if defined?(@lockfile) | ||
|
||
@lockfile = | ||
begin | ||
return unless lockfile_name | ||
return unless File.exist?(lockfile_name) | ||
|
||
File.read(lockfile_name) | ||
end | ||
end | ||
|
||
def fetcher_class(dep) | ||
return unless dep.source.is_a?(::Bundler::Source::Rubygems) | ||
|
||
dep.source.fetchers.first.fetchers.first.class.to_s | ||
end | ||
|
||
def ruby_version | ||
return nil unless gemfile_name | ||
|
||
@ruby_version ||= build_definition([]).ruby_version&.gem_version | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
97 changes: 97 additions & 0 deletions
97
bundler/helpers/v2/spec/functions/version_resolver_spec.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,97 @@ | ||
# frozen_string_literal: true | ||
|
||
require "native_spec_helper" | ||
require "shared_contexts" | ||
|
||
RSpec.describe Functions::VersionResolver do | ||
include_context "in a temporary bundler directory" | ||
include_context "stub rubygems compact index" | ||
|
||
let(:version_resolver) do | ||
described_class.new( | ||
dependency_name: dependency_name, | ||
dependency_requirements: dependency_requirements, | ||
gemfile_name: "Gemfile", | ||
lockfile_name: "Gemfile.lock" | ||
) | ||
end | ||
|
||
let(:dependency_name) { "business" } | ||
let(:dependency_requirements) do | ||
[{ | ||
file: "Gemfile", | ||
requirement: requirement_string, | ||
groups: [], | ||
source: source | ||
}] | ||
end | ||
let(:source) { nil } | ||
|
||
let(:rubygems_url) { "https://index.rubygems.org/api/v1/" } | ||
let(:old_index_url) { rubygems_url + "dependencies" } | ||
|
||
describe "#version_details" do | ||
subject do | ||
in_tmp_folder { version_resolver.version_details } | ||
end | ||
|
||
let(:project_name) { "gemfile" } | ||
let(:requirement_string) { " >= 0" } | ||
|
||
its([:version]) { is_expected.to eq(Gem::Version.new("1.4.0")) } | ||
its([:fetcher]) { is_expected.to eq("Bundler::Fetcher::CompactIndex") } | ||
|
||
context "with a private gemserver source" do | ||
include_context "stub rubygems compact index" | ||
|
||
let(:project_name) { "specified_source" } | ||
let(:requirement_string) { ">= 0" } | ||
|
||
before do | ||
gemfury_url = "https://repo.fury.io/greysteil/" | ||
gemfury_deps_url = gemfury_url + "api/v1/dependencies" | ||
|
||
stub_request(:get, gemfury_url + "versions"). | ||
to_return(status: 200, body: fixture("ruby", "gemfury-index")) | ||
stub_request(:get, gemfury_url + "info/business").to_return(status: 404) | ||
stub_request(:get, gemfury_deps_url).to_return(status: 200) | ||
stub_request(:get, gemfury_deps_url + "?gems=business,statesman"). | ||
to_return(status: 200, body: fixture("ruby", "gemfury_response")) | ||
stub_request(:get, gemfury_deps_url + "?gems=business"). | ||
to_return(status: 200, body: fixture("ruby", "gemfury_response")) | ||
stub_request(:get, gemfury_deps_url + "?gems=statesman"). | ||
to_return(status: 200, body: fixture("ruby", "gemfury_response")) | ||
end | ||
|
||
its([:version]) { is_expected.to eq(Gem::Version.new("1.9.0")) } | ||
its([:fetcher]) { is_expected.to eq("Bundler::Fetcher::Dependency") } | ||
end | ||
|
||
context "with a git source" do | ||
let(:project_name) { "git_source" } | ||
|
||
its([:version]) { is_expected.to eq(Gem::Version.new("1.6.0")) } | ||
its([:fetcher]) { is_expected.to be_nil } | ||
end | ||
|
||
context "when Bundler's compact index is down" do | ||
before do | ||
stub_request(:get, "https://index.rubygems.org/versions"). | ||
to_return(status: 500, body: "We'll be back soon") | ||
stub_request(:get, "https://index.rubygems.org/info/public_suffix"). | ||
to_return(status: 500, body: "We'll be back soon") | ||
stub_request(:get, old_index_url).to_return(status: 200) | ||
stub_request(:get, old_index_url + "?gems=business,statesman"). | ||
to_return( | ||
status: 200, | ||
body: fixture("ruby", | ||
"rubygems_responses", | ||
"dependencies-default-gemfile") | ||
) | ||
end | ||
|
||
its([:version]) { is_expected.to eq(Gem::Version.new("1.4.0")) } | ||
its([:fetcher]) { is_expected.to eq("Bundler::Fetcher::Dependency") } | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1708,29 +1708,4 @@ | |
end | ||
end | ||
end | ||
|
||
context "with bundler 2 support enabled" do | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These specs fail because the v2 version resolver is used and returns errors. |
||
let(:updater) do | ||
described_class.new( | ||
dependency_files: dependency_files, | ||
dependencies: dependencies, | ||
credentials: [{ | ||
"type" => "git_source", | ||
"host" => "github.com" | ||
}], | ||
repo_contents_path: repo_contents_path, | ||
options: { | ||
bundler_2_available: true | ||
} | ||
) | ||
end | ||
|
||
describe "updated_dependency_files" do | ||
it "fails as the native helper is not yet implemented" do | ||
expect { updater.updated_dependency_files }. | ||
to raise_error(Dependabot::NotImplemented, | ||
"Bundler 2 adapter does not yet implement update_lockfile") | ||
end | ||
end | ||
end | ||
end |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like it changed here: rubygems/rubygems@4e950f5