Use inferred flutter version #5207

sigurdm · 2022-05-31T14:30:36Z

This reorganizes the pub integration somewhat:

The pub integration dependency_services now is a dart package as a helper inside dependabot (with a git dependency on the pub client)
These helpers are compiled in the Dockerfile by a build script.
The version of flutter/dart to use is inferred by another helper.
Flutter is no longer installed by the dockerfile, but dynamically checked out into /tmp/flutter, where the correct flutter version is checked out.

Also the dart sdk is updated (this is less important, as the sdk version used for analysis is will be the one associated with the flutter version).

fixes: #5084

Installs the flutter version dynamically instead of fixed in the docker-file. Also sets up pub helpers. Specifically the executable for dependency_services now lives in dependabot.

sigurdm · 2022-05-31T14:31:05Z

@mctofu does this approach make sense to you?
cc @jonasfj

jonasfj · 2022-05-31T14:41:07Z

The actual dependency-services is still implemented in the pub repository, we just moved the bin/dependency_services.dart file that exports it in here. That makes it more clear that this is really reaching into the pub repository for internal stuff.

But that's fine, one day when dependency-services is more formalized maybe we can expose it properly and document it as part of a Dart SDK. For now, the dependency-services logic we have in pub is only intended for use by dependabot.

off-topic: or maybe dependency-services should be an LSP extension of sorts)

jeffwidman · 2022-05-31T18:55:59Z

cc @mattt in case this impacts #5024 related stuff.

mctofu

@mctofu does this approach make sense to you?

This does make sense to me. I'm going to find some time to give this a test run this week to get a better idea of how it's working.

There are some CI tasks failing on this branch. You can fix the npm CI errors by updating with the latest changes on main. The pub errors look to be from the linter which you can fix by running rubocop -A

pub/README.md

pub/lib/dependabot/pub/helpers.rb

mctofu · 2022-06-01T21:17:54Z

pub/lib/dependabot/pub/helpers.rb

+          "git",
+          "clone",
+          "--no-checkout",
+          # "--reference",


minor: it'd be best to remove this if we won't use it

pub/helpers/bin/dependency_services.dart

Co-authored-by: David McIntosh <804610+mctofu@users.noreply.github.com>

sigurdm · 2022-06-03T13:58:15Z

PTAL

passsy · 2022-06-07T20:49:39Z

pub/helpers/bin/infer_sdk_versions.dart

+      .toList();
+}
+
+/// The "best" Flutter release for a given set of constraints is the first one


does "first" mean: oldest or newest Flutter version?

It is the first in the list passed to this function.

I tried to rephrase a bit.

passsy · 2022-06-07T20:58:26Z

pub/helpers/bin/infer_sdk_versions.dart

+    final pubspec = loadYaml(File(pubspecPath).readAsStringSync(),
+        sourceUrl: Uri.file(pubspecPath));


Inferring is a good start, but dependabot should allow pointing to an exact git-ref (tag, sha1).

Hmm - do you have an idea for a good place to put this configuration?

What are scenarios where you would need this?

Scenario: We use flutterw to pin the flutter version. Others use fvm or custom solutions.

Tree options pop into my mind:

Lookup: dependabot checks the version of those pinning solutions (i.e. check git submodule sha1 or a configuration file)

Add a flutter version to .github/dependabot.yaml

Define a new dependabot.flutter_version property to pubspec.yaml of each package and read it

cc @jonasfj what do you think?

I think that it sufficient that we allow pinning the Flutter SDK constraint in environment.flutter.

Generally, the dart pub client will ignore the upper-bound in environment.flutter, as a result of a decision made in Flutter 2 that Flutter wouldn't break backwards compatibility.

But in dependabot, I think we made it respect the upper-bound, thus, the environment.flutter constraint can be used to pin a Flutter version. You can't pin to arbitrary sha or tag, but you can pin to stable and beta releases of Flutter.

My two cents is that this is a reasonable compromise.

Using the lower bound of environment.flutter would work for us. But it could prevent dependabot from adding the latest versions.

Using the upper bound wouldn't be practical. We'd have to change it every time we upgrade in all packages of our mono repo.

mctofu

@sigurdm: I'm trying to run this locally and running into an error. Do you get the same thing or have I done something wrong?

[dependabot-core-dev] ~/dependabot-core $ bin/dry-run.rb pub passsy/flutter_dependabot_example
warning: parser/current is loading parser/ruby27, which recognizes2.7.6-compliant syntax, but you are running 2.7.5.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
=> cloning into /home/dependabot/dependabot-core/tmp/passsy/flutter_dependabot_example
=> parsing dependency files
=> updating 5 dependencies: cupertino_icons, flutter, flutter_lints, flutter_test, go_router

=== cupertino_icons (1.0.4)
 => checking for updates 1/5
Traceback (most recent call last):
	15: from bin/dry-run.rb:635:in `<main>'
	14: from bin/dry-run.rb:635:in `each'
	13: from bin/dry-run.rb:643:in `block in <main>'
	12: from /home/dependabot/dependabot-core/pub/lib/dependabot/pub/update_checker.rb:13:in `latest_version'
	11: from /home/dependabot/dependabot-core/pub/lib/dependabot/pub/update_checker.rb:101:in `current_report'
	10: from /home/dependabot/dependabot-core/pub/lib/dependabot/pub/update_checker.rb:97:in `report'
	 9: from /home/dependabot/dependabot-core/pub/lib/dependabot/pub/helpers.rb:44:in `dependency_services_report'
	 8: from /home/dependabot/dependabot-core/pub/lib/dependabot/pub/helpers.rb:150:in `run_dependency_services'
	 7: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:49:in `in_a_temporary_directory'
	 6: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:49:in `chdir'
	 5: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:49:in `block in in_a_temporary_directory'
	 4: from /home/dependabot/dependabot-core/pub/lib/dependabot/pub/helpers.rb:157:in `block in run_dependency_services'
	 3: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:168:in `with_git_configured'
	 2: from /home/dependabot/dependabot-core/pub/lib/dependabot/pub/helpers.rb:167:in `block (2 levels) in run_dependency_services'
	 1: from /home/dependabot/dependabot-core/pub/lib/dependabot/pub/helpers.rb:167:in `chdir'
/home/dependabot/dependabot-core/pub/lib/dependabot/pub/helpers.rb:174:in `block (3 levels) in run_dependency_services': dependency_services failed: Package not available (the Flutter SDK is not available). (Dependabot::DependabotError)

sigurdm · 2022-06-09T13:11:07Z

@mctofu sorry - I had not tested this thoroughly. I was downloading the flutter sdk, but never using it from the resolution.

Now I've added a test of update-checking a package actually using flutter. PTAL

sigurdm · 2022-06-09T13:12:36Z

Now we get:

$ ../bin/dry-run.rb pub passsy/flutter_dependabot_example
warning: parser/current is loading parser/ruby27, which recognizes2.7.6-compliant syntax, but you are running 2.7.5.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
=> cloning into /home/dependabot/dependabot-core/pub/tmp/passsy/flutter_dependabot_example
=> parsing dependency files
=> updating 5 dependencies: cupertino_icons, flutter, flutter_lints, flutter_test, go_router

=== cupertino_icons (1.0.4)
 => checking for updates 1/5
 => latest available version is 1.0.5
 => latest allowed version is 1.0.5
 => requirements to unlock: own
 => requirements update strategy: 
 => updating cupertino_icons from 1.0.4 to 1.0.5

    ± pubspec.yaml
    ~~~
    12c12
    <   cupertino_icons: ^1.0.2
    ---
    >   cupertino_icons: ^1.0.5
    ~~~

    ± pubspec.lock
    ~~~
    45c45
    <     version: "1.15.0"
    ---
    >     version: "1.16.0"
    52c52
    <     version: "1.0.4"
    ---
    >     version: "1.0.5"
    59c59
    <     version: "1.2.0"
    ---
    >     version: "1.3.0"
    123c123
    <     version: "0.1.3"
    ---
    >     version: "0.1.4"
    149c149
    <     version: "1.8.1"
    ---
    >     version: "1.8.2"
    185,191d184
    <   typed_data:
    <     dependency: transitive
    <     description:
    <       name: typed_data
    <       url: "https://pub.dartlang.org"
    <     source: hosted
    <     version: "1.3.0"
    198c191
    <     version: "2.1.1"
    ---
    >     version: "2.1.2"
    200c193
    <   dart: ">=2.16.2 <3.0.0"
    ---
    >   dart: ">=2.17.0-0 <3.0.0"
    ~~~

=== flutter (0.0.0)
 => checking for updates 2/5
 => latest available version is 0.0.0
 => latest allowed version is 0.0.0
    (no update needed as it's already up-to-date)

=== flutter_lints (1.0.4)
 => checking for updates 3/5
 => latest available version is 2.0.1
 => latest allowed version is 2.0.1
 => requirements to unlock: own
 => requirements update strategy: 
 => updating flutter_lints from 1.0.4 to 2.0.1

    ± pubspec.yaml
    ~~~
    18c18
    <   flutter_lints: ^1.0.0
    ---
    >   flutter_lints: ^2.0.1
    ~~~

    ± pubspec.lock
    ~~~
    45c45
    <     version: "1.15.0"
    ---
    >     version: "1.16.0"
    59c59
    <     version: "1.2.0"
    ---
    >     version: "1.3.0"
    71c71
    <     version: "1.0.4"
    ---
    >     version: "2.0.1"
    102c102
    <     version: "1.0.1"
    ---
    >     version: "2.0.0"
    123c123
    <     version: "0.1.3"
    ---
    >     version: "0.1.4"
    149c149
    <     version: "1.8.1"
    ---
    >     version: "1.8.2"
    185,191d184
    <   typed_data:
    <     dependency: transitive
    <     description:
    <       name: typed_data
    <       url: "https://pub.dartlang.org"
    <     source: hosted
    <     version: "1.3.0"
    198c191
    <     version: "2.1.1"
    ---
    >     version: "2.1.2"
    200c193
    <   dart: ">=2.16.2 <3.0.0"
    ---
    >   dart: ">=2.17.0-206.0.dev <3.0.0"
    ~~~

=== flutter_test (0.0.0)
 => checking for updates 4/5
 => latest available version is 0.0.0
 => latest allowed version is 0.0.0
    (no update needed as it's already up-to-date)

=== go_router (3.0.5)
 => checking for updates 5/5
 => latest available version is 3.1.1
 => latest allowed version is 3.1.1
 => requirements to unlock: own
 => requirements update strategy: 
 => updating go_router from 3.0.5 to 3.1.1

    ± pubspec.yaml
    ~~~
    13c13
    <   go_router: 3.0.5
    ---
    >   go_router: ^3.1.1
    ~~~

    ± pubspec.lock
    ~~~
    45c45
    <     version: "1.15.0"
    ---
    >     version: "1.16.0"
    59c59
    <     version: "1.2.0"
    ---
    >     version: "1.3.0"
    88c88
    <     version: "3.0.5"
    ---
    >     version: "3.1.1"
    123c123
    <     version: "0.1.3"
    ---
    >     version: "0.1.4"
    149c149
    <     version: "1.8.1"
    ---
    >     version: "1.8.2"
    185,191d184
    <   typed_data:
    <     dependency: transitive
    <     description:
    <       name: typed_data
    <       url: "https://pub.dartlang.org"
    <     source: hosted
    <     version: "1.3.0"
    198c191
    <     version: "2.1.1"
    ---
    >     version: "2.1.2"
    200c193
    <   dart: ">=2.16.2 <3.0.0"
    ---
    >   dart: ">=2.17.0-0 <3.0.0"
    ~~~
🌍 Total requests made: '0'

mctofu · 2022-06-09T23:49:13Z

@sigurdm: Thanks, it's working locally for me now! There are a few tests failing now in pub/spec/dependabot/pub/infer_sdk_versions_spec.rb but overall it looks good!

mctofu

Thanks! I think this looks good to merge but let me know if you'd want to make any other changes first.

jonasfj

LGTM

pub/README.md

pub/helpers/pubspec.yaml

Co-authored-by: Jonas Finnemann Jensen <jopsen@gmail.com>

sigurdm · 2022-06-13T10:22:30Z

@mctofu I think I'm happy now.

sigurdm added 2 commits May 31, 2022 10:09

Infer the flutter version from pubspec.yaml

c5eb881

Installs the flutter version dynamically instead of fixed in the docker-file. Also sets up pub helpers. Specifically the executable for dependency_services now lives in dependabot.

Test of infer_sdk_versions tool

4841bee

mctofu reviewed Jun 1, 2022

View reviewed changes

sigurdm and others added 6 commits June 2, 2022 09:29

Update pub/README.md

a0d214e

Co-authored-by: David McIntosh <804610+mctofu@users.noreply.github.com>

lints

f1149cb

Remove commented code

6ba8da1

merge to main

473fddc

Scoping

3f9cc76

Work around flutter/flutter#54014

c68388f

sigurdm marked this pull request as ready for review June 7, 2022 07:21

sigurdm requested a review from a team June 7, 2022 07:21

sigurdm requested a review from a team as a code owner June 7, 2022 07:21

sigurdm requested a review from mctofu June 7, 2022 07:21

passsy reviewed Jun 7, 2022

View reviewed changes

mctofu reviewed Jun 7, 2022

View reviewed changes

sigurdm added 2 commits June 9, 2022 13:05

Actually use the flutter downloaded

9d368da

Typo

7d43821

Handle empty url given

a43dc29

Pull fixtures from right location

37065ae

mctofu approved these changes Jun 11, 2022

View reviewed changes

trailing newline in .gitignore

fbcf171

jonasfj reviewed Jun 13, 2022

View reviewed changes

pub/README.md Outdated Show resolved Hide resolved

pub/README.md Outdated Show resolved Hide resolved

pub/README.md Outdated Show resolved Hide resolved

pub/README.md Outdated Show resolved Hide resolved

pub/helpers/pubspec.yaml Outdated Show resolved Hide resolved

sigurdm and others added 6 commits June 13, 2022 10:19

Delete stray files

d08618d

Final newline in pubspec

65ebd08

Typo

befa583

Co-authored-by: Jonas Finnemann Jensen <jopsen@gmail.com>

Typo

be3113d

Co-authored-by: Jonas Finnemann Jensen <jopsen@gmail.com>

Typo

a5f0c57

Co-authored-by: Jonas Finnemann Jensen <jopsen@gmail.com>

Markdown formatting

d3fe6ce

Co-authored-by: Jonas Finnemann Jensen <jopsen@gmail.com>

mctofu merged commit 0f24b5a into dependabot:main Jun 13, 2022

This was referenced Jun 13, 2022

v0.192.0 #5256

Merged

Pub error: Unsupported operation: Trying to serialize a root package description. #5199

Closed

Bump pub revision to fix error without lockfile #5263

Merged

sigurdm mentioned this pull request Jun 16, 2022

[pub] Dependabot docker image doesn't update Dart 2.17 projects #5240

Closed

passsy mentioned this pull request Jun 17, 2022

Unable to parse Flutter version #5283

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use inferred flutter version #5207

Use inferred flutter version #5207

sigurdm commented May 31, 2022 •

edited

Loading

sigurdm commented May 31, 2022

jonasfj commented May 31, 2022 •

edited

Loading

jeffwidman commented May 31, 2022

mctofu left a comment

mctofu Jun 1, 2022

sigurdm Jun 2, 2022

sigurdm commented Jun 3, 2022

passsy Jun 7, 2022

sigurdm Jun 13, 2022

passsy Jun 7, 2022

sigurdm Jun 9, 2022

passsy Jun 9, 2022

sigurdm Jun 9, 2022

jonasfj Jun 10, 2022 •

edited

Loading

passsy Jun 10, 2022

mctofu left a comment

sigurdm commented Jun 9, 2022

sigurdm commented Jun 9, 2022

mctofu commented Jun 9, 2022

mctofu left a comment

jonasfj left a comment

sigurdm commented Jun 13, 2022

		final pubspec = loadYaml(File(pubspecPath).readAsStringSync(),
		sourceUrl: Uri.file(pubspecPath));

Use inferred flutter version #5207

Use inferred flutter version #5207

Conversation

sigurdm commented May 31, 2022 • edited Loading

sigurdm commented May 31, 2022

jonasfj commented May 31, 2022 • edited Loading

jeffwidman commented May 31, 2022

mctofu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sigurdm commented Jun 3, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonasfj Jun 10, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mctofu left a comment

Choose a reason for hiding this comment

sigurdm commented Jun 9, 2022

sigurdm commented Jun 9, 2022

mctofu commented Jun 9, 2022

mctofu left a comment

Choose a reason for hiding this comment

jonasfj left a comment

Choose a reason for hiding this comment

sigurdm commented Jun 13, 2022

sigurdm commented May 31, 2022 •

edited

Loading

jonasfj commented May 31, 2022 •

edited

Loading

jonasfj Jun 10, 2022 •

edited

Loading