pin flutter SDK version via git submodule

[chrisirhc]

• Introduce a git submodule to pin the version of flutter SDK under
  vendor/flutter.
• Use direnv to add vendor/flutter/bin to the current PATH so that
  subsequent calls to flutter on the shell uses the our pinned
  version of flutter.
• Update instructions to use pinned version of flutter.
• Pin to 18340ea16c of flutter which matches the current lowerbound
  version in pubsec.yaml (3.21.0-12.0.pre.26).

In addition, this PR also includes in a separate commit:

• ci-flutter-main job which runs on pushes, to validate the build with the latest main commit. (see Pin a precise version of Flutter #15 (comment) for context)

NOTE: Users can still choose to opt-out and use their own version of
flutter. This just provides a recommended and tested version on our
CI.

Remaining TODOs:

• Should envrc / direnv do a check to see if the flutter version matches expectations?
• Add Android Studio setup instructions

Closes #15

---

Trade-offs when using git submodule (this PR) compared to using fvm (#577)

Pros

• One fewer dependency and installation since it just depends on git
  • One less thing to learn/break.
• Does not depend on fvm which seems a bit clunky
• Easy to make contributions to flutter with the submodule setup once you find your way around submodules
• Few changes needed to CI and current setup process.
• flutter upgrade just works™️ , just need to commit the change of what the submodule commit is pointing to, which is simple.
• git tells us if the submodule is in the clean state

Cons

• Users need to run additional commands to setup submodule (This ends up being similar to commands to fvm, but we're not dealing with fvm's issues)
• Somewhat annoying that if you make any changes in the submodule, your work tree will be deemed "dirty". Perhaps a non-issue if there's some git command workaround.
  • Related https://stackoverflow.com/questions/31871888/what-goes-wrong-when-using-git-worktree-with-git-submodules indicates that there are some incompatibilities with git worktree.

[chrisirhc]

IMO this ends up being cleaner and still meets the goals of #15 .

[gnprice]

Thanks! I agree ­— I think this plan looks good. The main thing it needs is some more instructions, in particular on that Android Studio / IntelliJ setup we discussed around #577 (comment) .

The instructions for upgrading Flutter will also need some playtesting — I suspect the flutter upgrade step will want some kind of adjustment.

---

One peculiarity I notice, from playing with this, is that git submodule checks out the submodule at a detached HEAD, rather than at a named branch like main, and as a result flutter --version can't tell what channel it's coming from:

$ vendor/flutter/bin/flutter --version
Flutter 3.21.0-12.0.pre.26 • channel [user-branch] • unknown source
[…]

That contrasts with, for my existing main Flutter tree at a slightly older version:

$ ~/n/flutter/flutter/bin/flutter --version
Flutter 3.21.0-12.0.pre.29 • channel main • https://github.com/flutter/flutter.git
[…]

Not sure what-all downstream effects that might have. If it sufficiently confuses things, we might need to find a way to coax git submodule to keep things on a branch called main, rather than on detached HEADs. That or have a little script in tools/ that runs git submodule update --init and then fixes things up (e.g. with git checkout -B main in the submodule), or something like that.

---

Hmm, in fact, here's one downstream effect:

$ vendor/flutter/bin/flutter upgrade
Unable to upgrade Flutter: Your Flutter checkout is currently not on a release branch.
Use "flutter channel" to switch to an official channel, and retry. Alternatively, re-install Flutter by going to https://flutter.dev/docs/get-started/install.

So that use of a detached HEAD by git submodule breaks flutter upgrade.

I think taking flutter upgrade out of the instructions would be a fine solution to that — instead we can say something like git -C vendor/flutter pull --ff-only. We'll just need some kind of instructions that have been playtested to work.

[chrisirhc]

So that use of a detached HEAD by git submodule breaks flutter upgrade.

By adding a manual setup step to do git -C vendor/flutter checkout -B main so that the submodule is using a local main branch, we can let flutter it's on the main channel. This only needs to be done once. Subsequently, everything runs as expected with flutter upgrade. I've updated this PR to address that.

I'm now following up on the Android Studio investigation but it does look like manual instructions are needed.
Will updated the description of this PR shortly.

[chrisirhc]

Updated the PR, so it's ready for review.

[gnprice]

Thanks! Generally this looks great. Various comments below.

[chrisirhc]

Updated the PR and addressed the comments.

[gnprice]

Thanks again @chrisirhc for all your work on this! Sorry I haven't managed to return for another review for a bit — we've just this week gotten past the peak PR volume from GSoC application season.

I'll be going on vacation shortly for the next couple of weeks. I'll aim to pick this back up the week I'm back, as I catch up with things. Based on where it was at my last review, I imagine we'll be pretty close to merge. This will be a very nice improvement to have!

[gnprice]

Thanks @chrisirhc! Various small comments below. There's also one substantive question (about the resulting developer workflow) which I'll copy here for visibility:

What happens when as a contributor you rebase (or reset to a new origin/main), and the version that's pinned has changed? You'll need to run some git submodule command to update your local clone, right?

In fact, when I look at git help submodule, I'm not sure what an appropriate command would be. One obvious candidate is git submodule update… but I think that will take the submodule onto a detached HEAD, same as it does on first setup (as discussed above starting at #579 (comment)).

In that case we probably end up recommending the developer run a script like tools/setup-vendor-flutter any time git status tells them vendor/flutter is out of sync, or some sort of instruction like that.

---

Once we have all the phases of the main contributor workflow (so upgrade/rebase as well as setup) shaken out, I'll make one more editing pass over all the docs; and then I think it'll be time to play-test the instructions.

I'll handle that by setting up a branch with this PR plus an ad-hoc Flutter upgrade, in order to simulate that; then asking people to switch to the PR branch, follow the README instructions, then to the upgrade branch, and to test things work and give feedback on anything confusing.

[gnprice]

Similar to #579 (comment):

Let's try to write this message with the audience in mind of someone who's just cloned this repo and is starting to go through the README, and has never heard of Git submodules (and even if they have, doesn't know our particular layout or how we're using submodules).

So here:

Suggested change

	1.1. Make sure you have initialized submodule by doing:
	```sh
	# Update and initialize submodule
	git submodule update --init
	# Run setup-vendor-flutter
	tools/setup-vendor-flutter
	```
	1.1. Initialize the pinned Flutter tree:
	```sh
	git submodule update --init
	tools/setup-vendor-flutter
	```

[gnprice]

Relatedly: can we move the git submodule update --init inside the script, so that this step is just a single command?

Hmm, though on the other hand, and zooming out a bit: what happens when as a contributor you rebase (or reset to a new origin/main), and the version that's pinned has changed? You'll need to run some git submodule command to update your local clone, right?

If so, that seems like something we should mention in this doc too. But it also points to keeping an explicit git submodule step here in the setup after all, to avoid having something that's initially magic but where the developer later needs to look behind the magic anyway.

[chrisirhc]

Ah, yes, in my latest changes I've explored moving git submodule update --init into the setup-vendor-flutter script and making this script runnable any/all the time. If it's already set up, it'll be a noop.

Will take a stab at adding a note in the readme to run setup-vendor-flutter whenever user sees something different on vendor/flutter via git status.

[gnprice]

I've also rebased the PR branch, resolving a small merge conflict (foreshadowed at #579 (comment) 🙂 ) and updating the pinned version.

[chrisirhc]

Updated the PR and addressed the comments though I haven't squashed the commits yet.

One other thing to consider is to add a post-checkout git hook that runs setup-vendor-flutter whenever user does a checkout.
I don't want to add too much magic or over-scope this work, so leaving that out for now, but it might be something to consider based on feedback/play-testing.

A preview of the message if the setup-vendor-flutter fails, for review:

~/c/zulip-flutter (test-change-branch) [1]> ./tools/setup-vendor-flutter                                             (base) 
Initializing 'vendor/flutter' submodule:
================================================================
error: Your local changes to the following files would be overwritten by checkout:
        bin/internal/engine.version
Please commit your changes or stash them before you switch branches.
Aborting
fatal: Unable to checkout 'a46a7fdc3d622908c24c509ae48c874368c7eb12' in submodule path 'vendor/flutter'
================================================================
Failed to initialize the 'vendor/flutter' submodule, please resolve the issues indicated above.