Design: the jj init experience needs a rethink

[thoughtpolice]

From December 18th on Discord, while trying to follow up on #2512, I ended up stuck because the UX for initializing jj repositories is quite inconsistent. https://discord.com/channels/968932220549103686/969291218347524238/1186382367141670953

[12:59 PM] aseipp: I ran into #2512 this morning so I'm reviving it right now. I have a nit about the UI. For jj git clone, we can take a --colocate or --no-colocate flag. But if we do jj init, then the UX is totally different; a colocated repo is equivalent to jj init --destination=$D --git --git-repo=$D. Ok. So now what if git.colocate = true and then someone wants the equivalent of --no-colocate?

[1:00 PM] aseipp: I kind of feel like this strongly suggests that the functionality of jj init --git should be under jj git instead

[1:02 PM] aseipp: To elaborate, they would need to do something like jj init --git-repo=undefined or something. I also think -- at this exact moment, I am thinking about it more for the first time -- needing --git for jj init is actually a little strange, too

[1:06 PM] aseipp: I suppose the intent is jj init is able to initialize repositories for multiple backends. But that seems like it would bloat the UX and cause contradictions like this. I would basically have to rework the entire jj init --git UX for consistency with the jj git clone flow now, but then why even have it under jj init at that point, if I could just move it under jj git init for roughly the same amount of effort? For different backends I think the idea should be jj $FOO init is how you initialize repositories where your backend = $FOO. The jj native backend can simply be given the special title of jj init only, in contrast.

[1:07 PM] aseipp: We could also have jj init $FOO too. I'm not sure.

There was about 30 minutes of discussion or so but pretty much everyone I think is in agreement that jj init experience needs a bit of a rethink. #2512 is probably held up on this — as well as any other default options we might want to add for any given backend.

[martinvonz]

I agree. I added the "Good first issue" label because this seems pretty straightforward.

[essiene]

What's the consensus?

jj $backend init ?

I'm thinking of taking a stab at this. It feels like some larger refactoring than what I've touched so far will be required, so looks like a nice way to familiarise myself with internals.

[arxanas]

@essiene I think we want jj git init to be equivalent to current jj init --git-repo ., and then hide/remove the current jj init (or make it only initialize with the native backend; if we keep it, then it should probably warn if trying to initialize inside an existing Git repo).

[PhilipMetzger]

What's the consensus?

jj $backend init ?

Yes, that was the conclusion of the Discord discussion. Additionally a user-friendly message for jj init --git which says that the user should use jj git init instead.

[essiene]

Cool. Thanks for the clarifications. I'll assign this to myself, then write up a mini proposal to check my understanding of all that needs to change.

[essiene]

I'm trying to figure out what to do with the --git-repo option of jj init and I'm not sure I understand it properly. I don't use colocated repos, so this is purely me not understanding stuff. :-D

$ mkdir mygit
$ cd mygit
$ git init 
$ date > somefile
$ git add *
$ git commit -m 'commit'
$ cd ..
$ mkdir myjj1
$ jj init --git-repo=./mygit

What's the jj term for the repo in myjj1? I notice that I have to use jj git export/import to make changes available across both repos.

Now, what about:

$ mkdir mybaregit
$ cd mybaregit
$ git init --bare
$ date > somefile
$ git add *
$ git commit -m 'commit'
$ cd ..
$ jj git clone --colocate ./mybaregit myjj2

I believe myjj2 is a colocated repo? And I am able to push to the git repo. But what is different between myjj1 and myjj2 ?

[PhilipMetzger]

These are both "co-located" repositories, as both a .jj and .git exist in the toplevel directory. If you do a bare jj init --git, the internal Git repo will instead be placed under .jj/store/git/ IIRC.

[essiene]

Ahh.. Thanks.

So how about this:

jj git init

jj git init

init bare git repo; DESTINATION=.

jj git init ./path

init bare git repo; DESTINATION=./path

jj git init --colocated

init colocated git repo; --git-repo=. and DESTINATION=.

jj git init --colocated --git-repo=./gitpath

init colocated git repo; --git-repo=./gitpath and DESTINATION=.

jj git init --colocated --git-repo=./gitpath ./jjrepo

init colocated git repo; --git-repo=./gitpath and DESTINATION=./jjrepo

jj init

• Remove this? I don't like that idea. I think DVCS these days have an unspoken expectation of an init command to get rolling.
• Make this initialize to native backend? Maybe
• have a config option like init.backend; if this is set, use init that backend with default options. If not set, use the default backend and print a warning.

jj native init

• 'twould make sense for symmetry.
• Would mean a new command and new code that has to be maintained. Just mentioning for completeness

Thoughts?

[martinvonz]

I would expect --colocated to mean "make the .git a sibling to the .jj directory". For example, jj git init --colocate would create ./.jj and ./.git directories in the current directory, and jj git init --colocate /some/path would create /some/path/.jj and /some/path/.git. --colocate and --git-repo should probably be incompatible. Does that make sense?

[essiene]

And to finally check my understanding of that.

• git_backed_jj_repo:
  • colocated=true
    • jj=DESTINATION/.jj
    • git=DESTINATION/.git
  • colocated=false
    • use_default_git_repo_path
      • jj=DESTINATION/.jj
      • git=DESTINATION/.jj/repo/store/git
    • set --git-repo
      • jj=DESTINATION/.jj
      • git=$git_repo

If all the preceding is correct, then I believe I understand why --git-repo and --colocate should be mutually exclusive.

Correct?

[martinvonz]

Yep, that's all correct.

[essiene]

Thanks. I'll start by adding a jj git init command.

We still need to decide exactly what to do with the original jj init command, but a hint, pointing at jj git init should be sufficient in the short term.

[ilyagr]

If I understand correctly, jj git init --git-repo=somewhere_else will not turn on the automatic import and export of branches.

There will still be the special case of jj git init --git-repo=.. It might surprise users that (as currently implemented) that will turn on the automatic jj git import/export before/after every command. OTOH, at least at first, it would probably be good enough to either error out in this case or to print a message explaining this is equivalent to jj git init --colocate and will do the automatic import/export.

[martinvonz]

If I understand correctly, jj git init --git-repo=somewhere_else will not turn on the automatic import and export of branches.

Yes, that's correct. That mode predates colocated mode by quite a bit. It can still be useful in large repos because it lets you run git and jj commands on the repo without the automatic import/export (which can be slow on large repos).

OTOH, at least at first, it would probably be good enough to either error out in this case or to print a message explaining this is equivalent to jj git init --colocate and will do the automatic import/export.

Makes sense, but I don't think that's even a requirement for a first PR for jj git init since it doesn't seem any worse than what we have today. (Just clarifying that @essiene shouldn't feel obligated to implement that, but let me know if you do think it should be a requirement, @ilyagr).

[ilyagr]

I don't think this is mandatory. At first, we can just document --colocate as an alias for --git-repo=. in the help. We can make any further improvements later.

Aside: I thought jj git clone already ran into this issue, but no; it doesn't support --git-repo=PATH. IMO, that is fine. You can get the same effect with a jj git clone followed by jj init --git-repo=... somewhere else.

[essiene]

Post #2932 merge, some cleanups still left to do.

• What do we do about the jj init command itself?
  • Leave it in, to only deal with the native backend.
    The problem with this is that it will make other backends
    harder to discover.
  • Move native backend init to a jj native init (naming is hard!)
    or a better named subcommand. And completely delete jj init
    Are there other functionality that would fit under a native
    subcommand?
• We should normalize the user's view of colocated git repos.
  • Add flags like --allow-exiting and --autopropagate
    to jj git init --colocate. This would replace --git-repo=.
    and allow the user to turn control auto import/export if they
    want to. These can default to true (preferred) or false.
  • OR, just control the behavior from config like in cli/lib: add git.colocate to colocate repositories by default #2512, except
    the defaults are true and the user can switch them off.

Thoughts?

[yuja]

* What do we do about the `jj init` command itself?
  
  * Leave it in, to only deal with the native backend.
    The problem with this is that it will make other backends
    harder to discover.
  * Move native backend init to a `jj native init` (naming is hard!)
    or a better named subcommand. And completely delete `jj init`
    Are there other functionality that would fit under a `native`
    subcommand?

If we decide to remove jj init, maybe the native init function can be moved to jj debug init-native.

* We should normalize the user's view of `colocated` git repos.
  
  * Add flags like `--allow-exiting` and `--autopropagate`
    to `jj git init --colocate`. This would replace `--git-repo=.`
    and allow the user to turn control auto import/export if they
    want to. These can default to true (preferred) or false.

I often create jj repo for the existing git repo (by e.g. cargo new x && cd x && jj init --git-repo .), so it would be nice if that can be achieved with fewer options. So my vote is:

• no more flags
• let jj git init --colocated import existing .git repo if any
  (I don't have strong option, though.)

btw, there are jj git init --colocated (past) and jj git clone --colocate (present). Maybe we should pick one.

  * OR, just control the behavior from config like in [cli/lib: add `git.colocate` to colocate repositories by default #2512](https://github.com/martinvonz/jj/pull/2512), except
    the defaults are true and the user can switch them off.

I'm not sure about this. If we add config knob, we'll need to add --no-colocate flag as well. It also applies to the other flags. If we add --allow-existing and make it configurable, we'll need --no-allow-existing.

[PhilipMetzger]

• What do we do about the jj init command itself?

I really liked @chriskrycho's idea of making it interactive when we support multiple backends but that's probably a moment away.

• Leave it in, to only deal with the native backend.
  The problem with this is that it will make other backends
  harder to discover.
• Move native backend init to a jj native init (naming is hard!)
  or a better named subcommand. And completely delete jj init
  Are there other functionality that would fit under a native
  subcommand?

I think hiding or removing it until a second backend appears should be good enough. For consistency it could be worth it to just implement jj native init though.

We should normalize the user's view of colocated git repos.

• Add flags like --allow-exiting and --autopropagate
  to jj git init --colocate. This would replace --git-repo=.
  and allow the user to turn control auto import/export if they
  want to. These can default to true (preferred) or false.
• OR, just control the behavior from config like in cli/lib: add git.colocate to colocate repositories by default #2512, except
  the defaults are true and the user can switch them off.

I'm in favor of keeping jj git init --colocate as is and renaming --git-repo to something appropriate. We then can go through with #2512. I agree with Yuya to keep the flags as low as possible.

[NoahTheDuke]

To provide some feedback to #2932 (I'm using main @ 1be8225):

noah ~ 
$ git clone git@github.com:NoahTheDuke/dotfiles.git
Cloning into 'dotfiles'...
remote: Enumerating objects: 1137, done.
remote: Counting objects: 100% (353/353), done.
remote: Compressing objects: 100% (222/222), done.
remote: Total 1137 (delta 223), reused 234 (delta 109), pack-reused 784
Receiving objects: 100% (1137/1137), 6.91 MiB | 14.86 MiB/s, done.
Resolving deltas: 100% (643/643), done.
noah ~ 
$ cd dotfiles/
noah ~/dotfiles 
$ jj
Error: There is no jj repo in "."
Hint: It looks like this is a git repo. You can create a jj repo backed by it by running this:
jj init --git-repo=.
noah ~/dotfiles 
$ jj init
Error: The native backend is disallowed by default.
Hint: Did you mean to call `jj git init`?
Set `ui.allow-init-native` to allow initializing a repo with the native backend.
noah ~/dotfiles 
$ jj git init
Error: Did not create a jj repo because there is an existing Git repo in this directory.
Hint: To create a repo backed by the existing Git repo, run `jj git init --git-repo=.` instead.
noah ~/dotfiles 
$ jj init --git-repo=.
Done importing changes from the underlying Git repo.
The following remote branches aren't associated with the existing local branches:
  master@origin
Hint: Run `jj branch track master@origin` to keep local branches updated on future pulls.
warning: `--git` and `--git-repo` are deprecated.
Use `jj git init` instead
Initialized repo in "."
noah@H0H67M3 ~/dotfiles 
$ rm -rf .jj/
noah@H0H67M3 ~/dotfiles 
$ jj git init --git-repo=.
Done importing changes from the underlying Git repo.
The following remote branches aren't associated with the existing local branches:
  master@origin
Hint: Run `jj branch track master@origin` to keep local branches updated on future pulls.
Initialized repo in "."

I see that I misread the suggestion to run jj git init --git-repo=. as the earlier suggestion to run jj init --git-repo=., but even given that, there are a couple places where docs/flags could be improved:

1. jj suggests jj init --git-repo=..
2. jj init suggests jj git init with no flags.
3. jj git init suggests jj git init --git-repo=..
4. jj init --git-repo=. says that --git and --git-repo are deprecated and suggests jj git init with no flags.

It seems that jj git init --git-repo=. is correct and given the above comments, --git-repo=. will thankfully become --colocate which removes the redundancy ("why am I specifying git twice?"), but unifying all of the messaging to the same command is probably smart.

[essiene]

Thanks for testing. I didn't bother checking for other files that may refer to jj init... D'oh! My bad.

I'll go clean it up.

[martinvonz]

I'll go clean it up.

I think that's already done in #2983

[essiene]

Yay! \o/

[yuja]

jj init --git-repo=. says that --git and --git-repo are deprecated and suggests jj git init with no flags.

I haven't updated this part. It's not wrong, but might be better to suggest jj git init --git-repo={git_repo} if the argument is specified.

[JackTheMico]

I just realized this issue when I was trying to jj git remote add <to one of my github repo> and it failed because I used jj init to init the repo.