Add support for symlinks on Windows #2

martinvonz · 2021-01-08T19:44:43Z

We currently don't support symlinks on Windows (in fact, the project doesn't even build on Windows because of that). The advice I got from augie@google.com and former hg contributor "bmp" was (please correct me if I'm wrong):

Prior to Windows 10: No symlink support
Windows 10 without dev mode: No symlink support
Windows 10 dev mode: use FFI to call CreateSymbolicLink
Don't use the symlink creates, because it creates "junctions", which is not what we want.

Rust's standard library has std::os::windows::fs::symlink_file and std::os::windows::fs::symlink_dir, which both seem to call CreateSymbolicLinkW, only with different flags. Perhaps we can always use the symlink_file version? It's still unclear to me what the effect would be if a "file symbolic link" points to a target that's actually a directory.

The text was updated successfully, but these errors were encountered:

bpollack · 2021-01-08T19:54:10Z

Correct enough for government purposes.

To clarify, not to be annoying, but just in case it changes anything: Windows supports symlinks when Developer Mode isn't enabled, and has done so going all the way back to at least Vista. What actually changed was that, prior to Windows 10, you needed to be an admin to create them. The only (albeit major) thing Windows 10 with Developer Mode changes is that normal users can now create symlinks, rather than either needing to run a process as an admin or mucking about with group policy options (GPOs).

For reasons I'm not clear on, the underlying API for this (CreateSymbolicLinkW) still requires you to say whether you're linking a file or a directory. I have absolutely no idea why, but I don't //think// (from two minutes of mucking about) you can cheat, just pass 0x0, and be happy.

martinvonz · 2021-01-08T20:29:21Z

I think 0x0 is what std::os::windows::fs::symlink_file passes, so I guess we'll try to do that whether the target is a file or a directory. I don't have access to a Windows machine, but it sounded like Augie might be able to attempt it later. Thanks!

quark-zju · 2021-02-01T23:52:17Z

Based on tests using mklink /D vs mklink, I think symlink_dir is the one to be used. It seems to handle both the directory and file cases, while directory symlink crated by symlink_file won't allow cd into it.

martinvonz · 2021-02-02T07:16:32Z

Thanks for checking!

For example, ``` <<<<<<< Conflict 1 of 3 +++++++ Contents of side #1 left 3.1 left 3.2 left 3.3 %%%%%%% Changes from base to side #2 -line 3 +right 3.1 >>>>>>> ``` or ``` <<<<<<< Conflict 1 of 1 %%%%%%% Changes from base to side #1 -line 3 +right 3.1 +++++++ Contents of side #2 left 3.1 left 3.2 left 3.3 >>>>>>> ``` Currently, there is no way to disable these, this is TODO for a future PR. Other TODOs for future PRs: make these labels configurable. After that, we could support a `diff3/git`-like conflict format as well, in principle. Counting conflicts helps with knowing whether you fixed all the conflicts while you are in the editor. While labeling "side #1", etc, does not tell you the commit id or description as requested in #1176, I still think it's an improvement. Most importantly, I hope this will make `jj`'s conflict format less scary-looking for new users. I've used this for a bit, and I like it. Without the labels, I would see that the two conflicts have a different order of conflict markers, but I wouldn't be able to remember what that means. For longer diffs, it can be tricky for me to quickly tell that it's a diff as opposed to one of the sides. This also creates some hope of being able to navigate a conflict with more than 2 sides. Another not-so-secret goal for this is explained in #3109 (comment). The idea is a little weird, but I *think* it could be helpful, and I'd like to experiment with it.

Adds a new "ui.conflict-marker-style" config option. The "diff" option is the default jj-style conflict markers with a snapshot and a series of diffs to apply to the snapshot. New conflict marker style options will be added in later commits. The majority of the changes in this commit are from passing the config option down to the code that materializes the conflicts. Example of "diff" conflict markers: ``` <<<<<<< Conflict 1 of 1 +++++++ Contents of side jj-vcs#1 fn example(word: String) { println!("word is {word}"); %%%%%%% Changes from base to side jj-vcs#2 -fn example(w: String) { +fn example(w: &str) { println!("word is {w}"); >>>>>>> Conflict 1 of 1 ends } ```

Adds a new "snapshot" conflict marker style which returns a series of snapshots, similar to Git's "diff3" conflict style. The "snapshot" option uses a subset of the conflict hunk headers as the "diff" option (it just doesn't use "%%%%%%%"), meaning that the two options are trivially compatible with each other (i.e. a file materialized with "snapshot" can be parsed with "diff" and vice versa). Example of "snapshot" conflict markers: ``` <<<<<<< Conflict 1 of 1 +++++++ Contents of side jj-vcs#1 fn example(word: String) { println!("word is {word}"); ------- Contents of base fn example(w: String) { println!("word is {w}"); +++++++ Contents of side jj-vcs#2 fn example(w: &str) { println!("word is {w}"); >>>>>>> Conflict 1 of 1 ends } ```

Adds a new "ui.conflict-marker-style" config option. The "diff" option is the default jj-style conflict markers with a snapshot and a series of diffs to apply to the snapshot. New conflict marker style options will be added in later commits. The majority of the changes in this commit are from passing the config option down to the code that materializes the conflicts. Example of "diff" conflict markers: ``` <<<<<<< Conflict 1 of 1 +++++++ Contents of side jj-vcs#1 fn example(word: String) { println!("word is {word}"); %%%%%%% Changes from base to side jj-vcs#2 -fn example(w: String) { +fn example(w: &str) { println!("word is {w}"); >>>>>>> Conflict 1 of 1 ends } ```

Adds a new "snapshot" conflict marker style which returns a series of snapshots, similar to Git's "diff3" conflict style. The "snapshot" option uses a subset of the conflict hunk headers as the "diff" option (it just doesn't use "%%%%%%%"), meaning that the two options are trivially compatible with each other (i.e. a file materialized with "snapshot" can be parsed with "diff" and vice versa). Example of "snapshot" conflict markers: ``` <<<<<<< Conflict 1 of 1 +++++++ Contents of side jj-vcs#1 fn example(word: String) { println!("word is {word}"); ------- Contents of base fn example(w: String) { println!("word is {w}"); +++++++ Contents of side jj-vcs#2 fn example(w: &str) { println!("word is {w}"); >>>>>>> Conflict 1 of 1 ends } ```

If a file contains lines which look like conflict markers, then we need to make the real conflict markers longer so that the materialized conflicts can be parsed unambiguously. For instance, if we have a file explaining the differences between Jujutsu's conflict markers and Git's conflict markers, it could produce a conflict with long markers like this: ``` <<<<<<<<<<< Conflict 1 of 1 %%%%%%%%%%% Changes from base to side jj-vcs#1 -Git's conflict markers look like this: +Unlike Jujutsu, Git's conflict markers look like this: <<<<<<< left ======= right >>>>>>> +++++++++++ Contents of side jj-vcs#2 Jujutsu uses different conflict markers than Git, which just shows the sides of a conflict without a diff. >>>>>>>>>>> Conflict 1 of 1 ends ``` When parsing the conflict, we require that the conflict markers in the file are at least as long as the materialized conflict markers, and we only parse the longest conflict markers in the file.

If a file contains lines which look like conflict markers, then we need to make the real conflict markers longer so that the materialized conflicts can be parsed unambiguously. When parsing the conflict, we require that the conflict markers in the file are at least as long as the materialized conflict markers, and we only parse the longest conflict markers in the file. For instance, if we have a file explaining the differences between Jujutsu's conflict markers and Git's conflict markers, it could produce a conflict with long markers like this: ``` <<<<<<<<<<< Conflict 1 of 1 %%%%%%%%%%% Changes from base to side jj-vcs#1 Jujutsu uses different conflict markers than Git, which just shows the -sides of a conflict without a diff. +sides of a conflict without a diff: + +<<<<<<< +left +||||||| +base +======= +right +>>>>>>> +++++++++++ Contents of side jj-vcs#2 Jujutsu uses different conflict markers than Git: <<<<<<< %%%%%%% -base +left +++++++ right >>>>>>> >>>>>>>>>>> Conflict 1 of 1 ends ```

If a file contains lines which look like conflict markers, then we need to make the real conflict markers longer so that the materialized conflicts can be parsed unambiguously. When parsing the conflict, we require that the conflict markers in the file are longer than any existing "fake" conflict markers in the file, and we only parse the longest conflict markers in the file. For instance, if we have a file explaining the differences between Jujutsu's conflict markers and Git's conflict markers, it could produce a conflict with long markers like this: ``` <<<<<<<<<<< Conflict 1 of 1 %%%%%%%%%%% Changes from base to side jj-vcs#1 Jujutsu uses different conflict markers than Git, which just shows the -sides of a conflict without a diff. +sides of a conflict without a diff: + +<<<<<<< +left +||||||| +base +======= +right +>>>>>>> +++++++++++ Contents of side jj-vcs#2 Jujutsu uses different conflict markers than Git: <<<<<<< %%%%%%% -base +left +++++++ right >>>>>>> >>>>>>>>>>> Conflict 1 of 1 ends ```

If a file contains lines which look like conflict markers, then we need to make the real conflict markers longer so that the materialized conflicts can be parsed unambiguously. When parsing the conflict, we require that the conflict markers in the file are longer than any existing "fake" conflict markers in the file, and we only parse the longest conflict markers in the file. For instance, if we have a file explaining the differences between Jujutsu's conflict markers and Git's conflict markers, it could produce a conflict with long markers like this: ``` <<<<<<<<<<< Conflict 1 of 1 %%%%%%%%%%% Changes from base to side jj-vcs#1 Jujutsu uses different conflict markers than Git, which just shows the -sides of a conflict without a diff. +sides of a conflict without a diff: + +<<<<<<< +left +||||||| +base +======= +right +>>>>>>> +++++++++++ Contents of side jj-vcs#2 Jujutsu uses different conflict markers than Git: <<<<<<< %%%%%%% -base +left +++++++ right >>>>>>> >>>>>>>>>>> Conflict 1 of 1 ends ``` We should support these options for "git" conflict marker style as well, since Git actually does support producing longer conflict markers in some cases through .gitattributes: https://git-scm.com/docs/gitattributes#_conflict_marker_size We may also want to support passing the conflict marker length to merge tools as well in the future, since Git supports a "%L" parameter to pass the conflict marker length to merge drivers: https://git-scm.com/docs/gitattributes#_defining_a_custom_merge_driver

If a file contains lines which look like conflict markers, then we need to make the real conflict markers longer so that the materialized conflicts can be parsed unambiguously. When parsing the conflict, we require that the conflict markers are at least as long as the materialized conflict markers based on the current tree. This can lead to some unintuitive edge cases which will be solved in the next commit. For instance, if we have a file explaining the differences between Jujutsu's conflict markers and Git's conflict markers, it could produce a conflict with long markers like this: ``` <<<<<<<<<<< Conflict 1 of 1 %%%%%%%%%%% Changes from base to side jj-vcs#1 Jujutsu uses different conflict markers than Git, which just shows the -sides of a conflict without a diff. +sides of a conflict without a diff: + +<<<<<<< +left +||||||| +base +======= +right +>>>>>>> +++++++++++ Contents of side jj-vcs#2 Jujutsu uses different conflict markers than Git: <<<<<<< %%%%%%% -base +left +++++++ right >>>>>>> >>>>>>>>>>> Conflict 1 of 1 ends ``` We should support these options for "git" conflict marker style as well, since Git actually does support producing longer conflict markers in some cases through .gitattributes: https://git-scm.com/docs/gitattributes#_conflict_marker_size We may also want to support passing the conflict marker length to merge tools as well in the future, since Git supports a "%L" parameter to pass the conflict marker length to merge drivers: https://git-scm.com/docs/gitattributes#_defining_a_custom_merge_driver

If a file contains lines which look like conflict markers, then we need to make the real conflict markers longer so that the materialized conflicts can be parsed unambiguously. When parsing the conflict, we require that the conflict markers are at least as long as the materialized conflict markers based on the current tree. This can lead to some unintuitive edge cases which will be solved in the next commit. For instance, if we have a file explaining the differences between Jujutsu's conflict markers and Git's conflict markers, it could produce a conflict with long markers like this: ``` <<<<<<<<<<< Conflict 1 of 1 %%%%%%%%%%% Changes from base to side #1 Jujutsu uses different conflict markers than Git, which just shows the -sides of a conflict without a diff. +sides of a conflict without a diff: + +<<<<<<< +left +||||||| +base +======= +right +>>>>>>> +++++++++++ Contents of side #2 Jujutsu uses different conflict markers than Git: <<<<<<< %%%%%%% -base +left +++++++ right >>>>>>> >>>>>>>>>>> Conflict 1 of 1 ends ``` We should support these options for "git" conflict marker style as well, since Git actually does support producing longer conflict markers in some cases through .gitattributes: https://git-scm.com/docs/gitattributes#_conflict_marker_size We may also want to support passing the conflict marker length to merge tools as well in the future, since Git supports a "%L" parameter to pass the conflict marker length to merge drivers: https://git-scm.com/docs/gitattributes#_defining_a_custom_merge_driver

Currently, conflict markers are materialized in a format that cannot be parsed if a conflict appears at the end of the file, and there is a non-empty line which doesn't end with a newline character. To fix this issue, we can add an extra newline to every term of the conflict when this occurs and then ignore the added newlines when parsing the conflict. To ensure unambiguous parsing, we can add a "[noeol]" tag to the start conflict marker (`<<<<<<<`) when this occurs. I think this is the best solution to the problem because it should remain understandable to users even if they aren't familiar with file encodings. Many editors represent newlines at the end of files by showing an empty line at the end of the editor, so it should be intuitive that a term ending with an empty line represents a newline, while a term that doesn't end with an empty line represents the lack of a newline. For instance, a conflict with terms `["left\n", "base", "base\nright"]` would look like: ``` <<<<<<< Conflict 1 of 1 [noeol] +++++++ Contents of side jj-vcs#1 left %%%%%%% Changes from base to side jj-vcs#2 base +right >>>>>>> ```

Currently, conflict markers are materialized in a format that cannot be parsed if a conflict appears at the end of the file, and there is a non-empty line which doesn't end with a newline character. To fix this issue, we can add an extra newline to every term of the conflict when this occurs and then ignore the added newlines when parsing the conflict. To ensure unambiguous parsing, we can add a "[noeol]" tag to the start conflict marker (`<<<<<<<`) when this occurs. I think this is a good solution to the problem because it should remain understandable to users even if they aren't familiar with file encodings. Many editors represent newlines at the end of files by showing an empty line at the end of the editor, so it should be intuitive that a term ending with an empty line represents a newline, while a term that doesn't end with an empty line represents the lack of a newline. Also, adding a newline to every term of the hunk is easier than keeping track of which terms have added newlines and which don't, so it makes materialization/parsing easier. For instance, a conflict with terms `["left\n", "base", "base\nright"]` would look like: ``` <<<<<<< Conflict 1 of 1 [noeol] +++++++ Contents of side jj-vcs#1 left %%%%%%% Changes from base to side jj-vcs#2 base +right >>>>>>> ```

avamsi mentioned this issue Jan 21, 2023

FR: let git.(fetch|push) config options be lists? #1091

Closed

ilyagr referenced this issue in ilyagr/jj Jul 2, 2023

Failed attempt #2 to create a git-tracking conflict

018764b

ilyagr referenced this issue in ilyagr/jj Jul 2, 2023

Failed attempt #2 to create a git-tracking conflict

9c11616

ilyagr referenced this issue in ilyagr/jj Jul 3, 2023

Failed attempt #2 to create a git-tracking conflict

2ccda7b

ilyagr referenced this issue in ilyagr/jj Jul 3, 2023

Failed attempt #2 to create a git-tracking conflict

6dd45ff

ilyagr referenced this issue in ilyagr/jj Jul 3, 2023

Failed attempt #2 to create a git-tracking conflict

ebd7638

ilyagr referenced this issue in ilyagr/jj Jul 3, 2023

Failed attempt #2 to create a git-tracking conflict

9c68079

ilyagr referenced this issue in ilyagr/jj Jul 3, 2023

Failed attempt #2 to create a git-tracking conflict

29d5a4d

ilyagr referenced this issue in ilyagr/jj Jul 3, 2023

Failed attempt #2 to create a git-tracking conflict

a92cb9c

ilyagr referenced this issue in ilyagr/jj Jul 3, 2023

Failed attempt #2 to create a git-tracking conflict

5d1dbb7

ilyagr referenced this issue in ilyagr/jj Jul 3, 2023

Failed attempt #2 to create a git-tracking conflict

268dbc4

ilyagr referenced this issue in ilyagr/jj Jul 10, 2023

Failed attempt #2 to create a git-tracking conflict

14aa932

ilyagr referenced this issue in ilyagr/jj Jul 10, 2023

Failed attempt #2 to create a git-tracking conflict

4db7a26

ilyagr referenced this issue in ilyagr/jj Jul 11, 2023

Failed attempt #2 to create a git-tracking conflict

082a256

gulbanana mentioned this issue Feb 4, 2024

working_copy: implement symlinks on Windows with a helper function #2939

Merged

4 tasks

gulbanana closed this as completed in #2939 Mar 5, 2024

BrewTestBot mentioned this issue Mar 7, 2024

jj 0.15.0 Homebrew/homebrew-core#165306

Merged

BatmanAoD mentioned this issue Apr 18, 2024

Implement FR #2338 for current branches / automatically advancing branches #3129

Merged

ilyagr mentioned this issue Jun 26, 2024

diff: symlinks with external diff tools produce invalid directory structures #3963

Open

ilyagr mentioned this issue Jul 30, 2024

cli split: add experimental --restore-from (AKA --from) argument #4097

Draft

3 tasks

bnjmnt4n mentioned this issue Oct 7, 2024

FR: Show which revisions are involved in a conflict #4600

Closed

arxanas mentioned this issue Nov 25, 2024

Conflicts at the end of files that don't end in a newline are not materialized correctly #3968

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for symlinks on Windows #2

Add support for symlinks on Windows #2

martinvonz commented Jan 8, 2021 •

edited

Loading

bpollack commented Jan 8, 2021 •

edited

Loading

martinvonz commented Jan 8, 2021

quark-zju commented Feb 1, 2021 •

edited

Loading

martinvonz commented Feb 2, 2021

Add support for symlinks on Windows #2

Add support for symlinks on Windows #2

Comments

martinvonz commented Jan 8, 2021 • edited Loading

bpollack commented Jan 8, 2021 • edited Loading

martinvonz commented Jan 8, 2021

quark-zju commented Feb 1, 2021 • edited Loading

martinvonz commented Feb 2, 2021

martinvonz commented Jan 8, 2021 •

edited

Loading

bpollack commented Jan 8, 2021 •

edited

Loading

quark-zju commented Feb 1, 2021 •

edited

Loading