Git 2.24 breaks existing repositories: filename in tree entry contains backslash

[carlescufi]

• I was not able to find an open or closed issue matching what I'm seeing

What are users supposed to do here, rewrite the history of the repos so that the offending commit is removed?
The workaround is to disable core.protectNTFS:
git config --global core.protectNTFS false

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
  Git for Windows 2.24.1.windows.2 64-bit

$ git --version --build-options

git version 2.24.1.windows.2
cpu: x86_64
built from commit: 992f0773022527b1b0cb1e0c13aec97dd5248053
sizeof-long: 4
sizeof-size_t: 8

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.18363.535]

• What options did you set as part of the installation? Or did you choose the
  defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

Editor Option: VIM
Custom Editor Path:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Disabled
Enable Builtin Interactive Add: Disabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

Installed with Chocolatey

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

cmd.exe

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

git clone https://github.com/zephyrproject-rtos/civetweb.git

• What did you expect to occur after running these commands?

Cloned repo successfully.

• What actually happened instead?

Cloning into 'civetweb'...
remote: Enumerating objects: 38, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (31/31), done.
remote: Total 21161 (delta 17), reused 19 (delta 7), pack-reused 21123
Receiving objects: 100% (21161/21161), 24.53 MiB | 7.55 MiB/s, done.
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
error: filename in tree entry contains backslash: '\'
Resolving deltas: 100% (13566/13566), done.
Updating files: 100% (1251/1251), done.

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

https://github.com/zephyrproject-rtos/civetweb.git

[dscho]

@carlescufi did you find out which tree entries those might be? They really cannot be checked out on Windows as expected: if there is a backslash encoded in any file name in that repository, it will be mistaken for a directory separator on Windows, which is obviously not what we want here.

[mbolivar]

@carlescufi did you find out which tree entries those might be?

One commit with a bad file is zephyrproject-rtos/civetweb@9d3af98 which adds a test/\/a.txt. However, this file was later removed in zephyrproject-rtos/civetweb@83dc425, and no file in the latest master branch contains a backslash in its name.

They really cannot be checked out on Windows as expected:

No attempt is made to check out a file with a backslash when cloning the latest master.

And indeed, though git clone https://github.com/zephyrproject-rtos/civetweb prints errors, it exits with a 0 return code.

However, this git fetch fails with return code 1:

mkdir civetweb
cd civetweb
git init
git fetch https://github.com/zephyrproject-rtos/civetweb.git

That seems inconsistent and arguably wrong: since we can clone the repository without an error return code (just error messages), we should be able to fetch it, too, just not check out the bad revisions.

[carlescufi]

@dscho as @mbolivar pointed out there is no longer any file in the repository with a backslash in its name. This means that, short of changing the behavior of Git for Windows in an upcoming release, the only other alternative is to rewrite the history of the tree to completely remove any traces of commits that introduced filenames with backslashes. This might be very cumbersome on some repositories.

[dscho]

@carlescufi & @mbolivar I am sympathetic to your use case. However, I am still worried about lifting the protections that we carefully put in place. So I'd like to tread very, very carefully here.

To start us in a direction that hopefully will lead to an acceptable compromise that still keeps the Git for Windows users safe, but also allows users to clone civetweb as long as they do not try to check out commits with the inadmissible directory name, I opened #2437.

[carlescufi]

@dscho thank you for the understanding and the PR. Please let us know when we can fetch a binary from CI so we can test it ourselves and give a +1.

[dscho]

Please let us know when we can fetch a binary from CI so we can test it ourselves and give a +1.

Whoa. I would like to ask even more explicitly for help with thinking about the problem. Maybe you can give a hand with reviewing the code and trying to get it into a shape where we can be reasonably sure that it does not regress on the vulnerabilities? Before that, I won't even bother with thinking about binaries.

[carlescufi]

Please let us know when we can fetch a binary from CI so we can test it ourselves and give a +1.

Whoa. I would like to ask even more explicitly for help with thinking about the problem. Maybe you can give a hand with reviewing the code and trying to get it into a shape where we can be reasonably sure that it does not regress on the vulnerabilities? Before that, I won't even bother with thinking about binaries.

Sorry, since you had posted a Pull Request I assumed that you had considered those already. I will take a look at the current PR with @mbolivar and give feedback here.