Fix listing of file names with special characters to avoid crash and unrecognised names #204

[jmurty]

Tweak the file listing to fix two issues found to affect files with problematic names like "Terrible file""")))(((][][].secret.

This change may undo performance improvements achieved in #193.

• Ensure the git check-attr filter command does not quote file name characters by using -z arg and some extra trickery to keep space and newline delimiter characters for filter metadata and each file name respectively.

  This fixes a file named "Terrible file""")))(((][][].secret being listed as just Terrible file.

• Adjust checking for filter and crypt strings to work with changed output from check-attr command.

• Tweak eval 'echo $filename' portion of command to avoid script crash on files with special characters with error message like eval: line 192: syntax error near unexpected token

[andreineculau]

This change may undo performance improvements achieved in #193.

confirmed 😢

[andreineculau]

What exactly is the problem? I must be blind, but I've just created a "Terrible file""")))(((][][].secret file and ran my version of transcrypt (it says 2.3.2-pre but I'd have to double-check I don't have any local patches) without an issue 🤔
I see this is a filename you came up with, but the user didn't yet reply. What issue to you see with your suggested filename?

Do you actually get .git/crypt/transcrypt: eval: line 192: syntax error near unexpected token ('` ?

[jmurty]

The problem is when a file with problematic characters – like the "Terrible File" example – is added as a secret/encrypted file, its name gets munged by the _list_encrypted_files() function so that file can no longer be recognised as an encrypted one.

[andreineculau]

Thing is... I don't seem to get that error 🤔 I created a file with the mentioned filename, I staged it, and transcrypt -l was still happy 👀

[jmurty]

Hi @andreineculau are you sure you are checking a file with a problematic name when that file is also encrypted, not just staged as a standard file? The problem manifests as the problematic filename not being included in the output of transcrypt --list when it should be because the file is encrypted.

I have added a unit test that demonstrates the problem with the old transcrypt version from main (efd42e5) and proves that the fix applied on this branch solves it.

I have also made another change in 93ba18f which will hopefully improve performance again for repositories like yours with many files, by using an awk command instead of shell loop to handle the null-delimited list of filenames and their filter attributes.

Can you check the performance of this latest version of this PR?

[andreineculau]

this is what I see on my end

with the new 94b8599 version

$ pbpaste > transcrypt
$ cp transcrypt .git/crypt/transcrypt
$ echo foo > "\"Difficult file name\"\"\")))(((][][].secret"
$ git add .

$ git status --porcelain
A  "\"Difficult file name\"\"\")))(((][][].secret"
M  transcrypt

$ /usr/bin/time ./transcrypt -l
"Difficult file name""")))(((][][].secret
CONST.inc.secret
        0.34 real         0.18 user         0.13 sys

and with the old version I have (with the changes in #193)

$ echo foo > "\"Difficult file name\"\"\")))(((][][].secret"
$ git add .

$ git status --porcelain
A  "\"Difficult file name\"\"\")))(((][][].secret"

$ /usr/bin/time ./transcrypt -l
CONST.inc.secret

👆 NOTE that the difficult file is not detected as encrypted!

With the version in 6aa50df --- I just killed ./transcrypt -l after I counted 60 seconds...

So - since we have a test that confirms the fix 🤩 and I can confirm now that the performance is on par with #193 🤩 .... 👏❤️🚀:shipit: