Windows issues, work-arounds and a minor change request

[per-oestergaard]

Hi.
I am using this on Windows 10 and after some issues, it is working. For it to work for others, I think my change should be put back into the tool.

First the two hints -

• Set core.quotePath to false (git config --add core.quotePath false). Without this Danish (and similar) characters are escaped to \octal and that breaks further processing of those files
• Next, to install transact on Windows without bash, I simply added it as a git alias and use the built-in bash interpretator in git. This is done with git config --add alias.transcrypt = !~/path/to/transcrypt/transcrypt. After this, I can install it with 'git transcrypt'.

And then the required fix in transcrypt itself.

The input file seperator (IFS) must be set to linefeed to avoid having the bash for statement split the input into words on space. Any files I have with spaces were not processed. So I added IFS=$'\n' before the "for secret_file in" loops (just before https://github.com/elasticdog/transcrypt/blob/master/transcrypt#L338)

Hope this change can get in.

PS. A great tool. Looking at using it for client-side encrypted backup.

[jmurty]

Thanks for the detailed issue report @perost-l14

Could you please include here some example file names that caused you problems, especially those with spaces in the name and with non-ASCII Danish characters?

I suspect a couple of the problems you faced may not be due to running on Windows specifically, but flaws in the script when handling these kinds of file names.

Also, any extra information you can provide about how exactly you are running transcrypt could be helpful.

[per-oestergaard]

Thanks for the prompt response. Here is one of my files:

live/sub/2/Mig - røve.PNG

with

git config --add --local core.quotePath true
git ls-files live/sub/2/M*
"live/sub/2/Mig - r\303\270ve.PNG"

And the space in the file name messes up 'for secret_file in' as it splits it into words (the IFS thing).

I reran a test on whether core.quotePath has any impact at all. It does. Most things works; however, encryption fails -

*** WARNING : deprecated key derivation used.
Using -iter or -pbkdf2 would be better.
fatal: Unimplemented pathspec magic '"' in ':"live/sub/2/Mig - r\303\270ve.PNG"'

Note the -iter thing. There is another open issue on that.

My plan for this is -

• Create a repo on AWS CodeCommit (or somewhere else)
• Create 'staging' local .git outside my normal work folders
• Enabled transcrypt
• Add worktree to that to a work folder
• Adjust .gitignore and .gitattributes according to my requirements
• Have a git add + git commit loop running for moving changes into 'staging'
• Have a git push loop running for pushing changes from 'staging' to CodeCommit.

In case I have to recover files, I can clone CodeCommit, add transcrypt with the original password and then get to my files.
The downside is the that file size is a twice as big as the original file size plus the time for processing. The upside is that this is using trusted and battle tested software like git and openssl.

[jmurty]

Hi @perost-l14 I have applied a potential fix for most of the issues you reported on the branch improve-handling-of-challenging-file-names, could you test with that version of the transcrypt script to confirm whether it works for you.

In theory, this fixed version should work even if you have core.quotePath set to true, or unset since true is the default

Regarding your planned use case, I'm not sure I follow it exactly. But if you will need to encrypt many or all of your files for this "staging" approach then transcrypt may not be the right tool for the job. From the Caveats documentation:

There are much better options if your goal is to encrypt the entire repository.

[per-oestergaard]

Hi @jmurty
Apologize for the delay. The branched version works fine, also without core.quotePath. Thanks.

You may be right about my use case. I may or may not - to be decided - encrypt it all. However, if you have a good pointer to a solution where I can encrypt the entire repository (that does not require server-side changes, does client-side encryption and is in a format that is easy recoverable. I want to be able to use any git provider, keep control of keys and be guaranteed to be able to recover. I looked at https://github.com/spwhitton/git-remote-gcrypt; however, sound propriatary in their pack protocol if I understood it correctly), please share it.

[jmurty]

Thanks for confirming the fix @perost-l14 I appreciate it!

And regarding your use-case, because you want to be able to use "any git provider" I don't know of any better options. The only alternatives I'm aware of would involve storing your Git repo to some kind of dumb file storage service, not a proper Git provider at all.

The best approach might be just to try it and see how it goes. And let us know if you find other bugs or issues.

[per-oestergaard]

Thanks. I will try it out.

Regarding "any git provider". To be more clear, I mean any git provides that has a server (gitlab, github, codecommit etc.)
BTW: I am also experiencing the issue #55 and it would be great to have that fixed.

[per-oestergaard]

A hint: I am adding -delta to each line in .gitattributes I want to encrypt. In this way, I avoid having git packing (delta compression) files before pushing as that makes no sense on encrypted files. Makes push much faster. Example

*.jpg filter=crypt diff=crypt -delta

[per-oestergaard]

And a question: openssl enc uses the -a flag to generate base64 output. As base64 takes up a lot of space on the disk (and on a git server), couldn't it just as well be binary output? Text output makes sense when things needs to be compared; however, encryption makes the text undiff'able.

[jmurty]

Hi @perost-l14 I have created a new ticket #85 to collect and highlight some of your suggestions about how transcrypt's handling of large files could be improved, to make it easier to track your two latest comments/suggestions.

[jmurty]

Hi @perost-l14 since the fix for the main core.quotePath issue in this issue is merged to master I will close this one and leave #85 in place to track your other suggestions.