annex: web uploads

[kousu]

I'm not unclear if Gitea supports uploading from the web into Git LFS. If it does, then we should tweak that UI to make git-annex an option too. Otherwise the "Upload a File" menu might mislead people so that they upload files to .git/objects instead of .git/annex/objects, which isn't the end of the world if it happens once but could be a pain in general.

[kousu]

I did a quick experiment: it seems that if you have git lfs trackd a file extension (i.e. there's a line like *.dcm filter=lfs diff=lfs merge=lfs -text in .gitattributes)

then uploading a matching file

will cause it to get stored in LFS:

that's better than the UI I would have thought of! But I don't think my implementation in #1 behaves like this; so patch that and get it working.

[kousu]

Tracing:

Uploaded files get POSTed to /upload-file, and put in a data/tmp/

the server sends back a pointer to the temp file (it uses UUIDs)

the commit goes to POST /_upload/$BRANCH with the file pointers attached

that ends up calling into this function

https://github.com/neuropoly/gitea/blob/fa43bce541507c8a702723f84764a48db9278506/services/repository/files/upload.go#L52-L159

which goes and digs out the tmp files and has also makes a temporary full working-tree copy of the repo; because it has to, it can't git commit otherwise.

It then calls git check-attr --cached filter -- $filename, which is how it ends up reading .gitattributes:

https://github.com/neuropoly/gitea/blob/fa43bce541507c8a702723f84764a48db9278506/services/repository/files/upload.go#L97-L100

https://github.com/neuropoly/gitea/blob/fa43bce541507c8a702723f84764a48db9278506/modules/git/repo_attribute.go#L46

(git-check-attr produces something like

p115628@joplin:~/src/neurogitea/test/lfs-test$ git check-attr --cached filter -- "lol.nii.gz"
lol.nii.gz: filter: lfs

)

and then that's used to detect if it needs to run this special LFS case:

https://github.com/neuropoly/gitea/blob/fa43bce541507c8a702723f84764a48db9278506/services/repository/files/upload.go#L169-L181

[kousu]

So I think the equivalent annex code would be something like:

if setting.LFS.StartServer && filename2attribute2info[info.upload.Name] != nil && filename2attribute2info[info.upload.Name]["filter"] == "annex" { 
   // in the temporary repo
   system("git config annex.private true")
   system("git annex init")
   system("git annex dead here")
   copy info.upload to repo directory
   system("git add .")
   system("git annex copy --to origin")
   t.CommitTree(opts.LastCommitID, author, committer, treeHash, opts.Message, opts.Signoff)
   t.CommitTree("git-annex", author, committer, treeHash, opts.Message, opts.Signoff)
}

but don't use system() if you can avoid it. Maybe we can call git annex pre-commit or git annex smudge directly to avoid having to run git annex init?