Since introduction of git 2.38 repo archive tar ball checksums are inconsistent with older releases

[petergardfjall]

Description

Not entirely sure if this is a bug, a feature request or just an attempt to raise awareness of a potential issue to the community (I didn't see this mentioned in any other issue).

In short, the issue is that newer releases of Gitea do not produce tar.gz repo archives that are checksum-stable with older Gitea releases (dating back to v1.17.2). For most clients this may not be an issue, but for us, who use Gitea as an important part of our supply-chain it has dire consequences.

Details

It seems like somewhere between Gitea v1.17.2 and v1.19.3 an upgrade of the git command line tool in the Dockerfile was made to 2.38 (v1.19.3 appears to use 2.38.5). This brought with it a behavioral change in the git archive command, which
changed its default behavior from using the gzip(1) command to using an internal zlib library (see commit).

Since the GetArchive API call directly uses git archive under the hood, it means that the tar.gz repository archives produced by Gitea are no longer checksum-stable. This may not have been an explicit promise of the Gitea API,
but it is something that we relied upon.

Notably this issue was also faced by GitHub, who first reacted by letting clients know that this was never officially guaranteed by the API. Given the community outrage that followed and GitHub's important position in supply chains and supply chain tooling, GitHub soon rolled back the change (some more of this discussion is covered in this discussion)

On our end, to preserve the old git archive behavior, we needed to explicitly configure the git client used by Gitea like so:

git config --global tar.tgz.command 'gzip -cn'
git config --global tar.tar.gz.command 'gzip -cn'

Discussion

If one relies on Gitea to produce checksum-stable repo archives, newer versions of Gitea will not be consistent with older releases. Where do we go from here?

At a minimum, I think this needs to be documented, although I would prefer to see something more substantial.

If Gitea wants to be a strong player in a world increasingly concerned with supply chain attacks and the likes, it might be good to actually make promises regarding checksum stable archives. Adding a few tests would be an easy way to prevent unintended changes to produced archives. Changing the default back to the old behavior (of using gzip(1)) like GitHub did could also be an option.

Although a fix for this particular issue was outlined above, it does requires some "hackery" to get in place. One could perhaps consider exposing more of the git configuration through Gitea's own configuration "API"?

Gitea Version

v1.19.3

Can you reproduce the bug on the Gitea demo site?

Yes

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

Linux

How are you running Gitea?

Helm chart

Database

PostgreSQL

[delvh]

On our end, to preserve the old git archive behavior, we needed to explicitly configure the git client used by Gitea like so:
git config --global tar.tgz.command 'gzip -cn'
git config --global tar.tar.gz.command 'gzip -cn'

Luckily, Gitea already supports a mechanism for changing git internal behavior.
Add the following to your app.ini:

[git.config]
tar.tgz.command = gzip -cn
tar.tar.gz.command = gzip -cn

The only problem is we cannot just set that as default as not everyone will have gzip installed.
And if I have the choice between something being slightly broken and being completely broken, I prefer the slightly broken.

[petergardfjall]

Luckily, Gitea already supports a mechanism for changing git internal behavior.

Ah, I didn't notice this section of the documentation. That is useful, thanks.

[chenrui333]

Per suggestion in https://gitea.com/gitea/tea/issues/570, adding a note in here about the checksum mismatch that we have seen in the homebrew side, Homebrew/homebrew-core#139430

[lunny]

So I think this could be closed as it's related upstream.

ref 4f4be00d302bc52d0d9d5a3d4738bb525066c710