git2 repository update lead to read zlib stream error when fetch repo

[jongwu]

Problem
there maybe a bug in the latest cargo dependency repo of git2 which will lead to "error reading from the zlib stream" when fetch a certain repo (I don't know which of its group) in cargo build.
I have not detected why these happen, but I confirm that it is the git2 repo(0.12, 0.12+) in cargo dependency cause the error.
Steps
I have just one example to reproduce the error.

1. go get github.com/kata-containers/kata-containers
2. cd $(GOPATH)/github.com/kata-containers/kata-containers/ && git checkout 7087b5f43c86367f78d28
3. make
   be patient, then you will get the error log.

Possible Solution(s)
I'm not familiar with libgit2, so I can't tell why this happen, only, I can tell the error come from git_packfile_stream_read in the libgit2/src/pack.c

Notes

Output of cargo version:

I have tested cargo 1.43.1 even the latest cargo from source code (13bded9)

@wycats @halorgium @alex

[alexcrichton]

Thanks for the report! Would it be possible to minimize this a bit more? Or could you provide some logs of what happens? It'd be useful to know what Cargo is cloning. You might have some success digging with CARGO_LOG=trace as well.

[jongwu]

hello @alexcrichton , sorry to have not add the error log. here it is:

Updating git submodule https://chromium.googlesource.com/chromium/llvm-project/llvm/lib/Fuzzer`
error: failed to get grpcio as a dependency of package protocols v0.1.0 (/root/go/src/github.com/kata-containers/kata-containers/src/agent/protocols)

Caused by:
failed to load source for dependency grpcio

Caused by:
Unable to update https://github.com/alipay/grpc-rs?branch=rust_agent#eee4635f

Caused by:
failed to update submodule grpc-sys/grpc

Caused by:
failed to update submodule third_party/bloaty

Caused by:
failed to update submodule third_party/libFuzzer

Caused by:
failed to fetch submodule third_party/libFuzzer from https://chromium.googlesource.com/chromium/llvm-project/llvm/lib/Fuzzer
Caused by:
error reading from the zlib stream; class=Zlib (5)

`
I hope it will be helpful.

[alexcrichton]

Thanks! I'm indeed able to reproduce this with

git2::Repository::clone(                                                      
    "https://chromium.googlesource.com/chromium/llvm-project/llvm/lib/Fuzzer",
    "test",                                                                    
)                                                                             
.unwrap();                                                                    

Running in a debugger it does indeed look like libz is simply returning an error. The origin of the git error fails on this call to inflate where the error gets generated here.

I haven't extracted the zlib stream but it sort of seems like this is a legitimately invalid zlib stream, or at least according to libz it is. Either that or it's being fed into libz incorrectly by libgit2, which is also possible.

In any case this isn't an issue Cargo itself can deal with. At most it's an issue with libgit2, but it could also be an issue with libz or the server itself.

[jongwu]

@alexcrichton so, should we pass this ball to libgit2 community to resolve this issue?

[alexcrichton]

Indeed, I've opened up libgit2/libgit2#5525