fetchgit: shallow clone for submodules #254172
Conversation
|
Thanks for the review! |
|
Pick a random repo |
|
I think I have found that using looking at the script, I can see that it is trying to change directory into a Looking through nixpkgs, I can only see 1 package which has both options enabled (and it is gated by I will update this PR with some test cases to reproduce the results, and change this to a Draft for now. |
pineapplehunter
force-pushed
the
fetchgit-shallow
branch
from
b507625
to
89fa835
Compare
|
The issue seems to be $ nix-prefetch-git --url https://github.com/pineapplehunter/nix-test-repo-with-submodule --rev 26473335b84ead88ee0a3b649b1c7fa4a91cfd4a --fetch-submodules --leave-dotGit
Initialized empty Git repository in /tmp/git-checkout-tmp-dohXPkOr/nix-test-repo-with-submodule-2647333/.git/
remote: Enumerating objects: 4, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 4 (delta 0), reused 4 (delta 0), pack-reused 0
Unpacking objects: 100% (4/4), 416 bytes | 416.00 KiB/s, done.
From https://github.com/pineapplehunter/nix-test-repo-with-submodule
* branch HEAD -> FETCH_HEAD
Switched to a new branch 'fetchgit'
Submodule 'nix-test-repo-submodule' (https://github.com/pineapplehunter/nix-test-repo-submodule.git) registered for path 'nix-test-repo-submodule'
Cloning into '/tmp/git-checkout-tmp-dohXPkOr/nix-test-repo-with-submodule-2647333/nix-test-repo-submodule'...
Submodule path 'nix-test-repo-submodule': checked out '3223dfc5e18f5c15e0cb26ea7bdf5f12d215619b'
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), done.
Total 4 (delta 0), reused 0 (delta 0), pack-reused 0
Enumerating objects: 4, done.
Nothing new to pack.
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). |
pineapplehunter
force-pushed
the
fetchgit-shallow
branch
from
89fa835
to
bd38744
Compare
pineapplehunter
force-pushed
the
fetchgit-shallow
branch
from
bd38744
to
ea9af0d
Compare
|
@Artturin |
| if [ -d "$gitdir" ]; then | ||
| # if `.git` is a directory (in the root of the repo), | ||
| # make the content of it deterministic | ||
| make_deterministic_repo "$(readlink -f "$gitdir")" | ||
| else | ||
| # if `.git` is a file (in a submodule), | ||
| # get the path of the git directory by reading the file and make it deterministic | ||
| make_deterministic_repo "$(dirname $gitdir)/$(cat $gitdir | awk '{ print $2 }')" |
| [[ -z "$deepClone" ]] && [[ -z "$leaveDotGit" ]] && \ | ||
| clean_git submodule update --init --recursive -j ${NIX_BUILD_CORES:-1} --depth 1 || \ |
There was a problem hiding this comment.
| [[ -z "$deepClone" ]] && [[ -z "$leaveDotGit" ]] && \ | |
| clean_git submodule update --init --recursive -j ${NIX_BUILD_CORES:-1} --depth 1 || \ | |
| # leaveDotGit exception is because shallow cloning will change hashes | |
| [[ -z "$deepClone" ]] && [[ -z "$leaveDotGit" ]] && \ | |
| clean_git submodule update --init --recursive -j ${NIX_BUILD_CORES:-1} --depth 1 || \ |
pineapplehunter
force-pushed
the
fetchgit-shallow
branch
2 times, most recently
from
4f9a1c2
to
e2fa178
Compare
pineapplehunter
force-pushed
the
fetchgit-shallow
branch
from
e2fa178
to
da575f2
Compare
Co-authored-by: Artturi <Artturin@artturin.com>
|
Thanks |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/cant-build-solvespace-from-scratch-fetching-git-source-fails/46536/5 |
Description of changes
I enabled shallow cloning for submodules.
It will not shallow clone if
deepCloneorleaveDotGitis true. I belaeve you don't have to deep clone when onlyleaveDotGitis true, but I thought it may change the hash, so I put it there. If it's not an issue, I'm happy to remove this restriction.I tested with a few packages that have a submodule and it seemd to work without the hash changing. If there is a better way of testing I would like to know.
This should help for git repositorys with a large submodule (in my case the submodule was LLVM).
Things done
sandbox = trueset innix.conf? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)