Git submodules not being initialized?

[Hoverbear]

Hi!

I'm trying to use this tool to build vectordotdev/vector@0836bcd which depends on bytecodealliance/lucet@d741e2e

You can see here, one of the subcrates of lucet, lucet-wasi, depends on wasi-common which is a git submodule in the root.

https://github.com/bytecodealliance/lucet/blob/d741e2e30df19d1f8abd1a8660219ac494336b76/lucet-wasi/Cargo.toml#L25

This seems to not be handled. :(

.

You can reproduce this with:

git clone git@github.com:timberio/vector.git
git checkout 0836bcdc0c63e35061c003079edff06af6e3f198
make neu-release-binary-x86_64-unknown-linux-gnu

I see NixOS/nix#3166 but haven't managed to invoke it.

[nmattia]

I see NixOS/nix#3166 but haven't managed to invoke it.

The PR was merged very recently, did you install the latest nix executable?

[Hoverbear]

It's sadly not on the latest nixos-unstable :(

vector on  nix-builds [$!?] is 📦 v0.10.0 via ⬢ v12.18.0 via 🦀 v1.44.0 via ❄️  impure took 2s 
❯ nix repl                                                      
Welcome to Nix version 2.3.6. Type :? for help.

nix-repl> builtins.fetchGit { url = ""; submodules = true; }
error: unsupported argument 'submodules' to 'fetchGit', at (string):1:31

This is the latest: https://github.com/NixOS/nix/releases/tag/2.3.6

[evanjs]

I have been using:
nix run nixpkgs.nixUnstable --command nix-build -E "(with import <nixpkgs> {}; callPackage "./${1-default.nix}" {})" --show-trace to get submodule support, but have been encountering #109, so I can't say whether it's working or not :P

naersk, I mean.
Edit: this issue has been resolved with #109 (comment)

---

To get a nix with submodule support, you should be able to use:
nix run -f channel:nixos-unstable nixpkgs.nixUnstable --command nix-build -E "(with import <nixpkgs> {}; callPackage "./${1-default.nix}" {})"
or simply
nix run -f channel:nixos-unstable nixUnstable --command nix-build

Note the use of -f channel:nixos-unstable to ensure you pull from nixos-unstable
This will use nix (Nix) 2.4pre20200721_ff314f1 which has support for submodules via fetchGit

[nmattia]

Thanks @evanjs ! What's the content of 1-default.nix? Did you have to tweak naersk to make it work?

[evanjs]

${1-default.nix} allows the command to fallback to default.nix if $1 is not given via substitution

Contents of default.nix

{ stdenv
, callPackage
, sources ? import ../../../nix/sources.nix
# build inputs
, pkg-config
, udev

}:
let
  pname = "spi_ccp";
  version = "1.0.0";
  cargoLock = ./Cargo.lock;
  src = stdenv.mkDerivation rec {
    inherit pname version;
      src = builtins.fetchGit {
        url = "<private-git-url>";
        rev = "<rev>";
      };
      installPhase = ''
        mkdir -p $out $out/
        cp -R ./* $out/
        cp ${cargoLock} $out/Cargo.lock
      '';
    };
    naersk = callPackage sources.naersk {};
in
  naersk.buildPackage {
    name = pname;
    inherit src version;
    nativeBuildInputs = [
      pkg-config
    ];
    buildInputs = [
      udev
    ];
  }

The relevant bit being:

src = builtins.fetchGit {
        url = "<private-git-url>";
        rev = "<rev>";
      };

Naersk did not need to be modified to get this working, only Nix 2.4pre
i.e. what is currently available via the nixUnstable attribute on nixos-unstable (I see 2.4pre20200721_ff314f1 on my desktop)

[Patryk27]

This should be fixed now that we have #231 merged.

Please feel free to re-open this ticket if the issue still appears 🙂