libtool: 2.4.6 -> 2.4.7

[ghost]

Description of changes

Update libtool to 2.4.7

libtool2-macos11.patch has been merged upstream as 9e8c88251

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
  • powerpc64le-linux
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 22.05 Release Notes (or backporting 21.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[ghost]

@ofborg eval

[ghost]

@FRidh added this to WIP in Staging via automation 3 hours ago

@FRidh, this PR is ready to be reviewed and/or merged.

[ghost]

Squashed SuperSandro2000's recommendations.

[trofi]

substituteInPlace build-aux/ltmain.in --replace '#! /usr/bin/env sh' '#!${runtimeShell}' now leaks out /nix/store paths into tarballs generated under nixpkgs. build-aux/ltmain.sh gets packaged as part of make dist in libtool-aware projects. Encountered it today on ski. Executable reproducer:

$ git clone https://github.com/trofi/ski && cd ski
$ git checkout v1.4.0
$ nix-shell --pure -p autoconf -p lzip -p automake -p libtool -p pkg-config -p libelf -p ncurses -p gperf -p bison -p flex
$$ ./autogen.sh && ./configure && make dist
$$ tar xf ski-1.4.0.tar.xz
$$ fgrep -R  nix/store ski-1.4.0/
ski-1.4.0/build-aux/ltmain.sh:#!/nix/store/3j918i1nbwhby0y38bn2r438rjhh8f4d-bash-5.1-p16/bin/bash

Note how ski-1.4.0/build-aux/ltmain.sh embeds the path to /nix/store.

[ghost]

Thank you for catching this.

I'm investigating a few solutions.

[ghost]

I think the fix is

https://github.com/amjoseph-nixpkgs/nixpkgs/commit/b5aad8addf1d4a84dbf7545965584bd19fb7e42e

Testing now...

[trofi]

Another fallout from substituteInPlace libtoolize.in --replace '#! /usr/bin/env sh' '#!${runtimeShell}' is present in branches before 6becbd3 where we dropped help2man call.

When help2man call is still present we call libtoolize with wrong bash interpreter: host one instead of build one which causes very strange errors on systems with invalid binfmt entry for host registered. One of the ways to trigger it is to register non-existent binfmt handler for riscv64 and build pkgsCross.riscv64.libtool from 9af373a.

The simpler one is to just grep libtoolize at build:

$ nix-build -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/9af373a61647257d16ae6062cddaa9094d24920c.tar.gz --expr "with import <nixpkgs> {}; pkgsCross.riscv64.libtool.overrideAttrs (oa: { preBuild = ''make libtoolize; ! grep -P '#!.*riscv64' libtoolize''; })"
...
building
  GEN      libtoolize
#!/nix/store/b0lbpi22a000jkm7lp7ihi8w463al3ry-bash-5.1-p16-riscv64-unknown-linux-gnu/bin/bash
error: builder for '/nix/store/89cqzdsiqbzx4yng4nrlyd5nzydndr2g-libtool-riscv64-unknown-linux-gnu-2.4.7.drv' failed with exit code 1;

In that old 9af373a libtool used freshly built libtoolize to generate manpage and failed for some users.

If there is no immediate reason to patch libtoolize it might be worth tweaking shebang mangling and defet it to install phase.

[ghost]

[when cross-compiling]

is present in branches before 6becbd3 When help2man call is still present we call libtoolize with wrong bash interpreter: host one instead of build

It's very suspicious that I never encountered this. My laptop and routers are now 100% cross-compiled, and have been at-least-mostly-cross-compiled for quite some time. Since the code in question is over six months old, and not part of the current release, I'll just take your word for it.

If there is no immediate reason to patch libtoolize

There is: it's explained in the comment above in the code that does the patching. There's a reference to the libtool commit that made it necessary.

it might be worth tweaking shebang mangling

Write a PR -- it might look easier than it really is, and that's the only way to find out.

[trofi]

[when cross-compiling]

is present in branches before 6becbd3 When help2man call is still present we call libtoolize with wrong bash interpreter: host one instead of build

It's very suspicious that I never encountered this. My laptop and routers are now 100% cross-compiled, and have been at-least-mostly-cross-compiled for quite some time. Since the code in question is over six months old, and not part of the current release, I'll just take your word for it.

It required broken binfmt_misc interpreter to expose the bug. No need to take my word for it. We can observe the fact that libtoool relied on running foreign binaries by bringing hepl2man dependency back and faking an unhandled error from libtoolize.in against master:

--- a/pkgs/development/tools/misc/libtool/libtool2.nix
+++ b/pkgs/development/tools/misc/libtool/libtool2.nix
@@ -2,4 +2,5 @@
 , runtimeShell
 , file
+, help2man
 }:

@@ -33,4 +34,13 @@ stdenv.mkDerivation rec {
     # avoid help2man run after 'libtoolize.in' update
     touch doc/libtoolize.1
+  '' + lib.optionalString stdenv.targetPlatform.isRiscV ''
+    # force help2man run to expose wrong call of libtoolize with
+    # unsupported "host" interpreter.
+
+    # make sure we don't report "No such file or directory" but expose
+    # "broken interpreter". We should not run interpreters anyway.
+    sed -i 'exit 42' libtoolize.in
+    sed -i 'i1 echo "UNSUPPORTED #!${runtimeShell}" interpreter"' libtoolize.in
+    rm doc/libtoolize.1
   '';

@@ -39,5 +49,5 @@ stdenv.mkDerivation rec {
   # add autoconf and automake or help2man dependencies here. That way we can
   # avoid pulling in perl and get away with just an `m4` depend.
-  nativeBuildInputs = [ m4 file ];
+  nativeBuildInputs = [ m4 file ] ++ lib.optionals stdenv.targetPlatform.isRiscV [ help2man ];
   propagatedBuildInputs = [ m4 file ];

This fails as:

$ nix build --no-link -f. pkgsCross.riscv64.libtool -L
...
libtool-riscv64-unknown-linux-gnu>   GEN      doc/libtoolize.1
libtool-riscv64-unknown-linux-gnu> help2man: can't get `--help' info from libtoolize
libtool-riscv64-unknown-linux-gnu> Try `--no-discard-stderr' if option outputs to stderr

If there is no immediate reason to patch libtoolize

There is: it's explained in the comment above in the code that does the patching. There's a reference to the libtool commit that made it necessary.

it might be worth tweaking shebang mangling

Write a PR -- it might look easier than it really is, and that's the only way to find out.

I'll have to decline. Past experience with certain packages shows that there is no agreement on the final solution (usually after hours sunk into preparing and testing the fix). I'll leave the exercise to others.