Build issues since 5cdca5b1da7884c54260db473f892354bc79951b #13459

adamdmoss · 2022-05-14T01:57:33Z

System information

Type	Version/Name
Distribution Name	Ubuntu
Distribution Version	20.04
Kernel Version	5.13.0-28-lowlatency
Architecture	x64
OpenZFS Version	`5cdca5b`

Describe the problem you're observing

$ ./autogen.sh
...
$ ./configure --disable-silent-rules --enable-debuginfo --with-config=srpm
...
$ V=1 make -j16 deb-utils
...
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing po-directories commands
+ make -j8
make[2]: Entering directory '/tmp/zfs-build-adam-3eGSS2Ak/BUILD/zfs-2.1.99'
make[2]: warning: -j8 forced in submake: resetting jobserver mode.
./scripts/make_gitrev.sh include/zfs_gitrev.h
/bin/bash: ./scripts/make_gitrev.sh: No such file or directory
make[2]: *** [Makefile:13327: gitrev] Error 127
make[2]: Leaving directory '/tmp/zfs-build-adam-3eGSS2Ak/BUILD/zfs-2.1.99'
error: Bad exit status from /tmp/zfs-build-adam-3eGSS2Ak/TMP/rpm-tmp.4QyoI8 (%build)


RPM build errors:
    Bad exit status from /tmp/zfs-build-adam-3eGSS2Ak/TMP/rpm-tmp.4QyoI8 (%build)
make[1]: *** [Makefile:13490: rpm-common] Error 1
make[1]: Leaving directory '/zfs2/home/adam/code/zfs'
make: *** [Makefile:13449: rpm-utils-initramfs] Error 2

Describe how to reproduce the problem

Try to build any zfs master from 5cdca5b onwards. On my system, I guess. 😁

The text was updated successfully, but these errors were encountered:

adamdmoss · 2022-05-14T02:03:48Z

FWIW:

$ ls /tmp/zfs-build-adam-3eGSS2Ak/BUILD/zfs-2.1.99/scripts 
Makefile.am  zfs-helpers.sh  zfs.sh  zfs-tests.sh  zimport.sh  zloop.sh

make_gitrev.sh is indeed no longer copied to the temporary build directory. I can't quite figure out if 5cdca5b stops it being copied there, or is erroneously trying to run it from there instead of the source tree.

adamdmoss · 2022-05-14T02:54:34Z

I believe the top-level Makefile.am which defines EXTRA_DIST correctly includes scripts/Makefile.am amongst others, and they all add entries to EXTRA_DIST as required, including an entry for scripts/make_gitrev.sh.

However, one of the last things included is tests/Makefile.am which indirectly includes tests/zfs-tests/tests/Makefile.am which clears EXTRA_DIST rather than adding to it. So scripts/make_gitrev.sh (amongst others) incorrectly ends up not included in EXTRA_DIST after all by the time it's used.

At least, that's as far as I can figure things.

adamdmoss · 2022-05-14T02:56:26Z

FAO @nabijaczleweli if I'm right. Or even if I'm wrong. 😆

nabijaczleweli · 2022-05-14T12:47:32Z

Your analysis is indeed wrong, tests/zfs-tests/tests/Makefile.am is SUBDIRSed, not included (and automake errors out on =-assigning to a prevously-defined variable precisely to catch this).

Pre-unification scripts/Makefile.am says

EXTRA_SCRIPTS = \
        commitcheck.sh \
        common.sh.in \
        dkms.mkconf \
        dkms.postbuild \
        kmodtool \
        make_gitrev.sh \
        man-dates.sh \
        paxcheck.sh \
        mancheck.sh \
        zfs-tests-color.sh

EXTRA_DIST = \
        cstyle.pl \
        enum-extract.pl \
        zfs2zol-patch.sed \
        zol2zfs-patch.sed \
        $(EXTRA_SCRIPTS)

and now it's

EXTRA_SCRIPTS = \
	%D%/commitcheck.sh \
	%D%/common.sh.in \
	%D%/dkms.mkconf \
	%D%/dkms.postbuild \
	%D%/kmodtool \
	%D%/make_gitrev.sh \
	%D%/man-dates.sh \
	%D%/mancheck.sh \
	%D%/paxcheck.sh \
	%D%/zfs-tests-color.sh

EXTRA_DIST += \
	%D%/cstyle.pl \
	%D%/enum-extract.pl \
	%D%/zfs2zol-patch.sed \
	%D%/zol2zfs-patch.sed \
	$(EXTRA_SCRIPTS)

SHELLCHECKSCRIPTS += $(dist_scripts_SCRIPTS) $(EXTRA_SCRIPTS)

which is equivalent, and make dist lists it

nabijaczleweli@tarta:~/store/code/zfs$ tar -tvaf zfs-2.1.99.tar.gz | grep _gitrev
-rw-r--r-- nabijaczleweli/users       58 2022-05-14 14:41 zfs-2.1.99/include/zfs_gitrev.h
-rwxr-xr-x nabijaczleweli/users     2057 2022-05-10 23:44 zfs-2.1.99/scripts/make_gitrev.sh

That said, it's a source, not a build artifact, so it lives in the source directory, not the build one, which, to me, points to a broken build system in whatever makes the rpms, since we do invoke it via top_srcdir:

nabijaczleweli@tarta:~/store/code/zfs$ git grep 'scripts/make_gitrev.sh '
Makefile.am:    $(AM_V_GEN)$(top_srcdir)/scripts/make_gitrev.sh $(GITREV)
Makefile.am:    $(top_srcdir)/scripts/make_gitrev.sh -D $(distdir) $(GITREV)

(and likewise pre-unification):

nabijaczleweli@tarta:~/store/code/zfs$ git show 6fc34371e1df4d74dee6e50ffd9d9fae234b55ba:Makefile.am  | grep make_gitrev
        $(AM_V_GEN)$(top_srcdir)/scripts/make_gitrev.sh $(GITREV)
        $(AM_V_GEN)$(top_srcdir)/scripts/make_gitrev.sh -D $(distdir) $(GITREV)

To confirm, can you grep the resulting Makefile for -e 'srcdir =' -e 'builddir ='? They should be different.

adamdmoss · 2022-05-14T18:54:13Z

Your analysis is indeed wrong

figures. why'd I even try :)

To confirm, can you grep the resulting Makefile for -e 'srcdir =' -e 'builddir ='? They should be different.

~/code/zfs % grep -e 'srcdir =' -e 'builddir =' Makefile
abs_builddir = /home/adam/code/zfs
abs_srcdir = /home/adam/code/zfs
abs_top_builddir = /home/adam/code/zfs
abs_top_srcdir = /home/adam/code/zfs
builddir = .
srcdir = .
top_builddir = .
top_srcdir = .

~/code/zfs % grep -e 'srcdir =' -e 'builddir =' /tmp/zfs-build-adam-Tjthdc9K/BUILD/zfs-2.1.99/Makefile
abs_builddir = /tmp/zfs-build-adam-Tjthdc9K/BUILD/zfs-2.1.99
abs_srcdir = /tmp/zfs-build-adam-Tjthdc9K/BUILD/zfs-2.1.99
abs_top_builddir = /tmp/zfs-build-adam-Tjthdc9K/BUILD/zfs-2.1.99
abs_top_srcdir = /tmp/zfs-build-adam-Tjthdc9K/BUILD/zfs-2.1.99
builddir = .
srcdir = .
top_builddir = .
top_srcdir = .

Thanks.

nabijaczleweli · 2022-05-14T19:14:35Z

hm hm hm.Hm. Hm. Hm. On trunk:

+ make -j24
make[2]: Entering directory '/tmp/zfs-build-nabijaczleweli-GWaNpyv9/BUILD/zfs-2.1.99'
make[2]: warning: -j24 forced in submake: resetting jobserver mode.
./scripts/make_gitrev.sh include/zfs_gitrev.h
/bin/bash: line 1: ./scripts/make_gitrev.sh: No such file or directory
make[2]: *** [Makefile:13293: gitrev] Error 127
make[2]: Leaving directory '/tmp/zfs-build-nabijaczleweli-GWaNpyv9/BUILD/zfs-2.1.99'
error: Bad exit status from /tmp/zfs-build-nabijaczleweli-GWaNpyv9/TMP/rpm-tmp.LeLUyv (%build)
RPM build errors:
    line 99: It's not recommended to have unversioned Obsoletes: Obsoletes:      spl
    line 144: It's not recommended to have unversioned Obsoletes: Obsoletes:      libzpool2
    line 145: It's not recommended to have unversioned Obsoletes: Obsoletes:      libzpool4
    line 161: It's not recommended to have unversioned Obsoletes: Obsoletes:      libnvpair1
    line 179: It's not recommended to have unversioned Obsoletes: Obsoletes:      libuutil1
    line 205: It's not recommended to have unversioned Obsoletes: Obsoletes:      libzfs2
    line 206: It's not recommended to have unversioned Obsoletes: Obsoletes:      libzfs4
    line 228: It's not recommended to have unversioned Obsoletes: Obsoletes:      zfs-devel
    line 229: It's not recommended to have unversioned Obsoletes: Obsoletes:      libzfs2-devel
    line 230: It's not recommended to have unversioned Obsoletes: Obsoletes:      libzfs4-devel
    Bad exit status from /tmp/zfs-build-nabijaczleweli-GWaNpyv9/TMP/rpm-tmp.LeLUyv (%build)
make[1]: *** [Makefile:13458: rpm-common] Error 1
make[1]: Leaving directory '/home/nabijaczleweli/uwu/a3232e2ea610ba2c601f2c196b2f7b75f3ba17ad'
make: *** [Makefile:13417: rpm-utils-initramfs] Error 2
nabijaczleweli@tarta:~/uwu/a3232e2ea610ba2c601f2c196b2f7b75f3ba17ad$ tar -tvaf zfs-2.1.99.tar.gz | grep gitrev
-rw-r--r-- nabijaczleweli/users  58 2022-05-14 21:07 zfs-2.1.99/include/zfs_gitrev.h

But it is distributed pre-unification:

make[2]: Entering directory '/tmp/zfs-build-nabijaczleweli-rWELcgKZ/BUILD/zfs-2.1.99'
make[2]: warning: -j24 forced in submake: resetting jobserver mode.
./scripts/make_gitrev.sh include/zfs_gitrev.h
make  all-recursive
make[3]: Entering directory '/tmp/zfs-build-nabijaczleweli-rWELcgKZ/BUILD/zfs-2.1.99'
nabijaczleweli@tarta:~/uwu/6fc34371e1df4d74dee6e50ffd9d9fae234b55ba$ tar -tvaf zfs-2.1.99.tar.gz | grep gitrev
-rwxr-xr-x nabijaczleweli/users     2057 2022-05-14 21:10 zfs-2.1.99/scripts/make_gitrev.sh
-rw-r--r-- nabijaczleweli/users       58 2022-05-14 21:23 zfs-2.1.99/include/zfs_gitrev.h
nabijaczleweli@tarta:~/uwu/6fc34371e1df4d74dee6e50ffd9d9fae234b55ba$ tar -xOaf zfs-2.1.99.tar.gz zfs-2.1.99/include/zfs_gitrev.h
#define ZFS_META_GITREV "zfs-2.1.99-1149-g6fc34371e-dist"

Hm. I am not getting it in the distribution with that config, but I am with my usual config. hhmmhmhm

nabijaczleweli · 2022-05-14T20:38:23Z

nabijaczleweli@tarta:~/uwu/bison$ git bisect good
0a8b1fc6251af5d16d9f87ce696ec6f04edaba31 is the first bad commit
commit 0a8b1fc6251af5d16d9f87ce696ec6f04edaba31
Author: наб <nabijaczleweli@nabijaczleweli.xyz>
Date:   Sat Apr 9 18:35:53 2022 +0200

    autoconf: use include directives instead of recursing down scripts

    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
    Closes #13316

 Makefile.am         |  10 +++--
 configure.ac        |   1 -
 scripts/Makefile.am | 118 ++++++++++++++++++++++++++--------------------------
 3 files changed, 65 insertions(+), 64 deletions(-)

and

nabijaczleweli@tarta:~/uwu/bison$ tail -2 Makefile
printpsko:
        @echo $(EXTRA_DIST)
nabijaczleweli@tarta:~/uwu/bison$ make printpsko
autogen.sh copy-builtin config/config.awk config/rpm.am config/deb.am config/tgz.am AUTHORS CODE_OF_CONDUCT.md COPYRIGHT LICENSE META NEWS NOTICE README.md RELEASES.md module/lua/README.zfs module/os/linux/spl/README.md module/icp/algs/skein/THIRDPARTYLICENSE module/icp/algs/skein/THIRDPARTYLICENSE.descrip module/icp/asm-x86_64/aes/THIRDPARTYLICENSE.gladman module/icp/asm-x86_64/aes/THIRDPARTYLICENSE.gladman.descrip module/icp/asm-x86_64/aes/THIRDPARTYLICENSE.openssl module/icp/asm-x86_64/aes/THIRDPARTYLICENSE.openssl.descrip module/icp/asm-x86_64/modes/THIRDPARTYLICENSE.cryptogams module/icp/asm-x86_64/modes/THIRDPARTYLICENSE.cryptogams.descrip module/icp/asm-x86_64/modes/THIRDPARTYLICENSE.openssl module/icp/asm-x86_64/modes/THIRDPARTYLICENSE.openssl.descrip module/os/linux/spl/THIRDPARTYLICENSE.gplv2 module/os/linux/spl/THIRDPARTYLICENSE.gplv2.descrip module/zfs/THIRDPARTYLICENSE.cityhash module/zfs/THIRDPARTYLICENSE.cityhash.descrip

Note the lack of scripts/ stuff here! And, indeed,

#scriptsdir = $(datadir)/$(PACKAGE)
#dist_scripts_SCRIPTS = \
#       scripts/zfs-helpers.sh \
#       scripts/zfs-tests.sh \
#       scripts/zfs.sh \
#       scripts/zimport.sh \
#       scripts/zloop.sh

#EXTRA_SCRIPTS = \
#       scripts/commitcheck.sh \
#       scripts/common.sh.in \
#       scripts/dkms.mkconf \
#       scripts/dkms.postbuild \
#       scripts/kmodtool \
#       scripts/make_gitrev.sh \
#       scripts/man-dates.sh \
#       scripts/mancheck.sh \
#       scripts/paxcheck.sh \
#       scripts/zfs-tests-color.sh

is commented out because scripts/ requires CONFIG_USER. Hm.

This also affects EXTRA_DIST += $(addprefix %D%/,libzfsbootenv.abi libzfsbootenv.suppr), for example, but not the source itself? And it's still under CONFIG_USER when SUBDIRSing but it's included then??? Real normal hours.

nabijaczleweli · 2022-05-14T23:08:57Z

haha yeah. given /a/Makefile.am

EXTRA_DIST = amongus.jpeg
bin_PROGRAMS = amongus

and /Makefile.am.1

if CONFIG_FALSE
SUBDIRS = a
endif

vs /Makefile.am.2

if CONFIG_FALSE
include $(srcdir)/%D%/a/Makefile.am
endif

make dist of .1 will contain

a/amongus.c
a/amongus.jpeg

while the .2 one will only contain

a/amongus.c

And, AFAICT from the documentation, there's no actual way to do, say, "ALWAYS_EXTRA_DIST" or whatever. Normal day in autohell^Wautoreconf specifically.

nabijaczleweli · 2022-05-15T15:28:44Z

https://bugs.debian.org/1011024

Workaround-for: https://bugs.debian.org/1011024 Closes: openzfs#13459

nabijaczleweli · 2022-05-15T18:08:07Z

@adamdmoss can you try the top commit off https://github.com/openzfs/zfs/pull/13447/files?file-filters%5B%5D=.sh?

Workaround-for: https://bugs.debian.org/1011024 Closes: openzfs#13459

Workaround-for: https://bugs.debian.org/1011024 Closes: openzfs#13459 Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>

adamdmoss · 2022-05-15T23:00:20Z

That totally appears to work! Thanks!

thesamesam · 2022-05-15T23:28:41Z

Relying on downstreams to call autogen.sh feels janky. A lot of us intentionally don't do it given it's usually unnecessary and/or duplicates steps.

Workaround-for: https://bugs.debian.org/1011024 Closes: openzfs#13459 Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>

behlendorf · 2022-05-18T18:18:40Z

Rather than processing the Makefile.in what if we instead consolidated all of the EXTRA_DIST entries under if CONFIG_USER in to the top-level Makefile.am. We already do this for the the module sub-directory. Or potentially a dist-hook could be used for this. I tend to agree that putting this logic in the autogen.sh feels like the wrong approach.

Workaround-for: https://bugs.debian.org/1011024 Closes: openzfs#13459 Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>

The EXTRA_DIST variable is ignored when used in the FALSE conditional of a Makefile.am. This results in the `make dist` target omitting these files from the generated tarball unless CONFIG_USER is defined. This issue can be avoided by switching to use the dist_noinst_DATA variable which is handled as expected by autoconf. This change also adds support for --with-config=dist as an alias for --with-config=srpm. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes openzfs#13459

The EXTRA_DIST variable is ignored when used in the FALSE conditional of a Makefile.am. This results in the `make dist` target omitting these files from the generated tarball unless CONFIG_USER is defined. This issue can be avoided by switching to use the dist_noinst_DATA variable which is handled as expected by autoconf. This change also adds support for --with-config=dist as an alias for --with-config=srpm and updates the GitHub workflows to use it. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes openzfs#13459

The EXTRA_DIST variable is ignored when used in the FALSE conditional of a Makefile.am. This results in the `make dist` target omitting these files from the generated tarball unless CONFIG_USER is defined. This issue can be avoided by switching to use the dist_noinst_DATA variable which is handled as expected by autoconf. This change also adds support for --with-config=dist as an alias for --with-config=srpm and updates the GitHub workflows to use it. Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #13459 Closes #13505

The EXTRA_DIST variable is ignored when used in the FALSE conditional of a Makefile.am. This results in the `make dist` target omitting these files from the generated tarball unless CONFIG_USER is defined. This issue can be avoided by switching to use the dist_noinst_DATA variable which is handled as expected by autoconf. This change also adds support for --with-config=dist as an alias for --with-config=srpm and updates the GitHub workflows to use it. Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes openzfs#13459 Closes openzfs#13505

nabijaczleweli · 2022-10-11T07:46:25Z

So don't, the patched Makefile.in ends up in the make dist tarball, you can configure off it directly (as I assume you already are).

nabijaczleweli · 2022-10-11T11:29:57Z

Sorry, I'm having my e-mail replies I literally posted months ago drain rn — neomutt/neomutt#3498 (comment) — apparently defence contractor money is insufficient to process mail in less than five months; I posted this May 16th; seems to be the same thing in #12988 (comment)

Workaround-for: https://bugs.debian.org/1011024 Closes: openzfs#13459 Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>

adamdmoss added the Type: Defect Incorrect behavior (e.g. crash, hang) label May 14, 2022

nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue May 15, 2022

Manually unbreak if 0, EXTRA_DIST = (and dist_xxx_MANS =) being ignored

9f0fab9

Workaround-for: https://bugs.debian.org/1011024 Closes: openzfs#13459

nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue May 15, 2022

Manually unbreak if 0, EXTRA_DIST = (and dist_xxx_MANS =) being ignored

2f6ecb7

Workaround-for: https://bugs.debian.org/1011024 Closes: openzfs#13459

nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this issue May 15, 2022

Manually unbreak if 0, EXTRA_DIST = (and dist_xxx_MANS =) being ignored

84760ce

Workaround-for: https://bugs.debian.org/1011024 Closes: openzfs#13459

nabijaczleweli mentioned this issue May 19, 2022

Manually unbreak if 0, EXTRA_DIST = (and dist_xxx_MANS =) being ignored #13481

Closed

13 tasks

behlendorf mentioned this issue May 25, 2022

Replace EXTRA_DIST with dist_noinst_DATA #13505

Merged

13 tasks

behlendorf closed this as completed in #13505 May 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build issues since 5cdca5b1da7884c54260db473f892354bc79951b #13459

Build issues since 5cdca5b1da7884c54260db473f892354bc79951b #13459

adamdmoss commented May 14, 2022 •

edited

Loading

adamdmoss commented May 14, 2022 •

edited

Loading

adamdmoss commented May 14, 2022 •

edited

Loading

adamdmoss commented May 14, 2022

nabijaczleweli commented May 14, 2022 •

edited

Loading

adamdmoss commented May 14, 2022

nabijaczleweli commented May 14, 2022 •

edited

Loading

nabijaczleweli commented May 14, 2022 •

edited

Loading

nabijaczleweli commented May 14, 2022 •

edited

Loading

nabijaczleweli commented May 15, 2022

nabijaczleweli commented May 15, 2022

adamdmoss commented May 15, 2022

thesamesam commented May 15, 2022 •

edited

Loading

behlendorf commented May 18, 2022

nabijaczleweli commented Oct 11, 2022 via email

nabijaczleweli commented Oct 11, 2022 •

edited

Loading

Build issues since 5cdca5b1da7884c54260db473f892354bc79951b #13459

Build issues since 5cdca5b1da7884c54260db473f892354bc79951b #13459

Comments

adamdmoss commented May 14, 2022 • edited Loading

System information

Describe the problem you're observing

Describe how to reproduce the problem

adamdmoss commented May 14, 2022 • edited Loading

adamdmoss commented May 14, 2022 • edited Loading

adamdmoss commented May 14, 2022

nabijaczleweli commented May 14, 2022 • edited Loading

adamdmoss commented May 14, 2022

nabijaczleweli commented May 14, 2022 • edited Loading

nabijaczleweli commented May 14, 2022 • edited Loading

nabijaczleweli commented May 14, 2022 • edited Loading

nabijaczleweli commented May 15, 2022

nabijaczleweli commented May 15, 2022

adamdmoss commented May 15, 2022

thesamesam commented May 15, 2022 • edited Loading

behlendorf commented May 18, 2022

nabijaczleweli commented Oct 11, 2022 via email

nabijaczleweli commented Oct 11, 2022 • edited Loading

adamdmoss commented May 14, 2022 •

edited

Loading

adamdmoss commented May 14, 2022 •

edited

Loading

adamdmoss commented May 14, 2022 •

edited

Loading

nabijaczleweli commented May 14, 2022 •

edited

Loading

nabijaczleweli commented May 14, 2022 •

edited

Loading

nabijaczleweli commented May 14, 2022 •

edited

Loading

nabijaczleweli commented May 14, 2022 •

edited

Loading

thesamesam commented May 15, 2022 •

edited

Loading

nabijaczleweli commented Oct 11, 2022 •

edited

Loading