Makefile name collision

[teamblubee]

I am running into issues building openblas where the Makefile variable $(ARCH) has two different meanings.

1)architecture; x86, x86_64, etc..
2)ar command

Testing this project in FreeBSD jails I run into a bug where the host is detected as amd64 then the Makefile fails as there's no Makefile.amd64 okay setting ARCH to x86_64 then later the build fails because "x86_64" is not a valid command; In this case $(ARCH) should be the archive command ar.

I hacked together this patch to get the project to build but I hope that something more general could be implemented.

--- Makefile.system.orig	2018-10-07 17:11:10 UTC
+++ Makefile.system
@@ -1042,9 +1042,15 @@ else
 LIBPREFIX = lib$(LIBNAMEBASE)
 endif
 
-KERNELDIR	= $(TOPDIR)/kernel/$(ARCH)
 
+ifneq ($(ARCH), "amd64")
+include $(TOPDIR)/Makefile.x86_64
+KERNELDIR	= $(TOPDIR)/kernel/x86_64
+ARCH		= x86_64
+else
 include $(TOPDIR)/Makefile.$(ARCH)
+KERNELDIR	= $(TOPDIR)/kernel/$(ARCH)
+endif
 
 CCOMMON_OPT	+= -DASMNAME=$(FU)$(*F) -DASMFNAME=$(FU)$(*F)$(BU) -DNAME=$(*F)$(BU) -DCNAME=$(*F) -DCHAR_NAME=\"$(*F)$(BU)\" -DCHAR_CNAME=\"$(*F)\"

--- lapack/laswp/Makefile.orig	2018-10-07 18:35:51 UTC
+++ lapack/laswp/Makefile
@@ -16,7 +16,11 @@ slaswp_plus.$(PSUFFIX) slaswp_minus.$(PSUFFIX) dlaswp_
 qlaswp_plus.$(PSUFFIX) qlaswp_minus.$(PSUFFIX) \
 claswp_plus.$(PSUFFIX) claswp_minus.$(PSUFFIX) zlaswp_plus.$(PSUFFIX) zlaswp_minus.$(PSUFFIX) \
 xlaswp_plus.$(PSUFFIX) xlaswp_minus.$(PSUFFIX) : dummy
+ifneq ($(ARCH), "x86_64")
+	cd x86_64 && $(MAKE) ../$(@F)
+else
 	cd $(ARCH) && $(MAKE) ../$(@F)
+endif
 
 include ../../Makefile.tail
 

--- lapack-netlib/LAPACKE/src/Makefile.orig	2018-10-07 18:54:49 UTC
+++ lapack-netlib/LAPACKE/src/Makefile
@@ -2457,16 +2457,16 @@ all: ../../$(LAPACKELIB)
 .PHONY: ../../$(LAPACKELIB)
 
 ../../$(LAPACKELIB): $(OBJ_A) $(OBJ_B) $(DEPRECATED) $(EXTENDED) $(MATGEN)
-	$(ARCH) $(ARCHFLAGS) $@ $(OBJ_A)
-	$(ARCH) $(ARCHFLAGS) $@ $(OBJ_B)
+	ar $(ARCHFLAGS) $@ $(OBJ_A)
+	ar $(ARCHFLAGS) $@ $(OBJ_B)
 ifdef BUILD_DEPRECATED
-	$(ARCH) $(ARCHFLAGS) $@ $(DEPRECATED)
+	ar $(ARCHFLAGS) $@ $(DEPRECATED)
 endif
 ifdef (USEXBLAS)
-	$(ARCH) $(ARCHFLAGS) $@ $(EXTENDED)
+	ar $(ARCHFLAGS) $@ $(EXTENDED)
 endif
 ifdef LAPACKE_WITH_TMG
-	$(ARCH) $(ARCHFLAGS) $@ $(MATGEN)
+	ar $(ARCHFLAGS) $@ $(MATGEN)
 endif
 	$(RANLIB) $@

--- lapack-netlib/LAPACKE/utils/Makefile.orig	2018-10-07 18:55:08 UTC
+++ lapack-netlib/LAPACKE/utils/Makefile
@@ -186,7 +186,7 @@ OBJ = lapacke_cgb_nancheck.o \
 all: lib
 
 lib: $(OBJ)
-	$(ARCH) $(ARCHFLAGS) ../../$(LAPACKELIB) $^
+	ar $(ARCHFLAGS) ../../$(LAPACKELIB) $^
 	$(RANLIB) ../../$(LAPACKELIB)
 
 clean: cleanobj

--- lapack-netlib/SRC/Makefile.orig	2018-10-07 18:35:58 UTC
+++ lapack-netlib/SRC/Makefile
@@ -555,26 +555,26 @@ all: ../$(LAPACKLIB)
 .PHONY: ../$(LAPACKLIB)
 
 ../$(LAPACKLIB): $(ALLOBJ) $(ALLXOBJ) $(DEPRECATED)
-	$(ARCH) $(ARCHFLAGS) $@ $(ALLOBJ) $(ALLXOBJ) $(DEPRECATED)
+	ar $(ARCHFLAGS) $@ $(ALLOBJ) $(ALLXOBJ) $(DEPRECATED)
 	$(RANLIB) $@
 
 single: $(SLASRC) $(DSLASRC) $(SXLASRC) $(SCLAUX) $(ALLAUX)
-	$(ARCH) $(ARCHFLAGS) ../$(LAPACKLIB) $(SLASRC) $(DSLASRC) \
+	ar $(ARCHFLAGS) ../$(LAPACKLIB) $(SLASRC) $(DSLASRC) \
 	$(SXLASRC) $(SCLAUX) $(ALLAUX)
 	$(RANLIB) ../$(LAPACKLIB)
 
 complex: $(CLASRC) $(ZCLASRC) $(CXLASRC) $(SCLAUX) $(ALLAUX)
-	$(ARCH) $(ARCHFLAGS) ../$(LAPACKLIB) $(CLASRC) $(ZCLASRC) \
+	ar $(ARCHFLAGS) ../$(LAPACKLIB) $(CLASRC) $(ZCLASRC) \
 	$(CXLASRC) $(SCLAUX) $(ALLAUX)
 	$(RANLIB) ../$(LAPACKLIB)
 
 double: $(DLASRC) $(DSLASRC) $(DXLASRC) $(DZLAUX) $(ALLAUX)
-	$(ARCH) $(ARCHFLAGS) ../$(LAPACKLIB) $(DLASRC) $(DSLASRC) \
+	ar $(ARCHFLAGS) ../$(LAPACKLIB) $(DLASRC) $(DSLASRC) \
 	$(DXLASRC) $(DZLAUX) $(ALLAUX)
 	$(RANLIB) ../$(LAPACKLIB)
 
 complex16: $(ZLASRC) $(ZCLASRC) $(ZXLASRC) $(DZLAUX) $(ALLAUX)
-	$(ARCH) $(ARCHFLAGS) ../$(LAPACKLIB) $(ZLASRC) $(ZCLASRC) \
+	ar $(ARCHFLAGS) ../$(LAPACKLIB) $(ZLASRC) $(ZCLASRC) \
 	$(ZXLASRC) $(DZLAUX) $(ALLAUX)
 	$(RANLIB) ../$(LAPACKLIB)

--- lapack-netlib/TESTING/MATGEN/Makefile.orig	2018-10-07 18:43:00 UTC
+++ lapack-netlib/TESTING/MATGEN/Makefile
@@ -60,23 +60,23 @@ ALLOBJ = $(SMATGEN) $(CMATGEN) $(SCATGEN) $(DMATGEN) $
 .PHONY: ../../$(TMGLIB)
 
 ../../$(TMGLIB): $(ALLOBJ)
-	$(ARCH) $(ARCHFLAGS) $@ $^
+	ar $(ARCHFLAGS) $@ $^
 	$(RANLIB) $@
 
 single: $(SMATGEN) $(SCATGEN)
-	$(ARCH) $(ARCHFLAGS) ../../$(TMGLIB) $^
+	ar $(ARCHFLAGS) ../../$(TMGLIB) $^
 	$(RANLIB) ../../$(TMGLIB)
 
 complex: $(CMATGEN) $(SCATGEN)
-	$(ARCH) $(ARCHFLAGS) ../../$(TMGLIB) $^
+	ar $(ARCHFLAGS) ../../$(TMGLIB) $^
 	$(RANLIB) ../../$(TMGLIB)
 
 double: $(DMATGEN) $(DZATGEN)
-	$(ARCH) $(ARCHFLAGS) ../../$(TMGLIB) $^
+	ar $(ARCHFLAGS) ../../$(TMGLIB) $^
 	$(RANLIB) ../../$(TMGLIB)
 
 complex16: $(ZMATGEN) $(DZATGEN)
-	$(ARCH) $(ARCHFLAGS) ../../$(TMGLIB) $^
+	ar $(ARCHFLAGS) ../../$(TMGLIB) $^
 	$(RANLIB) ../../$(TMGLIB)
 
 $(SCATGEN): $(FRC)

The main issue is caused by the Makefile variable name ARCH that has different meanings based on context.

[martin-frbg]

The AR/ARCH confusion is a result of LAPACK being a separate project - the lapack-netlib tree is imported from https://github.com/Reference-LAPACK/lapack. To me the fundamental problem appears to be that your build environment is autodetected as "amd64" although this should get mapped to "x86_64" within c_check since version 0.2.13, about 4 years ago. Which version of OpenBLAS are you trying to build ?

[teamblubee]

The AR/ARCH confusion is a result of LAPACK being a separate project - the lapack-netlib tree is imported from https://github.com/Reference-LAPACK/lapack. To me the fundamental problem appears to be that your build environment is autodetected as "amd64" although this should get mapped to "x86_64" within c_check since version 0.2.13, about 4 years ago. Which version of OpenBLAS are you trying to build ?

I am using this github repo commit: 6e2c494

[brada4]

It looks wrong. if arch is say aarch64 you should not build forcibly x86_64/amd64 assembly which will fail.

[teamblubee]

It looks wrong. if arch is say aarch64 you should not build forcibly x86_64/amd64 assembly which will fail.

Your right, my code changes was just a way to get the project to build on this current machine which is x86_64.

This patch would break other platforms. You understand this codebase a lot better than I do, I would hope that you could recommend a fix.

A quick solution is use different variable names. Having $(ARCH) return platform architecture during one phase of the build and $(ARCH) returning the archive command in another phase of the build puts one in a tough situation.

Would it be possible to be ARCH and ARCH_CMD or something to differentiate the variables?

[brada4]

It is a massive change, i'd say impossible.
ARCH is used for archiver only in imported LAPACK code, which does not care about it being CPU architecture. ARCH_CMD will not do, it is other project.

[teamblubee]

It is a massive change, i'd say impossible.
ARCH is used for archiver only in imported LAPACK code, which does not care about it being CPU architecture. ARCH_CMD will not do, it is other project.

There's no way to avoid this variable name collision?

[brada4]

Easy workaround is to install binary port in chroot jail.

Can you post build logs (record via "script", then upload/attach) from freebsd and from chroot jail?

[teamblubee]

Here's the build log for the successul chrooted build with the patches: https://transfer.sh/ks19G/12amd64-FLANG.log

build log without the patches, the build detects amd64 instead of x86_64 and fails:
https://transfer.sh/Q3PgN/12amd64-FLANG.log

Now after that failure I am logged into the chrooted env, if I go to the src directory and run gmake the build succeeds: https://transfer.sh/6W2ey/GMAKE

[martin-frbg]

At first glance the problem seems to be that you have ARCH set in the environment (which is probably legit but confuses the build system). I suspect this is what will need to be worked around.

[brada4]

Build log from build on a clean FreeBSD machine as unprivileged user?
What is the breaking difference between chroot and pure system...

[brada4]

Could you build netlib BLAS+LAPACK from ports with same "poudriere" configuration?
I think setting variable ARCH breaks that too.

[teamblubee]

The current ports tree is broken but I have fixed it in my local repo, it successfully builds in poudriere: https://transfer.sh/Rn6MG/12amd64-FLANG.log

[brada4]

Have you "fixed" anything , or simply identified regression that poudriere uses "popular" variable ARCH for some esoteric purpose?

PS #1779 and #1763 reflects work on revolutionizing the port.

[brada4]

Quick fix would be unset-ting ARCH environment variable in offended ports

[teamblubee]

the jail doesn't allow to unsetenv

root@12amd64-FLANG:~ # /bin/sh unsetenv
/bin/sh: cannot open unsetenv: No such file or directory

[brada4]

it is shell built-in
ksh/bash:
unset ARCH
csh
unsetenv ARCH

[teamblubee]

FreeBSD jails has several steps

pre-fetch

do-fetch

post-fetch

pre-extract

do-extract

post-extract

pre-patch

do-patch

post-patch

pre-configure

do-configure

post-configure

pre-build

do-build

post-build

pre-install

do-install

post-install

post-stage

pre-package

do-package

post-package

Where you can overwrite and apply custom commands. I've tried placing unsetenv HOST
it does not work, and gives the error I posted above.

After the build fails, then I get dropped into a shell that allows me to run unsetenv.

[martin-frbg]

Seems related to #1753 (which should already take care of the LAPACK make.inc part of the problem), just that ARCH may need to be overridden in Makefile.prebuid or Makefile.system

[brada4]

or a paranoid entrance to /Makefile that says to unset variables used internally?

[martin-frbg]

Can't reproduce your problem with current develop (though I am on Linux, setting ARCH=amd64 from outside should have the same unwanted effect there). Is "openblas-0.2.20" just an oversight in the package name quoted in your log, or could it be you accidentally tried to build that old version there ?

[teamblubee]

That's an artifact from the older code.

I am pulling directly from this repo and commit
GH_ACCOUNT= xianyi
GH_PROJECT= OpenBLAS
GH_TAGNAME= 6e2c494

[brada4]

@teamblubee any step invoking 'make' should be invoked without ARCH variable set.
Since you said many packages are affected, and there is no documentation of variable ARCH, nor it is in any "production release, I think it is right time to rename it to POUDRIERE_ARCH or something before anything more starts relying on it.

[teamblubee]

@teamblubee any step invoking 'make' should be invoked without ARCH variable set.
Since you said many packages are affected, and there is no documentation of variable ARCH, nor it is in any "production release, I think it is right time to rename it to POUDRIERE_ARCH or something before anything more starts relying on it.

Let me bring this up on that project and see how that goes.

[brada4]

Thanks, any feedback welcome.
v11 is not affected at all.
v12 - If you have chance, scan (failed) build logs for any mention of ARCH outside initial environment, that is clear sign it breaks stuff. As we found here reference lapack & openblas are known to be affected. If no general solution is achieved I am afraid one has to unset ARCH piece by piece.

[brada4]

@martin-frbg @teamblubee I think worth updating topic with FreeBSD 12, to avoid general panic

[martin-frbg]

Hmm. I can now reproduce with make ARCH=amd64 but not with ARCH in the shell environment, and LAPACK make.inc appears unaffected. My current favorite workaround is putting

ifdef ARCH
ifeq ($(ARCH), amd64)
override ARCH=x86_64
endif
endif

near the top of Makefile.system (not sure if the outer ifdef is strictly necessary)

[brada4]

or

ifdef ARCH
override undefine ARCH
endif

[brada4]

Neither helps against environment. It seems to pick environemnt by preference anyway.

[martin-frbg]

@brada4 (undefine ARCH) nope, that would nuke the c_check result and end with "Makefile. not found".
(environment) fixable with the "override ARCH" from #1753 - which unfortunately went on the RISCV branch only.

[brada4]

Maybe it outputs a warning
I could not manage to override ARCH=zzz make ARCH=zzz or readonly ARCH=zzz ; make

[teamblubee]

I tested the latest commit: 697dc1b

The error still persists but after your examples I added this patch file to the Makefile and the project builds no problems

--- Makefile.system.orig	2018-10-09 06:59:21 UTC
+++ Makefile.system
@@ -9,6 +9,12 @@ ifndef TOPDIR
 TOPDIR = .
 endif
 
+ifdef ARCH
+ifeq ($(ARCH), amd64)
+override ARCH=x86_64
+endif
+endif
+
 NETLIB_LAPACK_DIR = $(TOPDIR)/lapack-netlib
 
 # Default C compiler

Can a change like this be upstreamed?

[martin-frbg]

Could it be that you picked up only the first half of what went in as PR #1799 ? I only merged it a few hours ago, and the part affecting Makefile.system apparently went in as c0d7cd3

[teamblubee]

Could it be that you picked up only the first half of what went in as PR #1799 ? I only merged it a few hours ago, and the part affecting Makefile.system apparently went in as c0d7cd3

Thank you, I can confirm that tag: c0d7cd3
fixes the issue.

I appreciate you taking the time to help!

[brada4]

For completeness should be also
i386 -> x86
aarch64 -> arm64
powerpc64 -> power
to match other architectures too

[martin-frbg]

Can we expect these names to show up at all ?

[brada4]

@teamblubee is in position to tell if amd64 crossbuilds for i386 what we do get ;)

[teamblubee]

@teamblubee is in position to tell if amd64 crossbuilds for i386 what we do get ;)

I do not have an i386 env setup at this moment, I'll create one and report back.

[teamblubee]

I won't be able to test this for a while as my dev toolchain has some issue and I am in the middle of another project; I will test this when I am able to update my toolchain.