error: export ordinal too large: 78056 when cross-compiling using GCC 11 in MXE

[HuidaeCho]

Expected behavior and actual behavior.

Expected behavior

I try to cross-compile GDAL 3.3.2 using GCC 11 in MXE. It should compile successfully at least using configure --with-hide-internal-symbols with a reduced number of symbols less than 65536, which is the limit for Windows DLLs.

Actual behavior

In almost (?) the last step libtool --mode=link ... -o libgdal.la fails with mxe/usr/bin/x86_64-w64-mingw32.shared-ld: error: export ordinal too large: 78056.

I found this ticket and tried configure --with-hide-internal-symbols. It added -fvisibility=hidden to CFLAGS and compiled object files with this flag. All was good until the same last linking step where I got exactly the same error with the same number of symbols greater than 65536.

Steps to reproduce the problem.

GCC 11 with binutils 2.36

• gcc 11.2.0
• binutils 2.36.1

git clone https://github.com/HuidaeCho/mxe.git
cd mxe

# this branch has updated dependencies
git checkout updated

# let's try --with-hide-internal-symbols
sed -i 's/\(--with-pg=no\)/\1 --with-hide-internal-symbols=yes/' src/gdal.mk

cat<<EOT > settings.mk
MXE_TARGETS := x86_64-w64-mingw32.shared
override MXE_PLUGIN_DIRS += plugins/gcc11
EOT

make gdal

GCC 9 with binutils 2.28

• gcc 9.3.0
• binutils 2.28

Now, if you use GCC 9 with binutils 2.28, it seems to compile fine, BUT binutils 2.28 doesn't check for the Windows limit of 65536 symbols and can create broken DLLs. In the same mxe directory,

cat<<EOT > settings.mk
MXE_TARGETS := x86_64-w64-mingw32.shared
override MXE_PLUGIN_DIRS += plugins/gcc9
EOT
make gdal

I used Dependency Walker to check the number of symbols that this successfully (?) compiled DLL exports. It hit the max ordinal number of 65535, so I believe that this DLL may be broken because Dependency Walker doesn't seem to display more than 65536 symbols (?).

This screenshot shows the first several symbols.

Compared to GDAL 2.2.4 which was compiled successfully and only exports 43784 symbols (below), the list for 3.3.2 has many N/A O# (resolved ordinal imports) along with C (resolved C imports) for the same ordinal numbers. Maybe, that's what GCC 11 does?

Operating system

Updated MXE master branch on Slackware current

GDAL version and provenance

The 3.3.2 version from http://download.osgeo.org/gdal/3.3.2/gdal-3.3.2.tar.xz

[rouault]

ok, I've reproduced that issue when building in non-optimized mode with "x86_64-w64-mingw32-gcc-posix (GCC) 9.3-posix 20200320". Can you check the CFLAGS in GDALmake.opt to see if they include -O2 ?

[shawnlaffan]

I've hit the same issue in the past. I reduced the number of drivers being built and skipped gnm to get it back under the limit but it would be nice to not have to worry about such things.

There are no occurrences of -O in GDALmake.opt in the build.

Full CFLAGS line is below.

For context, this is from the build system that underlies the perl bindings in Geo::GDAL::FFI. The calls on Windows are run via MSYS1 but the gcc, ld etc are from x86_64-w64-mingw32.

CFLAGS		= -DHAVE_AVX_AT_COMPILE_TIME -DHAVE_SSSE3_AT_COMPILE_TIME -DHAVE_SSE_AT_COMPILE_TIME -IC:/shawn/perls/5.32.1.1_PDL/perl/site/lib/auto/share/dist/Alien-sqlite/include -IC:/shawn/perls/5.32.1.1_PDL/perl/site/lib/auto/share/dist/Alien-sqlite/include -IC:/shawn/perls/5.32.1.1_PDL/perl/site/lib/auto/share/dist/Alien-proj/include -I/include -fvisibility=hidden -Wall  -Wextra -Winit-self -Wunused-parameter -Wmissing-prototypes -Wmissing-declarations -Wformat -Werror=format-security -Wno-format-nonliteral -Wlogical-op -Wshadow -Werror=vla -Wdate-time -Wnull-dereference -Wduplicated-cond -Wfloat-conversion $(USER_DEFS)

[rouault]

There are no occurrences of -O in GDALmake.opt in the build.

that's definitely an issue. You want a -O2 build for production speed ! And that would also likely solve the issue about the number of symbols

[shawnlaffan]

Yes indeed. But should they come from GDAL or the calling system?

[rouault]

But should they come from GDAL or the calling system?

a regular run of ./configure should include the -O2 flag. Perhaps something is overriding the CFLAGS/CXXFLAGS when invoking ./configure ?

[shawnlaffan]

It does looks like CFLAGS and CXXFLAGS are being overridden, probably somewhere in Alien::Build.

Example build logs from a failed run are here:
https://github.com/shawnlaffan/geo-gdal-ffi-canary/runs/5029680620?check_suite_focus=true#step:15:113
https://github.com/shawnlaffan/geo-gdal-ffi-canary/runs/5029680620?check_suite_focus=true#step:15:313

The build completes after changing the configure call to include CFLAGS='-O2' CXXFLAGS='-O2'.

More generally, should GDAL allow such an override? i.e. would it be better to append to CFLAGS and CXXFLAGS if their respective env vars are set?

[rouault]

It does looks like CFLAGS and CXXFLAGS are being overridden, probably somewhere in Alien::Build.

indeed, the build log shows:
CFLAGS="-IC:/cx/lib/perl5/MSWin32-x64-multi-thread/auto/share/dist/Alien-sqlite/include -IC:/cx/lib/perl5/MSWin32-x64-multi-thread/auto/share/dist/Alien-sqlite/include -IC:/cx/lib/perl5/MSWin32-x64-multi-thread/auto/share/dist/Alien-proj/include -IC:/cx/lib/perl5/MSWin32-x64-multi-thread/auto/share/dist/Alien-geos-af/include"

More generally, should GDAL allow such an override?

that's the standard behaviour with autotools. Users overriding them are responsible of including the right optimization level.
Anyway with the new CMake build system that will be in GDAL 3.5.0, for non-optimized builds with mingw, the -Os optimisation is automatically added to avoid that issue.

I believe we can close this issue

[shawnlaffan]

Thanks @rouault, and good to hear the cmake system won't have this problem.