Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

patches/util-linux libusb: patch configure scripts for reproducibility #1321

Closed

Conversation

tlaurion
Copy link
Collaborator

All hardcode_into_libs=yes -> hardcode_into_libs=no

Also

  • modules/libusb: remove --disable-tests (not recognized at configure step)

Adresses two modules that were not reproducible under linuxboot/heads-wiki#70

…de_into_libs=yes -> hardcode_into_libs=no

- modules/libusb: remove --disable-tests (not recognized at configure step)
@tlaurion
Copy link
Collaborator Author

Artifacts for same commit on CircleCI and local build do not show any reproducibility errors for util-linux and libusb.

Reminder:

  • tools.cpio packs all compiled modules libs and binaries
  • heads.cpio packs all scripts under initrd
  • modules.cpio packs all kernel modules not built-in the kernel
    All of the above is packed under initrd.cpio.xz

So downloading artifacts of a CircleCI for a specific commit and doing a diffoscope against a local build for same commit id gives good insight of reproducibility issues.


Still issues with other modules:

  • modules/htop-verification (/bin/hotp-verification)
  • modules/json-c (impacts /lib/libjson.c.so.5)
  • modules/qrencode (impacts /lib/libqrencode.so.3

@tlaurion
Copy link
Collaborator Author

@JonathonHall-Purism : please review!

@rbreslow
Copy link
Contributor

rbreslow commented Mar 2, 2023

The only thing I could find on Google matching hardcode_into_libs is documentation for libtool: https://ftp.gnu.org/old-gnu/Manuals/libtool-1.4.2/html_node/libtool_70.html.

Whether the platform supports hardcoding of run-paths into libraries. If enabled, linking of programs will be much simpler but libraries will need to be relinked during installation. Set to yes or no.

How does this affect reproducibility?

@tlaurion
Copy link
Collaborator Author

tlaurion commented Mar 2, 2023

How does this affect reproducibility?
@rbreslow : by removing paths to libs in produced binaries. Have not reported output.


@rbreslow let's show it by comparing x230-htop-maximized tools.cpio from master and this PR.
Reminder: everything modules is packed under tools.cpio prior of being packed alongside of heads.cpio and modules.cpio under initrd.cpio.xz (as reported under hashes.txt)

wget https://output.circle-artifacts.com/output/job/980953df-a9b8-44b2-ad04-f23ba4e13802/artifacts/0/build/x86/x230-hotp-maximized/tools.cpio -O /tmp/master_tools.cpio
wget https://output.circle-artifacts.com/output/job/29c3cb4d-e50a-4ee1-9ca4-d7eae542a3b8/artifacts/0/build/x86/x230-hotp-maximized/tools.cpio -O /tmp/1321_tools.cpio
diffoscope /tmp/master_tools.cpio /tmp/1321_tools.cpio --html-dir 1321
firefox 1321/index.html

Basically paths to lib disappear for libblkid.so.1:
2023-03-02-132317


No change for libusb.... hmmm

@JonathonHall-Purism
Copy link
Collaborator

hardcode_into_libs prevents writing an RPATH into the library file:

jhall@toaster:~/workspace/pureboot-boards/qemu/tools-extract$ readelf -d lib/libblkid.so.1 

Dynamic section at offset 0x40dc0 contains 26 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libuuid.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so]
 0x000000000000000e (SONAME)             Library soname: [libblkid.so.1]
 0x000000000000000f (RPATH)              Library rpath: [/home/jhall/workspace/pureboot-boards/qemu/build/x86/util-linux-2.29.2/.libs://lib]
                                                         ^----- here
...

100% agree for this change for libblkid 👍 We should be fine with no RPATH or RUNPATH here since everything will be under /lib.

I don't see any rpaths or runpaths in the libusb libs though (nor any other mention of my work tree location), so I think that change is not needed. My libusb build actually did match the one from the x230-hotp-maximized artifacts linked above, maybe libusb is OK as-is @tlaurion ? Do you have differing bins we can take a closer look at?

jhall@toaster:~/workspace/pureboot-boards/qemu$ sha256sum tools-extract{,-master-x230}/lib/libusb-1.0.so.0
8db48dfb4b077c3a56502e58bc39d5b607bc2c847979593e9388bce3dcd897ad  tools-extract/lib/libusb-1.0.so.0
8db48dfb4b077c3a56502e58bc39d5b607bc2c847979593e9388bce3dcd897ad  tools-extract-master-x230/lib/libusb-1.0.so.0
jhall@toaster:~/workspace/pureboot-boards/qemu$ sha256sum tools-extract{,-master-x230}/lib/libusb-0.1.so.4 
64613ad34d5cf8436b7284c4ea38e2ed1704bdf799695ea8bb7216c8061e11e6  tools-extract/lib/libusb-0.1.so.4
64613ad34d5cf8436b7284c4ea38e2ed1704bdf799695ea8bb7216c8061e11e6  tools-extract-master-x230/lib/libusb-0.1.so.4

The only other lib that I see a work tree path in is libjson-c, it's capturing source file paths. Maybe -fdebug-prefix-map did not apply correctly?

jhall@toaster:~/workspace/pureboot-boards/qemu/tools-extract$ strings lib/libjson-c.so.5 | grep pureboot-boards
/home/jhall/workspace/pureboot-boards/qemu/build/x86/json-c-0.14/json_object.c
/home/jhall/workspace/pureboot-boards/qemu/build/x86/json-c-0.14/json_tokener.c

@tlaurion
Copy link
Collaborator Author

tlaurion commented Mar 7, 2023

Superseded by #1329

@tlaurion tlaurion closed this Mar 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants