Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow using mender-artifact without cgo #605

Merged
merged 7 commits into from
May 6, 2024
Merged

Allow using mender-artifact without cgo #605

merged 7 commits into from
May 6, 2024

Conversation

alfrunes
Copy link
Contributor

@alfrunes alfrunes commented Apr 25, 2024 •
edited
Loading

  • Adds an alternative dependency for lzma with cgo disabled
  • Disables pkcs11 if built with nopkcs11 build tag.

@alfrunes alfrunes requested review from lluiscampos, a user and kjaskiewiczz April 25, 2024 18:23
@mender-test-bot
Copy link

@alfrunes, Let me know if you want to start the integration pipeline by mentioning me and the command "start pipeline".

my commands and options

You can trigger a pipeline on multiple prs with:

  • mentioning me and start pipeline --pr mender/127 --pr mender-connect/255

You can start a fast pipeline, disabling full integration tests with:

  • mentioning me and start pipeline --fast

You can trigger GitHub->GitLab branch sync with:

  • mentioning me and sync

You can cherry pick to a given branch or branches with:

  • mentioning me and:
 cherry-pick to:
 * 1.0.x
 * 2.0.x

@alfrunes alfrunes mentioned this pull request Apr 25, 2024
Closed
lluiscampos
lluiscampos reviewed Apr 26, 2024
View reviewed changes
Dockerfile Outdated Show resolved Hide resolved
ghost
ghost approved these changes Apr 26, 2024
View reviewed changes
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-approving, but see comments below.

artifact/compressor_lzma.go Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
@alfrunes alfrunes requested review from lluiscampos and a user April 26, 2024 13:04
@alfrunes
Copy link
Contributor Author

@mender-test-bot start pipeline

@mender-test-bot
Copy link

Hello 😺 I created a pipeline for you here: Pipeline-1269333209

Build Configuration Matrix

Key Value
AUDITLOGS_REV master
BUILD_BEAGLEBONEBLACK true
BUILD_CLIENT true
BUILD_QEMUX86_64_BIOS_GRUB true
BUILD_QEMUX86_64_BIOS_GRUB_GPT true
BUILD_QEMUX86_64_UEFI_GRUB true
BUILD_VEXPRESS_QEMU true
BUILD_VEXPRESS_QEMU_FLASH true
BUILD_VEXPRESS_QEMU_UBOOT_UEFI_GRUB true
CREATE_ARTIFACT_WORKER_REV master
DEPLOYMENTS_ENTERPRISE_REV master
DEPLOYMENTS_REV master
DEVICEAUTH_ENTERPRISE_REV master
DEVICEAUTH_REV master
DEVICECONFIG_REV master
DEVICECONNECT_REV master
DEVICEMONITOR_REV master
GENERATE_DELTA_WORKER_REV master
GUI_REV master
INTEGRATION_REV master
INVENTORY_ENTERPRISE_REV master
INVENTORY_REV master
IOT_MANAGER_REV master
MENDER_ARTIFACT_REV pull/605/head
MENDER_BINARY_DELTA_REV master
MENDER_CLI_REV master
MENDER_CONFIGURE_MODULE_REV master
MENDER_CONNECT_REV master
MENDER_CONVERT_REV master
MENDER_GATEWAY_REV master
MENDER_REV master
MENDER_SETUP_REV master
MENDER_SNAPSHOT_REV master
MONITOR_CLIENT_REV master
MTLS_AMBASSADOR_REV master
RUN_BACKEND_INTEGRATION_TESTS true
RUN_INTEGRATION_TESTS true
TENANTADM_REV master
TEST_QEMUX86_64_BIOS_GRUB true
TEST_QEMUX86_64_BIOS_GRUB_GPT true
TEST_QEMUX86_64_UEFI_GRUB true
TEST_VEXPRESS_QEMU true
TEST_VEXPRESS_QEMU_FLASH true
TEST_VEXPRESS_QEMU_UBOOT_UEFI_GRUB true
USERADM_ENTERPRISE_REV master
USERADM_REV master
WORKFLOWS_ENTERPRISE_REV master
WORKFLOWS_REV master

lluiscampos
lluiscampos approved these changes Apr 26, 2024
View reviewed changes
Copy link
Contributor

@lluiscampos lluiscampos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you 🏅

@alfrunes
Copy link
Contributor Author

@mender-test-bot start pipeline

@mender-test-bot
Copy link

Hello 😺 I created a pipeline for you here: Pipeline-1271644953

Build Configuration Matrix

Key Value
AUDITLOGS_REV master
BUILD_BEAGLEBONEBLACK true
BUILD_CLIENT true
BUILD_QEMUX86_64_BIOS_GRUB true
BUILD_QEMUX86_64_BIOS_GRUB_GPT true
BUILD_QEMUX86_64_UEFI_GRUB true
BUILD_VEXPRESS_QEMU true
BUILD_VEXPRESS_QEMU_FLASH true
BUILD_VEXPRESS_QEMU_UBOOT_UEFI_GRUB true
CREATE_ARTIFACT_WORKER_REV master
DEPLOYMENTS_ENTERPRISE_REV master
DEPLOYMENTS_REV master
DEVICEAUTH_ENTERPRISE_REV master
DEVICEAUTH_REV master
DEVICECONFIG_REV master
DEVICECONNECT_REV master
DEVICEMONITOR_REV master
GENERATE_DELTA_WORKER_REV master
GUI_REV master
INTEGRATION_REV master
INVENTORY_ENTERPRISE_REV master
INVENTORY_REV master
IOT_MANAGER_REV master
MENDER_ARTIFACT_REV pull/605/head
MENDER_BINARY_DELTA_REV master
MENDER_CLI_REV master
MENDER_CONFIGURE_MODULE_REV master
MENDER_CONNECT_REV master
MENDER_CONVERT_REV master
MENDER_GATEWAY_REV master
MENDER_REV master
MENDER_SETUP_REV master
MENDER_SNAPSHOT_REV master
MONITOR_CLIENT_REV master
MTLS_AMBASSADOR_REV master
RUN_BACKEND_INTEGRATION_TESTS true
RUN_INTEGRATION_TESTS true
TENANTADM_REV master
TEST_QEMUX86_64_BIOS_GRUB true
TEST_QEMUX86_64_BIOS_GRUB_GPT true
TEST_QEMUX86_64_UEFI_GRUB true
TEST_VEXPRESS_QEMU true
TEST_VEXPRESS_QEMU_FLASH true
TEST_VEXPRESS_QEMU_UBOOT_UEFI_GRUB true
USERADM_ENTERPRISE_REV master
USERADM_REV master
WORKFLOWS_ENTERPRISE_REV master
WORKFLOWS_REV master

lluiscampos
lluiscampos reviewed Apr 29, 2024
View reviewed changes
Makefile Outdated
fi; \
done
rm -f coverage-tmp.txt
go test -tags '$(TAGS)' -covermode=atomic -coverprofile=coverage.txt ./...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this is the reason for the coverage drop. I don't remember the details, but I looked in this years ago and ended up concluding that it is necessary to do package by package 🤷‍♂️

alfrunes added 5 commits May 6, 2024 09:17
@alfrunes
chore: Allow using mender-artifact without cgo enabled
db8d13c 
Because of the openssl PKCS11 dependency, it's not possible to compile
mender-artifact without `cgo` enabled.

Signed-off-by: Alf-Rune Siqveland <alf.rune@northern.tech>
@alfrunes
chore: Add Go alternative for lzma compression
7bb582f 
Adds pure Go lzma compression library if compiled without `cgo` enabled.

Signed-off-by: Alf-Rune Siqveland <alf.rune@northern.tech>
@alfrunes
refac(make): build-native-% targets print warning without pkcs11
de0b00e 
Signed-off-by: Alf-Rune Siqveland <alf.rune@northern.tech>
@alfrunes
chore: Remove CGO variant of lzma compression library
2c4b20a 
Signed-off-by: Alf-Rune Siqveland <alf.rune@northern.tech>
@alfrunes
chore(make): Refactor coverage target and disable pkcs11 on non-linux
22386d1 
Signed-off-by: Alf-Rune Siqveland <alf.rune@northern.tech>
@alfrunes
Copy link
Contributor Author

alfrunes commented May 6, 2024

@mender-test-bot start pipeline

@mender-test-bot
Copy link

Hello 😺 I created a pipeline for you here: Pipeline-1279567938

Build Configuration Matrix

Key Value
AUDITLOGS_REV master
BUILD_BEAGLEBONEBLACK true
BUILD_CLIENT true
BUILD_QEMUX86_64_BIOS_GRUB true
BUILD_QEMUX86_64_BIOS_GRUB_GPT true
BUILD_QEMUX86_64_UEFI_GRUB true
BUILD_VEXPRESS_QEMU true
BUILD_VEXPRESS_QEMU_FLASH true
BUILD_VEXPRESS_QEMU_UBOOT_UEFI_GRUB true
CREATE_ARTIFACT_WORKER_REV master
DEPLOYMENTS_ENTERPRISE_REV master
DEPLOYMENTS_REV master
DEVICEAUTH_ENTERPRISE_REV master
DEVICEAUTH_REV master
DEVICECONFIG_REV master
DEVICECONNECT_REV master
DEVICEMONITOR_REV master
GENERATE_DELTA_WORKER_REV master
GUI_REV master
INTEGRATION_REV master
INVENTORY_ENTERPRISE_REV master
INVENTORY_REV master
IOT_MANAGER_REV master
MENDER_ARTIFACT_REV pull/605/head
MENDER_BINARY_DELTA_REV master
MENDER_CLI_REV master
MENDER_CONFIGURE_MODULE_REV master
MENDER_CONNECT_REV master
MENDER_CONVERT_REV master
MENDER_GATEWAY_REV master
MENDER_REV master
MENDER_SETUP_REV master
MENDER_SNAPSHOT_REV master
MONITOR_CLIENT_REV master
MTLS_AMBASSADOR_REV master
RUN_BACKEND_INTEGRATION_TESTS true
RUN_INTEGRATION_TESTS true
TENANTADM_REV master
TEST_QEMUX86_64_BIOS_GRUB true
TEST_QEMUX86_64_BIOS_GRUB_GPT true
TEST_QEMUX86_64_UEFI_GRUB true
TEST_VEXPRESS_QEMU true
TEST_VEXPRESS_QEMU_FLASH true
TEST_VEXPRESS_QEMU_UBOOT_UEFI_GRUB true
USERADM_ENTERPRISE_REV master
USERADM_REV master
WORKFLOWS_ENTERPRISE_REV master
WORKFLOWS_REV master

alfrunes added 2 commits May 6, 2024 09:49
@alfrunes
ci: Add -coverpkg when generating coverage
ba669f0 
Adds coverage accross package boundaries.

Signed-off-by: Alf-Rune Siqveland <alf.rune@northern.tech>
@alfrunes
chore: Change lzma match finder algorithm to HashChain4
ccc3133 
The binary tree matcher is far too slow for github.com/ulikunitz/xz

Signed-off-by: Alf-Rune Siqveland <alf.rune@northern.tech>
@alfrunes
Copy link
Contributor Author

alfrunes commented May 6, 2024

@mender-test-bot start pipeline

@mender-test-bot
Copy link

Hello 😺 I created a pipeline for you here: Pipeline-1280060584

Build Configuration Matrix

Key Value
AUDITLOGS_REV master
BUILD_BEAGLEBONEBLACK true
BUILD_CLIENT true
BUILD_QEMUX86_64_BIOS_GRUB true
BUILD_QEMUX86_64_BIOS_GRUB_GPT true
BUILD_QEMUX86_64_UEFI_GRUB true
BUILD_VEXPRESS_QEMU true
BUILD_VEXPRESS_QEMU_FLASH true
BUILD_VEXPRESS_QEMU_UBOOT_UEFI_GRUB true
CREATE_ARTIFACT_WORKER_REV master
DEPLOYMENTS_ENTERPRISE_REV master
DEPLOYMENTS_REV master
DEVICEAUTH_ENTERPRISE_REV master
DEVICEAUTH_REV master
DEVICECONFIG_REV master
DEVICECONNECT_REV master
DEVICEMONITOR_REV master
GENERATE_DELTA_WORKER_REV master
GUI_REV master
INTEGRATION_REV master
INVENTORY_ENTERPRISE_REV master
INVENTORY_REV master
IOT_MANAGER_REV master
MENDER_ARTIFACT_REV pull/605/head
MENDER_BINARY_DELTA_REV master
MENDER_CLI_REV master
MENDER_CONFIGURE_MODULE_REV master
MENDER_CONNECT_REV master
MENDER_CONVERT_REV master
MENDER_GATEWAY_REV master
MENDER_REV master
MENDER_SETUP_REV master
MENDER_SNAPSHOT_REV master
MONITOR_CLIENT_REV master
MTLS_AMBASSADOR_REV master
RUN_BACKEND_INTEGRATION_TESTS true
RUN_INTEGRATION_TESTS true
TENANTADM_REV master
TEST_QEMUX86_64_BIOS_GRUB true
TEST_QEMUX86_64_BIOS_GRUB_GPT true
TEST_QEMUX86_64_UEFI_GRUB true
TEST_VEXPRESS_QEMU true
TEST_VEXPRESS_QEMU_FLASH true
TEST_VEXPRESS_QEMU_UBOOT_UEFI_GRUB true
USERADM_ENTERPRISE_REV master
USERADM_REV master
WORKFLOWS_ENTERPRISE_REV master
WORKFLOWS_REV master

@alfrunes
Copy link
Contributor Author

alfrunes commented May 6, 2024

Turns out that the BinaryTree matcher for the new lzma implementation (github.com/ulikunitz/xz) seems to be hanging when compressing objects. I changed the default config to use HashTable4 (hc4 in xz(1)) instead.

@alfrunes alfrunes merged commit 8b1dd0d into mendersoftware:master May 6, 2024
21 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lluiscampos lluiscampos lluiscampos approved these changes

@kjaskiewiczz kjaskiewiczz Awaiting requested review from kjaskiewiczz

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@alfrunes @mender-test-bot @lluiscampos