new: arm64 support #143

FedeDP · 2022-05-03T07:41:22Z

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area cmd

/area pkg

/area docs

/area tests

What this PR does / why we need it:

Builds and pushes driverkit builder images for arm64 too
Adds support for arm64 in various builders
Adds support for cross-building on docker processor, using multiarch/qemu-user-static. By default, current running architecture will be built; but one has the option to force a different arch with the architecture option.
Added option to pass kernelurls directly from the config file/cli argument. kernel-crawler will dump driverkit-like configs with the kernel urls already provided, as it fetched them.
Bumped all the deps versions

Still needed:

~~Double check the very first commit; i am not sure if CircleCI will work. If anyone with a strong knowledge could double check it, it would really help me!~~ Fixed with 6f9deb0

Which issue(s) this PR fixes:

Fixes #142

supersedes #119

Special notes for your reviewer:

NOTE: the diff is big but most of the changes are from go.sum.

Does this PR introduce a user-facing change?:

new: added support for arm64 builders

FedeDP · 2022-05-03T07:41:50Z

/cc @zuc @maxgio92

FedeDP · 2022-05-03T08:15:05Z

I am not getting why tests are failing. On my setup they pass with no issues.
Log does not tell much: https://app.circleci.com/pipelines/github/falcosecurity/driverkit/232/workflows/a32bfd26-2481-4de6-b03d-40ea06fe162e/jobs/314

FedeDP · 2022-05-03T12:28:24Z

Builders:

AmazonLinux -> after applying fix: reversed symbolic link in amazonlinux2 builder script #141 -> build gets interrupted while building the kernel module:

DEBU Q+ make KERNELDIR=/tmp/kernel CC=/usr/bin/gcc-8 LD=/usr/bin/ld.bfd CROSS_COMPILE=
DEBU *make -C /tmp/kernel M=/tmp/driver modules
DEBU *make[1]: Entering directory '/tmp/kernel'
DEBU   CC [M]  /tmp/driver/main.o
DEBU -  CC [M]  /tmp/driver/dynamic_params_table.o
DEBU &  CC [M]  /tmp/driver/fillers_table.o
DEBU $  CC [M]  /tmp/driver/flags_table.o
DEBU #  CC [M]  /tmp/driver/ppm_events.o
DEBU $  CC [M]  /tmp/driver/ppm_fillers.o
DEBU log pipe close                                error=EOF
DEBU context canceled

It was a timeout issue. Increasing the timeout to 120 fixed the issue. I am going to increase the default timeout to 120.

Ubuntu/Debian -> seems related to gcc version:

make KERNELDIR=/tmp/kernel-download/usr/src/linux-headers-5.11.0-49-generic 
make -C /tmp/kernel-download/usr/src/linux-headers-5.11.0-49-generic M=/tmp/driver modules 
make[1]: Entering directory '/tmp/kernel-download/usr/src/linux-headers-5.11.0-49-generic'
CC [M]  /tmp/driver/main.o   
gcc: error: unrecognized command line option '-mbranch-protection=pac-ret+leaf+bti' 
gcc: error: unrecognized command line option '-mstack-protector-guard=sysreg'; did you mean '-fstack-protector-strong'? 
gcc: error: unrecognized command line option '-mstack-protector-guard-reg=sp_el0'; did you mean '-fstack-protector-all'? 
gcc: error: unrecognized command line option '-mstack-protector-guard-offset=1344'; did you mean '-fstack-protector-strong'? 
make[2]: *** [scripts/Makefile.build:288: /tmp/driver/main.o] Error 1 
make[1]: *** [Makefile:1849: /tmp/driver] Error 2

The issue is only present on newest kernel (like 5.10+).
Most probably, we are having issues with our old gcc-8 version. I think #113 could be a very good option to expand our support matrix.
If that get merged, i will push a new builder image with newer debian and gcc 10+ (of course after testing that it is effective in building newest kernels for arm64, and possibly amd64).

For reference, i was able to build debian kernel 5.10.103-1-amd64 , but not 5.10.103-1-arm64. I was instead able to build 4.19.232-1-arm64.

FedeDP · 2022-05-05T14:09:07Z

NOTE to maintainers: should we document (at least in README) supported gcc/clang versions?
Btw possibly the requested gcc version could be used to select the right builderimage (again, given #113 is merged).

leodido · 2022-05-05T14:11:08Z

NOTE to maintainers: should we document (at least in README) supported gcc/clang versions?

Yes, please.

Btw possibly the requested gcc version could be used to select the right builderimage (again, given #113 is merged).

I would not tie the two things so tight right now...

FedeDP · 2022-05-05T14:12:01Z

I will document them :) Thanks!

FedeDP · 2022-05-06T13:38:24Z

.circleci/config.yml

    steps:
      - checkout
      - run:
          name: Test
          command: make test
  "images":
    docker:
-      - image: docker:stable
+      - image: cimg/base:stable


Switched to cimg/base:stable as it already provides docker buildx command for us, when a 20.10.x version of docker is requested.

FedeDP · 2022-05-06T13:38:41Z

.circleci/config.yml

@@ -2,30 +2,29 @@ version: 2
 jobs:
  "test":
    docker:
-      - image: docker.io/golang:1.14.0
+      - image: docker.io/golang:1.17.0


Golang1.16 had an issue that prevented tests to pass.

FedeDP · 2022-05-09T09:33:27Z

I would not tie the two things so tight right now...

Mmmh i am not sure: i think we need this; it is the easiest way: the distro builder specifies one of the supported gcc/clang, and based on that, the build processor will choose the correct builder image.
It seems a pretty simple and straightforward logic and would allow us to build a larger number of drivers.

I think i will open a PR once #113 is merged, to achieve that. We can then discuss on it :)

Moreover, upload release binary package for arm64. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

…rting order. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

…rmations. Added readme about supported gcc and llvm versions and architectures. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

… the docker job. This allows us to use docker buildx already provided by circle ci image. See https://support.circleci.com/hc/en-us/articles/360058095471-How-To-Use-Docker-Buildx-in-Remote-Docker- for more info. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

Signed-off-by: Federico Di Pierro <nierro92@gmail.com> Co-authored-by: David Windsor <dwindsor@secureworks.com>

FedeDP · 2022-05-11T12:47:31Z

Uh, rebased on top of #113 🥇

FedeDP · 2022-05-25T09:43:13Z

/cc @dwindsor 🥳

dwindsor · 2022-05-25T16:48:32Z

pkg/driverbuilder/builder/vanilla.go

+	kv := kernelReleaseFromBuildConfig(c.Build)
+
+	var urls []string
+	if c.KernelUrls == nil {


Hmm, so if a user wants to specify e.g. one additional server in which to look for kernel-devel packages, IIUC he'd have to also enumerate all of the other URLs aside from the one he wants to add? I'm thinking the common use case here will be a user who wants to search for kernel packages from the usual locations plus additional locations? So maybe have --kernelurls be append by default?

For me, kernelurls is to pass the exact url for downloading a kernel-devel package (in case of Ubuntu, we need 2 urls, the arch one and the _all one).

Therefore there is no need to append it.

Idea is that it is much simpler for an user to search for the exact url for its kernel devel, than to possibly adjust kernels scraping code inside driverkit, thus even if driverkit does not support discovering your url, you can still build your driver.
Moreover, this way we are able to support:

various ubuntu flavors, like ubuntu-kvm, ubuntu-gcp and so on

ubuntu/debian kernelversions like 32~16.04

Note that this is part of a broader scope together with https://github.com/falcosecurity/kernel-crawler, that outputs a driverkit-like json for any found kernel with the urls to download its devel packages; see: https://github.com/falcosecurity/kernel-crawler/blob/main/kernels/x86_64/list.json

What i am trying to say is that in the future users can try their luck by providing exact kernel url parameter whenever a "kernel header not found" error happens; and in case, fixing their configs to provide the kernelurls.

In the end, the current kernel url auto-formatting method will remain as a fallback for when no kernelurls param is passed.

That's of course IMHO :)

Idea is that it is much simpler for an user to search for the exact url for its kernel devel, than to possibly adjust kernels scraping code inside driverkit, thus even if driverkit does not support discovering your url, you can still build your driver.

👆 Definitely. This is a powerful addition!

The case I'm thinking of is, deploying Falco to a fleet of machines that has many instances of "kernel headers not found" because they have many e.g. ubuntu-gcp machines, but with different kernel versions. So, we know the kernel headers for ubuntu-gcp can all be found at e.g. pub.ubuntu.com/blah/gcp/packages/kernel-devel-1.2.3-4.deb. Could we have kernelurls accept the directory to pub.ubuntu.com/blah/gcp/packages and have it crawl that directory via the usual means, auto-extracting the URLs to kernel-devel packages it finds there?

I see; perhaps that would be useful to also use a privare deb repository (for example).

I think it is a great idea; we might think of something like:

if kernelurls contains some formatting options, like %M.%m.%p -> we format it with proper variables and use it

if kernelurls does not contain any formatting option, it means that it is a fullpath and we can use it without further effort

This is a nice solution, even if i'll have to write some formatting specs (like, in the above example: "%M" -> major, "%m" -> minor and "%p" -> patch).

What do you think?
Btw i can also try to split this feature from this PR if you wish; it helped me test various arm64 builds but it is not strictly related to it.

if kernelurls contains some formatting options, like %M.%m.%p -> we format it with proper variables and use it

Ahh, I think I see what you mean. So, for e.g. in the previous case mentioned with a fleet of ubuntu-gcp machines, one would do something like:

driverkit --kernelurls pub.ubuntu.com/blah/ubuntu-gcp/packages/kernel-devel-%M.%m-%p.deb

If so, this is a great idea! Allowing the user to specify the package URL format at runtime protects us against having to constantly update the builders if/when a vendor changes the formatting of their packages.

Doesn't necessarily need to be separate from this PR - I think it makes sense to include it here. We'll get this merged and then cut a new release. 🚀

Actually, thinking about this more: this will be a significant amount of unrelated work to add to this PR. Support for formatting specs will have to be added to each builder, so it does make sense to put it in another PR. No need to bother splitting it off into another branch if you've already started! 😬

dwindsor · 2022-05-31T13:37:49Z

/lgtm

poiana · 2022-05-31T13:37:51Z

@dwindsor: changing LGTM is restricted to collaborators

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

FedeDP · 2022-05-31T13:38:49Z

/hold

I want to test the shiny new qemu 7.0.0 for the x86_64 -> arm64 cross build :)

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

Moreover, it seems `go` installation is mandatory, otherwise buildx build failed at the link stage with: ``` /usr/local/go/pkg/tool/linux_arm64/link: running gcc failed: exit status 1 collect2: error: ld returned 1 exit status ``` Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

FedeDP · 2022-06-01T11:55:14Z

Tested and fixed qemu integration.
Moreover, double tested and fixed docker buildx command to build driverkit and builder images.

As agreed with @dwindsor, I will remove the kernelurls from this PR because i think it is better to push it in its own one.
Bear with me if i am not able to rebase this PR removing the kernelurls and instead add a commit on top to remove it, but i think that it would require manually fixing an subsequent commit tests.
🚀

I will unhold the PR once kernelurls is removed from this :)

…part of another PR. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

FedeDP · 2022-06-03T09:07:03Z

/unhold

Removed kernelurls option :)

dwindsor · 2022-06-03T11:38:42Z

/lgtm

poiana · 2022-06-03T11:38:46Z

LGTM label has been added.

Git tree hash: 49bb73c0e9fae80a82b00376ebfafad8283b45cf

poiana · 2022-06-03T11:52:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dwindsor, FedeDP

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [dwindsor]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

poiana added kind/cleanup do-not-merge/work-in-progress dco-signoff: yes kind/feature New feature or request area/build Further information is requested area/cmd area/pkg labels May 3, 2022

poiana requested review from fntlnz and leodido May 3, 2022 07:41

poiana added the size/XXL label May 3, 2022

poiana requested review from maxgio92 and zuc May 3, 2022 07:41

FedeDP mentioned this pull request May 3, 2022

Driverkit Debian/Ubuntu builder wrong kernel version #142

Closed

FedeDP mentioned this pull request May 5, 2022

new(driverkit): allow configurable builder images #113

Merged

FedeDP force-pushed the new/arm64_support branch from 752bf4c to 6e7b9ed Compare May 5, 2022 14:32

FedeDP changed the title ~~wip: new: arm64 support~~ new: arm64 support May 6, 2022

poiana removed the do-not-merge/work-in-progress label May 6, 2022

FedeDP commented May 6, 2022

View reviewed changes

FedeDP mentioned this pull request May 9, 2022

Cut 0.32.0 Release falcosecurity/falco#1993

Closed

53 tasks

This was referenced May 10, 2022

Driverkit fails to build Ubuntu releases #126

Closed

Properly support more ubuntu flavours #144

Closed

new: build and push driverkit docker images for arm64 too.

ee75bd9

Moreover, upload release binary package for arm64. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

FedeDP and others added 7 commits May 11, 2022 14:46

fix: fixed tests.

26d07a1

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

update(docs): updated docs text.

0a67cc5

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

fix(build): fixed builder image on x86_64 maintaining dependencies so…

d567439

…rting order. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

chore: dropped leftover debug print and updated README with some info…

60e8932

…rmations. Added readme about supported gcc and llvm versions and architectures. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

fix(driverbuilder): fixed al driverbuilder urls length check.

71daf51

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

fix(build): add zstd to builder image deps.

9fd9a99

Signed-off-by: Federico Di Pierro <nierro92@gmail.com> Co-authored-by: David Windsor <dwindsor@secureworks.com>

FedeDP force-pushed the new/arm64_support branch from e5ecd74 to 9fd9a99 Compare May 11, 2022 12:46

dwindsor reviewed May 25, 2022

View reviewed changes

poiana added the do-not-merge/hold label May 31, 2022

FedeDP added 2 commits June 1, 2022 10:44

update(pkg): fixed qemu for x86_64 -> aarch64 cross build.

a0d2e46

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

update(cmd,docs,pkg): removed kernelurls option because it will be …

ecf7ab5

…part of another PR. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

poiana removed the do-not-merge/hold label Jun 3, 2022

poiana assigned dwindsor Jun 3, 2022

poiana added the lgtm label Jun 3, 2022

poiana added the approved label Jun 3, 2022

dwindsor approved these changes Jun 3, 2022

View reviewed changes

poiana merged commit c3f4504 into falcosecurity:master Jun 3, 2022

FedeDP deleted the new/arm64_support branch June 3, 2022 11:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new: arm64 support #143

new: arm64 support #143

FedeDP commented May 3, 2022 •

edited

Loading

FedeDP commented May 3, 2022

FedeDP commented May 3, 2022

FedeDP commented May 3, 2022 •

edited

Loading

FedeDP commented May 5, 2022

leodido commented May 5, 2022 •

edited

Loading

FedeDP commented May 5, 2022

FedeDP May 6, 2022

FedeDP May 6, 2022

FedeDP commented May 9, 2022

FedeDP commented May 11, 2022

FedeDP commented May 25, 2022

dwindsor May 25, 2022 •

edited

Loading

FedeDP May 26, 2022

FedeDP May 26, 2022

dwindsor May 27, 2022

FedeDP May 30, 2022

dwindsor May 30, 2022

dwindsor May 31, 2022

dwindsor commented May 31, 2022

poiana commented May 31, 2022

FedeDP commented May 31, 2022 •

edited

Loading

FedeDP commented Jun 1, 2022

FedeDP commented Jun 3, 2022

dwindsor commented Jun 3, 2022

poiana commented Jun 3, 2022

poiana commented Jun 3, 2022

new: arm64 support #143

new: arm64 support #143

Conversation

FedeDP commented May 3, 2022 • edited Loading

FedeDP commented May 3, 2022

FedeDP commented May 3, 2022

FedeDP commented May 3, 2022 • edited Loading

FedeDP commented May 5, 2022

leodido commented May 5, 2022 • edited Loading

FedeDP commented May 5, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

FedeDP commented May 9, 2022

FedeDP commented May 11, 2022

FedeDP commented May 25, 2022

dwindsor May 25, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dwindsor commented May 31, 2022

poiana commented May 31, 2022

FedeDP commented May 31, 2022 • edited Loading

FedeDP commented Jun 1, 2022

FedeDP commented Jun 3, 2022

dwindsor commented Jun 3, 2022

poiana commented Jun 3, 2022

poiana commented Jun 3, 2022

FedeDP commented May 3, 2022 •

edited

Loading

FedeDP commented May 3, 2022 •

edited

Loading

leodido commented May 5, 2022 •

edited

Loading

dwindsor May 25, 2022 •

edited

Loading

FedeDP commented May 31, 2022 •

edited

Loading