tetragon: Add killer sensor #1205

olsajiri · 2023-07-10T15:40:42Z

adding support to have fast way (when multi kprobe link is not available)
of attaching killer/override program to multiple syscalls

details in: https://github.com/olsajiri/tetragon/blob/killer/docs/content/en/docs/concepts/tracing-policy/selectors.md#notify-killer-action

netlify · 2023-07-10T15:41:07Z

✅ Deploy Preview for tetragon ready!

Name	Link
🔨 Latest commit	`2d80318`
🔍 Latest deploy log	https://app.netlify.com/sites/tetragon/deploys/6527e4f135be40000834b7d2
😎 Deploy Preview	https://deploy-preview-1205--tetragon.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

kkourt

🚀

Have a couple of nits, but this looks good! Thanks!

kkourt · 2023-10-09T15:30:19Z

pkg/k8s/apis/cilium.io/client/crds/v1alpha1/cilium.io_tracingpoliciesnamespaced.yaml

@@ -576,6 +576,9 @@ spec:
                      - generated_syscalls
                      - generated_ftrace
                      type: string
+                    validated:
+                      description: List was validated
+                      type: boolean


So from what I can tell, this option means that no validation will be performed.
Maybe ignore-validation is a better name?

it's there to ensure the validation goes through just once, so it gets validated,
sets the flag so another sensor won't run it again, I'll update changelog with that, sry

is there a way we could avoid implementation things to be visible/editable from the UI (the yaml)?

this is typically that could go into the status of the resource instead of the spec.

kkourt · 2023-10-09T15:35:40Z

pkg/k8s/apis/cilium.io/v1alpha1/types.go

@@ -315,3 +315,8 @@ type PodInfoList struct {
 	metav1.ListMeta `json:"metadata,omitempty"`
 	Items           []PodInfo `json:"items"`
 }
+
+type KillerSpec struct {
+	// syscalls to kill


How about description: syscalls where killer is executed in?

AFAIU, these are the syscalls where we will load the killer and do the check on whether to "kill" the process or not.

kkourt · 2023-10-09T15:45:04Z

api/v1/tetragon/tetragon.proto

@@ -433,6 +433,8 @@ message ProcessTracepoint {
    // Arguments definition of the observed tracepoint.
    // TODO: once we implement all we want, rename KprobeArgument to GenericArgument
    repeated KprobeArgument args = 6;
+    // Action performed when the kprobe matched.


s/kprobe/tracepoint/?

kkourt · 2023-10-09T15:48:46Z

docs/content/en/docs/concepts/tracing-policy/selectors.md

@@ -897,6 +898,95 @@ broken.

 Socket tracking is only available on kernel >=5.3.

+### Notify Killer action
+
+The `NotifyKiller` action notifies the killer program to kill or override syscall.


Suggested change

The `NotifyKiller` action notifies the killer program to kill or override syscall.

The `NotifyKiller` action notifies the killer program to kill or override a syscall.

kkourt · 2023-10-09T15:49:09Z

docs/content/en/docs/concepts/tracing-policy/selectors.md

+The `NotifyKiller` action notifies the killer program to kill or override syscall.
+
+The specs needs to have `killer` program definition, that instructs tetragon to load
+`killer` program and attach it to specified syscalls.


Suggested change

`killer` program and attach it to specified syscalls.

the `killer` program and attach it to specified syscalls.

mtardy

Thanks, only docs nits and agree with Kornilios points above!! The way your commits are done makes this really easy to review, would love to learn to do the same.

mtardy · 2023-10-11T07:59:01Z

docs/content/en/docs/concepts/tracing-policy/selectors.md

+If pecified the argError will be passed to `bpf_override_return` helper to override the syscall return value.
+If pecified the argSig will be passed to `bpf_send_signal` helper to override the syscall return value.


Suggested change

If pecified the argError will be passed to `bpf_override_return` helper to override the syscall return value.

If pecified the argSig will be passed to `bpf_send_signal` helper to override the syscall return value.

If specified the argError will be passed to `bpf_override_return` helper to override the syscall return value.

If specified the argSig will be passed to `bpf_send_signal` helper to override the syscall return value.

mtardy · 2023-10-11T08:00:37Z

docs/content/en/docs/concepts/tracing-policy/selectors.md

+Note the `NotifyKiller` with killer program is meant to be used only on kernel versions
+with no support for fast attach of multiple kprobes (`kprobe_multi` link).


Would it be possible to move that to the very beginning, after the "The NotifyKiller action notifies the killer program to kill or override syscall." sentence? Also, could we have a little more details on the why of this feature? Why this is interesting and maybe useful for the reader?

ok, I'll add some description at the beginning, thanks

Adding Validated field to ListSpec to ensure the validation goes through just once, so it gets validated, sets the flag so another sensor won't run it again. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Adding preValidateLists function that iterates all the lists and validates them. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Passing symbol argument to kprobeAttach so we can use it in followich changes with different symbols. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Adding LoadKprobeProgramAttachMany function that loads kprobe program and attaches it to symbols provided in array of strings. Note the program gets loaded just once. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Add possibility to define killer in the schema like: killers: - syscalls: - "sys_dup" "sys.." Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Adding killer sensor that attaches to defined syscalls and overrides them or/and kill the process. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Adding bpf_killer bpf program that allows to (when attached to syscall) override syscall or kill current process. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Adding support to notify killer program attached to syscalls with another action spec, like: matchActions: - action: "NotifyKiller" argError: -1 argSig: 9 It's possible to specify error for override and signal number to kill the current process with. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

So we can use it through kubectl. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Adding example killer policy that setups sys_enter raw syscalls and add filter for sys_dup and sys_dup2 syscalls plus match for /usr/bin/bash binary and kill it through killer. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Adding action field to tracepoint event, so it's available in the event in case the action is triggered. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Catching up with kprobe/tracepoint event action enum definitions. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Adding test for killer sensor and testing both killing application matching the selector and overriding syscall that matches the selector. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

Signed-off-by: Jiri Olsa <jolsa@kernel.org>

olsajiri force-pushed the killer branch 3 times, most recently from c13973c to 21daede Compare July 11, 2023 13:31

olsajiri force-pushed the killer branch 5 times, most recently from b27c59f to 69718d7 Compare August 22, 2023 13:37

olsajiri force-pushed the killer branch 2 times, most recently from 967b56c to 8bf3a4e Compare August 30, 2023 09:41

olsajiri added the release-note/minor This PR introduces a minor user-visible change label Aug 30, 2023

olsajiri force-pushed the killer branch 7 times, most recently from 7b67b6d to 2725b32 Compare September 5, 2023 11:13

olsajiri force-pushed the killer branch 4 times, most recently from 1c0e273 to edc5374 Compare September 20, 2023 16:41

olsajiri force-pushed the killer branch 7 times, most recently from ec97b4c to bf75c17 Compare October 6, 2023 13:31

olsajiri changed the title ~~Killer~~ tetragon: Add killer sensor Oct 6, 2023

olsajiri force-pushed the killer branch 2 times, most recently from dd6bd1c to 82a64dc Compare October 7, 2023 15:12

olsajiri marked this pull request as ready for review October 9, 2023 08:52

olsajiri requested review from a team and mtardy as code owners October 9, 2023 08:52

olsajiri requested a review from kkourt October 9, 2023 08:53

kkourt approved these changes Oct 9, 2023

View reviewed changes

mtardy approved these changes Oct 11, 2023

View reviewed changes

olsajiri added 14 commits October 12, 2023 10:13

tetragon: Add Validated field to ListSpec

eda3ea1

Adding Validated field to ListSpec to ensure the validation goes through just once, so it gets validated, sets the flag so another sensor won't run it again. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

tetragon: Add preValidateLists function

29c37f2

Adding preValidateLists function that iterates all the lists and validates them. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

tetragon: Pass symbol argument to kprobeAttach

c785367

Passing symbol argument to kprobeAttach so we can use it in followich changes with different symbols. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

tetragon: Add LoadKprobeProgramAttachMany function

2245bce

Adding LoadKprobeProgramAttachMany function that loads kprobe program and attaches it to symbols provided in array of strings. Note the program gets loaded just once. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

tetragon: Add support for killer schema

5d9cc46

Add possibility to define killer in the schema like: killers: - syscalls: - "sys_dup" "sys.." Signed-off-by: Jiri Olsa <jolsa@kernel.org>

tetragon: Add killer sensor

eb7b9b0

Adding killer sensor that attaches to defined syscalls and overrides them or/and kill the process. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

tetragon: Add bpf_killer bpf program

af2ddf2

Adding bpf_killer bpf program that allows to (when attached to syscall) override syscall or kill current process. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

tetragon: Add NotifyKiller to Action kubebuilder:validation enum

a62464d

So we can use it through kubectl. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

tetragon: Add example killer policy

da8e135

Adding example killer policy that setups sys_enter raw syscalls and add filter for sys_dup and sys_dup2 syscalls plus match for /usr/bin/bash binary and kill it through killer. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

tetragon: Add action field to tracepoint event

8181f8e

Adding action field to tracepoint event, so it's available in the event in case the action is triggered. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

tetragon: Update kprobe/tracepoint event action definitions

d2965c5

Catching up with kprobe/tracepoint event action enum definitions. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

tetragon: Add killer tests

9d1e2ed

Adding test for killer sensor and testing both killing application matching the selector and overriding syscall that matches the selector. Signed-off-by: Jiri Olsa <jolsa@kernel.org>

tetragon: Add doc for killer sensor

2d80318

Signed-off-by: Jiri Olsa <jolsa@kernel.org>

olsajiri force-pushed the killer branch from 82a64dc to 2d80318 Compare October 12, 2023 12:22

olsajiri merged commit c4cf61b into cilium:main Oct 12, 2023
33 checks passed

kkourt added release-note/major This PR introduces major new functionality and removed release-note/minor This PR introduces a minor user-visible change labels Nov 1, 2023

p-linnane mentioned this pull request Nov 1, 2023

tetra 1.0.0 Homebrew/homebrew-core#153073

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tetragon: Add killer sensor #1205

tetragon: Add killer sensor #1205

olsajiri commented Jul 10, 2023 •

edited

Loading

netlify bot commented Jul 10, 2023 •

edited

Loading

kkourt left a comment

kkourt Oct 9, 2023 •

edited

Loading

olsajiri Oct 12, 2023

mtardy Oct 12, 2023

kkourt Oct 9, 2023

kkourt Oct 9, 2023

kkourt Oct 9, 2023

kkourt Oct 9, 2023

mtardy left a comment

mtardy Oct 11, 2023

mtardy Oct 11, 2023

olsajiri Oct 12, 2023

	The `NotifyKiller` action notifies the killer program to kill or override syscall.
	The `NotifyKiller` action notifies the killer program to kill or override a syscall.

	`killer` program and attach it to specified syscalls.
	the `killer` program and attach it to specified syscalls.

		If pecified the argError will be passed to `bpf_override_return` helper to override the syscall return value.
		If pecified the argSig will be passed to `bpf_send_signal` helper to override the syscall return value.

		Note the `NotifyKiller` with killer program is meant to be used only on kernel versions
		with no support for fast attach of multiple kprobes (`kprobe_multi` link).

tetragon: Add killer sensor #1205

tetragon: Add killer sensor #1205

Conversation

olsajiri commented Jul 10, 2023 • edited Loading

netlify bot commented Jul 10, 2023 • edited Loading

✅ Deploy Preview for tetragon ready!

kkourt left a comment

Choose a reason for hiding this comment

kkourt Oct 9, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtardy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

olsajiri commented Jul 10, 2023 •

edited

Loading

netlify bot commented Jul 10, 2023 •

edited

Loading

kkourt Oct 9, 2023 •

edited

Loading