0.51: kubernetes scanner requires role permissions

[jkroepke]

Description

I'm running trivy with an limited scope of permissions

Before 0.51, i'm runing trivy with following settings:

      trivy --quiet -q kubernetes --report=all \
        --components workload \
        --disable-node-collector \
        --cache-dir /tmp/.trivycache/ \
        --no-progress \
        --ignore-unfixed \
        --exit-code 0 \
        --slow \
        --format table \
        --scanners vuln \
        --ignorefile /.trivyignore \
        --vuln-type os \
        -n opsstack \
        -o "/scans/report.all.html" \
        deploy,sts,ds;

with works fine. After upgrade, I'm using the following command

      trivy kubernetes --report=all \
        --disable-node-collector \
        --cache-dir /tmp/.trivycache/ \
        --no-progress \
        --ignore-unfixed \
        --exit-code 0 \
        --slow \
        --format table \
        --scanners vuln \
        --ignorefile /.trivyignore \
        --vuln-type os \
        --include-namespaces opsstack \
        -o "/scans/report.all.html" \
        --include-kinds deploy,sts,ds;

I'm getting an error that trivy has no access to roles. However, trivy should not ask for roles.

Desired Behavior

Except that trivy is scanning the workload of deployments, statefulsets and daemonsets

Actual Behavior

2024-05-07T15:49:56Z FATAL Fatal error get k8s artifacts error: failed listing resources for gvr: rbac.authorization.k8s.io/v1, Resource=roles - roles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:opsstack:opsstack-trivy" cannot list resource "roles" in API group "rbac.authorization.k8s.io" at the cluster scope

Reproduction Steps

1. Setup trivy using a context with namespace access only
2. Run the command above
3.
...

Target

Kubernetes

Scanner

Vulnerability

Output Format

Table

Mode

Standalone

Debug Output

2024-05-07T15:51:34Z    DEBUG   Parsed severities       severities=[UNKNOWN LOW MEDIUM HIGH CRITICAL]
2024-05-07T15:51:34Z    DEBUG   Ignore statuses statuses=[0 1 2 4 5 6 7]
2024-05-07T15:51:40Z    FATAL   Fatal error
  - get k8s artifacts error:
    github.com/aquasecurity/trivy/pkg/k8s/commands.clusterRun
        /home/runner/work/trivy/trivy/pkg/k8s/commands/cluster.go:49
  - failed listing resources for gvr: rbac.authorization.k8s.io/v1, Resource=roles - roles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:opsstack:opsstack-trivy" cannot list resource "roles" in API group "rbac.authorization.k8s.io" at the cluster scope

Operating System

Linux

Version

Version: 0.51.1

Checklist

• Run trivy image --reset
• Read the troubleshooting

[chen-keinan]

@jkroepke this functionality is not new, trivy list roles/rolebinding and clusterroles/clusterrolebinding in order to assess RBAC resources.
I can suggest two things:

• use --exclude-namespaces or include-namespaces flags to target specific RBAC namespaces
• use --exclude-kinds clusterroles,roles to exclude these resources from scanning

[jkroepke]

@chen-keinan if you mention, that is not new - why it happens since 0.51?

use --exclude-kinds clusterroles,roles to exclude these resources from scanning

Running the command

      trivy kubernetes --report=all \
        --disable-node-collector \
        --cache-dir /tmp/.trivycache/ \
        --no-progress \
        --ignore-unfixed \
        --exit-code 0 \
        --slow \
        --format table \
        --scanners vuln \
        --ignorefile /.trivyignore \
        --vuln-type os \
        --include-namespaces opsstack \
        -o "/scans/report.all.html" \
        --exclude-kinds clusterroles,roles

2024-05-09T13:05:35Z FATAL Fatal error get k8s artifacts error: failed listing resources for gvr: rbac.authorization.k8s.io/v1, Resource=roles - roles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:opsstack:opsstack-trivy" cannot list resource "roles" in API group "rbac.authorization.k8s.io" at the cluster scope: Azure does not have opinion for this user.

trivy list roles/rolebinding and clusterroles/clusterrolebinding in order to assess RBAC resources.

How to avoid that? How I can restore the --components workload behavior from 0.50?

[chen-keinan]

@jkroepke use the following flags--disable-node-collector and --exclude-namespaces kube-system in conjunction also you can add --exclude-kinds serviceaccount,roles if needed

[jkroepke]

The flag --disable-node-collector is already included.

The flag --include-namespaces opsstack should already cover --exclude-namespaces kube-system?

Same question for the initial suggestion: I'm original using --include-kinds deploy,sts,ds what should exclude everything, like role, sa, clusterrole and more.

Here is the command which includes your suggestions:

 trivy kubernetes --report=all \
        --disable-node-collector \
        --cache-dir /tmp/.trivycache/ \
        --no-progress \
        --ignore-unfixed \
        --exit-code 0 \
        --slow \
        --format table \
        --scanners vuln \
        --ignorefile /.trivyignore \
        --vuln-type os \
        --exclude-namespaces kube-system \
        --exclude-kinds clusterroles,serviceaccount,roles

and results into the same error again:

2024-05-09T13:49:47Z    FATAL   Fatal error     get k8s artifacts error: failed listing resources for gvr: rbac.authorization.k8s.io/v1, Resource=roles - roles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:opsstack:opsstack-trivy" cannot list resource "roles" in API group "rbac.authorization.k8s.io" at the cluster scope: Azure does not have opinion for this user.

[jkroepke]

@chen-keinan seems like there is an bug in trivy.

[chen-keinan]

@jkroepke thanks for the input converting to issue and I'll will create a fix for it.

[chen-keinan]

issue opened #6692

[chen-keinan]

@ill have a look and update you

[chen-keinan]

@jkroepke the reason we are listing nodes is for looking vulnerabilities inside node components: kubelet, containerd, kube-proxy and etc.
I'm creating a fix on this to make sure it will not fail on permission for it.

[jkroepke]

Hey @chen-keinan

what about an flag that only validate the workload?

Like we had in 0.50 and below

Can we reopen this discussion until the issue is resolved?

[chen-keinan]

I have added a fix below which should sort out functionality, no need other workaround

[chen-keinan]

lets have discussion on the issue #6692, its open now

[chen-keinan]

I have opened the issue #6692 following investigation

Added a fix: #7107