Skip to content

Demonstrating how you can take an action to your intrusions detected by Falco using OpenFaaS functions

Notifications You must be signed in to change notification settings

developer-guy/falco-the-kubernetes-response-engine-using-openfaas-functions

Repository files navigation

OpenFaaS k3s Falco CIVO

Kubernetes Response Engine powered by OpenFaaS

Although Falco can be used to detect any intrusion attempts and sends alerts to channels according to the given rules afterwards, it does not have any remediation system. This is why we need something called Kubernetes Response Engine. It simply aims to catch alerts and take actions on it. These actions can be designed as fine-grained serverless functions.

Think of a scenario you want to take action to your alerts that being notified by Falco, but the important thing that we should notice here is we are going to do different things for the different levels of alerts. For example, we are going to send the notification for notice level alert but for the warning level alert we are going to delete the pod, the overall architecture will be like this:

                +-----------+
                |   Falco   +
                +-----^-----+
                      |
              +-------v-------+
              >OpenFaaS (dispatch-fn)+
              +-------v-------+
+-----------+         |          +-----------+
| notify-fn <---------+----------> delete-fn |
+-----v-----+ notice     warning +-----+-----+
      |                                |
      | send alert          delete pod |
      |                                |
+-----v-----+                    +-----v-----+       
|   Slack   |                    | Pwned Pod |
+-----------+                    +-----------+

🎁 Table of Contents

Prerequisites

  • civo cli v0.7.6
  • Helm v3.5.1
  • faas-cli 0.13.9
  • arkade 0.7.10
  • kubectl v1.20.2

We are going to do this demo on macOS Catalina 1.15.7, you can find the prerequisites on brew.

Target Audience

If you want to:

  • Set up a K3S cluster on CIVO
  • Set up the Falco
  • Create custom serverless functions using OpenFaaS
  • Subscribe Falco events from serverless functions

What is ...?

Built for speed and simplicity, with K3s under the hood

Join the #Kube100 beta: Apply to join today (Get free credit to test-drive the world’s first K3s-powered, managed Kubernetes service)

  • Simple, fast and powerful

Spin up Kubernetes in under 2 minutes, without the bloat, using the lightweight K3s distribution

  • Management CLI

Manage your clusters with the custom-built CLI and streamline your deployment with simple REST API.

  • Application marketplace

Launch clusters with preinstalled applications, or install on the fly using Civo's Kubernetes marketplace.

  • Bandwidth Pooling

Use all your bandwidth allowance across all your Civo services. Only pay for what you use.

  • Marketplace

Install applications quickly and easily to your clusters during or after set up with Civo marketplace. Also you can contribute an application to marketplace

  • Custom Firewalls

Configure custom firewalls from your Civo dashboard using OpenStack’s security groups.

The certified Kubernetes distribution built for IoT & Edge computing

  • Perfect for Edge

K3s is a highly available, certified Kubernetes distribution designed for production workloads in unattended, resource-constrained, remote locations or inside IoT appliances. Simplified & Secure

  • Simplified & Secure

K3s is packaged as a single <40MB binary that reduces the dependencies and steps needed to install, run and auto-update a production Kubernetes cluster.

  • Optimized for ARM

Both ARM64 and ARMv7 are supported with binaries and multiarch images available for both. K3s works great from something as small as a Raspberry Pi to an AWS a1.4xlarge 32GiB server.

Cloud-Native runtime security, de facto Kubernetes threat detection engine

  • Strengthen container security

The flexible rules engine allows you to describe any type of host or container behavior or activity.

  • Reduce risk via immediate alerts

You can immediately respond to policy violation alerts and integrate Falco within your response workflows.

  • Leverage most current detection rules

Falco out-of-the box rules alert on malicious activity and CVE exploits.

OpenFaaS® makes it simple to deploy both functions and existing code to Kubernetes

  • Anywhere

Avoid lock-in through the use of Docker. Run on any public or private cloud.

  • Any code

Build both microservices & functions in any language. Legacy code and binaries.

  • Any scale

Auto-scale for demand or to zero when idle.

Hands-on Demonstration

Create CIVO Playground

civo_dashboard_settings

$ civo apikey save my-awesome-key $KEY
Saved the API Key $KEY as my-awesome-key
$ civo apikey list
+----------------+---------+
| Name           | Default |
+----------------+---------+
| my-awesome-key | $KEY    |
+----------------+---------+
$ civo apikey current my-awesome-key
Set the default API Key to be my-awesome-key
  • Create a cluster
$ civo kubernetes create playground --size=g3.k3s.medium --nodes=3 --region NYC1 --wait
Created Kubernetes cluster playground
$ civo kubernetes show playground
          ID : 79435efe-2dac-403d-bfd2-f6644988830a
        Name : playground
       Nodes : 3
        Size : g3.k3s.medium
      Status : ACTIVE
     Version : 1.20.0-k3s2
API Endpoint : https://212.2.243.151:6443
   Master IP : 212.2.243.151
DNS A record : 79435efe-2dac-403d-bfd2-f6644988830a.k8s.civo.com
Nodes:
+-------------+---------------+--------+---------------+-----------+------+----------+
| Name        | IP            | Status | Size          | Cpu Cores | Ram  | SSD disk |
+-------------+---------------+--------+---------------+-----------+------+----------+
| master-7c8a | 212.2.243.151 | ACTIVE | g3.k3s.medium |         2 | 4096 |       25 |
| node-04ed   |               | ACTIVE | g3.k3s.medium |         2 | 4096 |       25 |
| node-5258   |               | ACTIVE | g3.k3s.medium |         2 | 4096 |       25 |
+-------------+---------------+--------+---------------+-----------+------+----------+

civo_dashboard_settings

  • Configure the playground
$ civo kubernetes config playground --save --local-path ./kubeconfig
Access your cluster with:
KUBECONFIG=./kubeconfig kubectl get node
  • Ensure all is OK
$ export KUBECONFIG=./kubeconfig
$ kubectl get node
NAME                                  STATUS   ROLES                  AGE   VERSION
k3s-playground-66b18d51-node-04ed     Ready    <none>                 40h   v1.20.2+k3s1

You can find more details about civo cli here.

  • Set up OpenFaaS

You can install OpenFaaS from Civo marketplace -

or via arkade

arkade provides a portable marketplace for downloading your favourite devops CLIs and installing helm charts, with a single command. You can also download CLIs like kubectl, kind, kubectx and helm faster than you can type "apt-get/brew update"

$ arkade install openfaas
Using Kubeconfig: /Users/batuhan.apaydin/.kube/config
Using Kubeconfig: /Users/batuhan.apaydin/.kube/config
Client: x86_64, Darwin
2021/03/11 21:35:24 User dir established as: /Users/batuhan.apaydin/.arkade/
"openfaas" already exists with the same configuration, skipping

Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "nats" chart repository
...Successfully got an update from the "kyverno" chart repository
...Successfully got an update from the "dq-helm-charts" chart repository
...Successfully got an update from the "falcosecurity" chart repository
...Successfully got an update from the "openfaas" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈Happy Helming!⎈

VALUES values.yaml
Command: /Users/batuhan.apaydin/.arkade/bin/helm [upgrade --install openfaas openfaas/openfaas --namespace openfaas --values /var/folders/pf/6h9t0mnd4d342ncgpjq_3zl80000gp/T/charts/openfaas/values.yaml --set basicAuthPlugin.replicas=1 --set basic_auth=true --set clusterRole=false --set gateway.directFunctions=false --set openfaasImagePullPolicy=IfNotPresent --set faasnetes.imagePullPolicy=Always --set queueWorker.maxInflight=1 --set serviceType=NodePort --set operator.create=false --set gateway.replicas=1 --set ingressOperator.create=false --set queueWorker.replicas=1]
Release "openfaas" does not exist. Installing it now.
NAME: openfaas
LAST DEPLOYED: Thu Mar 11 21:35:30 2021
NAMESPACE: openfaas
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
To verify that openfaas has started, run:

  kubectl -n openfaas get deployments -l "release=openfaas, app=openfaas"
=======================================================================
= OpenFaaS has been installed.                                        =
=======================================================================

# Get the faas-cli
curl -SLsf https://cli.openfaas.com | sudo sh

# Forward the gateway to your machine
kubectl rollout status -n openfaas deploy/gateway
kubectl port-forward -n openfaas svc/gateway 8080:8080 &

# If basic auth is enabled, you can now log into your gateway:
PASSWORD=$(kubectl get secret -n openfaas basic-auth -o jsonpath="{.data.basic-auth-password}" | base64 --decode; echo)
echo -n $PASSWORD | faas-cli login --username admin --password-stdin

faas-cli store deploy figlet
faas-cli list

# For Raspberry Pi
faas-cli store list \
 --platform armhf

faas-cli store deploy figlet \
 --platform armhf

# Find out more at:
# https://github.com/openfaas/faas

Thanks for using arkade!

Also, there are other ways to install OpenFaaS, here is the official documentation.

  • Set up Falco
$ arkade install falco \
        --set falco.jsonOutput=true \
        --set falco.httpOutput.enabled=true \
        --set falco.httpOutput.url=http://gateway.openfaas:8080/function/dispatch-fn
...

After everyting is ok, configure the faas-cli in order to deploy functions.

# Forward the gateway to your machine
$ kubectl port-forward -n openfaas svc/gateway 8080:8080 &

# If basic auth is enabled, you can now log into your gateway:
$ PASSWORD=$(kubectl get secret -n openfaas basic-auth -o jsonpath="{.data.basic-auth-password}" | base64 --decode; echo)
$ echo -n $PASSWORD | faas-cli login --username admin --password-stdin

Let's see what functions present.

$ faas-cli list
Function                        Invocations     Replicas

You should see nothing.

NOTE: Don't forget to apply roles.yaml for delete-pod-fn because this function will remove the pod if necessarry, so, it needs some priviliges for that.

Now Deploy the functions.

$ kubectl apply -f roles.yaml
$ faas-cli deploy -f stack.yml
...

Once it's completed, you should see the pods that's running on openfaas-fn namespace, and you should also see functions by typing faas-cli list command.

$ faas-cli list
Function                        Invocations     Replicas
dispatch-fn                     37              1
notifier-fn                     23              1
delete-pod-fn                   0               1

$ kubectl get pods --namespace=openfaas-fn
Found existing alias for "kubectl get pods". You should use: "kgp"
NAME                             READY   STATUS    RESTARTS   AGE
dispatch-fn-b5b68df77-r7crl      1/1     Running   0          7m23s
notifier-fn-5fd846c887-hgpf4     1/1     Running   0          4m48s
delete-pod-fn-55cc89965c-fs55q   1/1     Running   0          37s

If everything works well, you should see the Slack messages in the #falco-alert channel. Here is the screenshot above that shows this.

Learn more about how you can set up a Slack workspace from this link.

falco_alerts

The next thing that you need to do this, you should create an alert at Warning level, in order to that, you need to create a Pod based on the alpine image, and you should connect to it, and access some important folders like /etc/shadow. Once this is done, you should notice that the Pod gets killed by the delete-pod-fn function.

Let's test it with creating a Pod.

$  kubectl run alpine -n default --image=alpine --restart='Never' -- sh -c "sleep 600"
Found existing alias for "kubectl". You should use: "k"
pod/alpine created

Connect to it.

$ kubectl exec -ti alpine -- sh
Found existing alias for "kubectl exec -ti". You should use: "keti"
/ #

Once you connect it, open a second terminal and watch the state of the Pod.

$ watch kubectl get pods
Every 2.0s: kubectl get pods
NAME     READY   STATUS    RESTARTS   AGE
alpine   1/1     Running   0          106s

Then, in the first terminal, run the cat /etc/shadow, then look at the second terminal, you should notice that Pod's status changed from Running to Terminating, and your connection to the Pod is lost. pod_delete

About

Demonstrating how you can take an action to your intrusions detected by Falco using OpenFaaS functions

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 3

  •  
  •  
  •  

Languages