Skip to content

Commit

Permalink
update retina capture cli docs
Browse files Browse the repository at this point in the history
Signed-off-by: Kamil Przepiorowski <kamil.prz@gmail.com>
  • Loading branch information
kamilprz committed Sep 26, 2024
1 parent 9870ac4 commit 42a4324
Show file tree
Hide file tree
Showing 2 changed files with 188 additions and 115 deletions.
273 changes: 173 additions & 100 deletions docs/04-Captures/cli.md
Original file line number Diff line number Diff line change
@@ -1,203 +1,259 @@
# Capture with Retina CLI

Retina capture command allows the user to capture network traffic and metadata for the capture target, and then send the capture file to the location by Output Configuration.
The Retina capture command allows users to capture network traffic and metadata for the capture target, and send the data to the location defined by output configuration.

Note: captures can also be performed with a [Capture CRD](../05-Concepts/CRDs/Capture.md) after [installing Retina](../02-Installation/01-Setup.md) with capture support.
> Note: captures can also be performed with a [Capture CRD](../05-Concepts/CRDs/Capture.md) after [installing Retina](../02-Installation/01-Setup.md) **with capture support**.
## Retina capture create
## Starting a Capture

`kubectl retina capture create` creates a Capture with underlying Kubernetes jobs.
`kubectl retina capture create [--flags]` creates a Capture with underlying Kubernetes jobs.

### No wait(default true)
## Flags

Do not wait for the long-running capture job to finish.
<!-- TODO:
1. remove unnecessary flags from the capture targets
3. is name a required flag
-->

By default, Retina capture CLI will exit before the jobs are completed. With `--no-wait=false`, the CLI will wait until the jobs are completed and clean up the Kubernetes resources created.
### [required] Capture Target

#### Example
The capture target indicates where the packet capture will be performed. There are three choices here, each of which is its own flag. They are described below.

- do not wait for the long-running capture job to finish
#### Node Selectors

`kubectl retina capture create --name capture-test --host-path /mnt/capture --node-selectors "kubernetes.io/os=linux" --no-wait=true`
Capture network captures on nodes filtered by the provided by node selectors.

### Namespace(default "default")
**Example:**

Namespace to host capture job and the other k8s resources for a network capture. Please make sure the namespace exists.
`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux"`

#### Example
#### Node Names

- deploy the capture job in namespace `capture`
Capture network captures on nodes filtered by the provided node names.

`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux"`
**Example:**

`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-names "aks-nodepool1-41844487-vmss000000,aks-nodepool1-41844487-vmss000001"`

#### Pod Selectors & Namespace Selectors (Pairs)

Capture network captures on pods filtered by the provided pod-selector and namespace-selector **pairs**.

### Capture Target(required)
**Example:**

Capture target indicates the target on which the network packets capture will be performed, and the user can select either node, by node-selectors or node-names, or Pods, by pod-selector and namespace-selector pair.
`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --pod-selectors="k8s-app=kube-dns" --namespace-selectors="kubernetes.io/metadata.name=kube-system"`

---

### [required] Output Configuration

The output configuration indicates the location where the capture will be stored. At least one location needs to be specified. This can either be the host path on the node, or a remote storage option.

#### Examples
Blob-upload requires a Blob Shared Access Signature (SAS) with the write permission to the storage account container, to create SAS tokens in the Azure portal, please read: [Create SAS Tokens in the Azure Portal](https://learn.microsoft.com/en-us/azure/cognitive-services/translator/document-translation/how-to-guides/create-sas-tokens?tabs=Containers#create-sas-tokens-in-the-azure-portal).

- capture network packets on the node selected by node selectors
#### Host Path

Store the capture file in the node's specified host path. In the below example this is `/mnt/capture`.

**Example:**

`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux"`

- capture network packets on the node selected by node names
#### PVC

`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-names "aks-nodepool1-41844487-vmss000000,aks-nodepool1-41844487-vmss000001"`
Store the capture file to a PVC, in the example below `mypvc` with access mode ReadWriteMany in namespace `capture`.

- capture network packets on the pod selected by pod-selector and namespace-selector pairs
**Example:**

`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --pod-selectors="k8s-app=kube-dns" --namespace-selectors="kubernetes.io/metadata.name=kube-system"`
`kubectl retina capture create --name capture-test --pvc mypvc --namespace capture --node-selectors "kubernetes.io/os=linux"`

### Stop Capture(optional with default values)
#### Storage Account

The Capture can be stopped in either way below:
Store the capture file to a storage account (blob) through the use of a SAS.

- In a given time, by the `duration` flag, or when the allowed maximum capture file reaches by the `max-size` flag. When both are specified, the capture will stop when either condition first meets.
- On demand by deleting the capture before the specified conditions meets.
**Example:**

The network traffic will be uploaded to the specified output location.
`kubectl retina capture create --name capture-test --blob-upload <Blob SAS URL with write permission> --node-selectors "kubernetes.io/os=linux"`

#### Capture Duration
#### AWS S3

Duration of capturing packets(default 1m)
Store the capture file to AWS S3.

##### Example
**Example:**

- stop the capture in 2 minutes
`kubectl retina capture create --name capture-test --s3-bucket "your-bucket-name" --s3-region "eu-central-1" --s3-access-key-id "your-access-key-id" --s3-secret-access-key "your-secret-access-key" --node-selectors "kubernetes.io/os=linux"`

`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux" --duration=2m`
#### S3-compatible service

#### Maximum Capture Size
Store the capture file to S3-compatible service (such as MinIO).

Limit the capture file to MB in size(default 100MB)
**Example:**

##### Example
`kubectl retina capture create --name capture-test --s3-bucket "your-bucket-name" --s3-endpoint "https://play.min.io:9000" --s3-access-key-id "your-access-key-id" --s3-secret-access-key "your-secret-access-key" --node-selectors "kubernetes.io/os=linux"`

- stop the capture when the capture file size reaches 50MB
---

`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux" --max-size=50`
### No Wait

#### Packet Size(optional)
- Syntax: `--no-wait=<bool>`
- Default value: `true`
- Optional

Limits the each packet to bytes in size and packets longer than PacketSize will be truncated.
This is beneficial when the user wants to reduce the capture file size or hide customer data for security concern.
By default, Retina capture CLI will exit before the jobs are completed. With `--no-wait=false`, the CLI will wait until the jobs are completed and clean up the Kubernetes resources created.

**Example:**

##### Example
`kubectl retina capture create --name capture-test --host-path /mnt/capture --node-selectors "kubernetes.io/os=linux" --no-wait=true`

- limit each packet size to 96 bytes
---

`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux" --packet-size=96`
### Namespace

- Syntax: `--namespace <string>`
- Default value: `default`
- Optional

Sets the namespace which hosts the capture job and the other Kubernetes resources for a network capture. **Ensure the namespace exists**.

**Example:**

`kubectl retina capture create --name capture-test --host-path /mnt/capture --node-selectors "kubernetes.io/os=linux" --namespace capture`

### Capture Configuration(optional)
---

Capture configuration indicates the configurations of the network capture.
### Include / Exclude Filters

#### Packet capture filters
- Syntax (Include): `--include-filter="<string>"`
- Syntax (Exclude): `--exclude-filter="<string>"`
- Default value: `""`
- Optional

Packet capture filters represent a range of filters to be included/excluded in the capture. This is not available on Windows.
Represent a range of IP/port filters to be included or excluded from the capture. This is **not** available on Windows.

##### Example
**Example:**

`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux" --exclude-filter="10.224.0.26:80,10.224.0.33:8080" --include-filter="10.224.0.42:80,10.224.0.33:8080"`
`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux" --include-filter="10.224.0.42:80,10.224.0.33:8080" --exclude-filter="10.224.0.26:80,10.224.0.34:8080"`

#### Tcpdump Filter
---

Raw tcpdump flags which works only for Linux. Available tcpdump filters can be found in [TCPDUMP MAN PAGE](https://www.tcpdump.org/manpages/tcpdump.1.html)
NOTE: this includes only tcpdump flags, for expression part, please user [Packet include/exclude filters
](#packet-capture-filters).
### Tcpdump Filters

##### Example
- Syntax: `--tcpdump-filter="<string>"`
- Default value: `""`
- Optional

- filter DNS query
Raw tcpdump flags which only work on Linux. Available tcpdump filters can be found in the [TCPDUMP MAN PAGE](https://www.tcpdump.org/manpages/tcpdump.1.html).

NOTE: this includes only tcpdump flags, for boolean expressions, please use [Packet include/exclude filters](#include--exclude-filters).

**Example:**

`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux" --tcpdump-filter="udp port 53"`

#### Include Metadata
---

If true, collect static network metadata into the capture file(default true)
### Include Metadata

##### Example
- Syntax: `--include-metadata=<bool>`
- Default value: `true`
- Optional

- disable collecting network metadata
Collect static network metadata into the capture file if true.

**Example:**

`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux" --include-metadata=false`

### Job Number Limit(optional)
---

### Job Number Limit

The maximum number of jobs can be created for each capture. The default value 0 indicates no limit.
This can be configured by CLI flags for each CLI command, or config map consumed by retina-operator.
When creating a job requires job number exceeds this limit, it will fail with prompt like
`Error: the number of capture jobs 3 exceeds the limit 2`
- Syntax: `--job-num-limit=<int>`
- Default value: `0`
- Optional

#### Example
The maximum number of jobs which can be created for each capture. The default value 0 indicates no limit. This can be configured by CLI flags for each CLI command, or by a config map consumed by the retina-operator.

When creating a job requires job number exceeds this limit, it will fail with prompt like `Error: the number of capture jobs 3 exceeds the limit 2`.

**Example:**

`kubectl retina capture create --name capture-test --job-num-limit=10 --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux"`

### Output Configuration(required)
---

OutputConfiguration indicates the location capture will be stored, and at least one location should be specified.
### Packet Size Limit

Blob-upload requires a Blob Shared Access Signature with the write permission to the storage account container, to create SAS tokens in the Azure portal, please read [Create SAS Tokens in the Azure Portal](https://learn.microsoft.com/en-us/azure/cognitive-services/translator/document-translation/how-to-guides/create-sas-tokens?tabs=Containers#create-sas-tokens-in-the-azure-portal)
- Syntax: `--packet-size=<int>`
- Default value: `0`
- Optional

#### Examples
Limit the packet size in bytes. Packets longer than the defined maximum size will be truncated. The default value 0 indicates no limit. This is beneficial when the user wants to reduce the capture file size or hide customer data due to security concerns.

- store the capture file in the node host path `/mnt/capture`
**Example:**

`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux"`
`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux" --packet-size=96`

- store the capture file to a PVC `mypvc` with access mode ReadWriteMany in namespace `capture`
---

`kubectl retina capture create --name capture-test --pvc mypvc --namespace capture --node-selectors "kubernetes.io/os=linux"`
## Stopping a Capture

- store the capture file to a storage account
The Capture can be stopped in a number of ways:

`kubectl retina capture create --name capture-test --blob-upload <Blob SAS URL with write permission> --node-selectors "kubernetes.io/os=linux"`
- In a given time, by the `duration` flag, or when the file reaches the maximum allowed file size defined by the `max-size` flag. When both are specified, the capture will stop whenever either condition is first met.
- On demand by deleting the capture before the specified conditions meets.

- store the capture file to AWS S3
The network traffic will be uploaded to the specified output location.

`kubectl retina capture create --name capture-test --s3-bucket "your-bucket-name" --s3-region "eu-central-1" --s3-access-key-id "your-access-key-id" --s3-secret-access-key "your-secret-access-key" --node-selectors "kubernetes.io/os=linux"`
### Capture Duration

- store the capture file to S3-compatible service (like MinIO)
- Syntax: `--duration=<string>`
- Default value: `1m`
- Optional

`kubectl retina capture create --name capture-test --s3-bucket "your-bucket-name" --s3-endpoint "https://play.min.io:9000" --s3-access-key-id "your-access-key-id" --s3-secret-access-key "your-secret-access-key" --node-selectors "kubernetes.io/os=linux"`
Maximum duration of the packet capture - in minutes.

### Debug mode
**Example:**

With debug mode, when `--debug` is specified, we can overwrite the capture job Pod image from the default official `GHCR` one.
`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux" --duration=2m`

#### Examples
### Capture Size

- use `ghcr.io` image in default debug mode
- Syntax: `--max-size=<int>`
- Default value: `100`
- Optional

`kubectl retina capture create --name capture-test --host-path /mnt/test --namespace capture --node-selectors "kubernetes.io/os=linux" --debug`
Maximum size of the capture file - in MB.

- use customized retina-agent image
**Example:**

`RETINA_AGENT_IMAGE=<YOUR RETINA AGENT IMAGE> kubectl retina capture create --name capture-test --host-path /mnt/test --namespace capture --node-selectors "kubernetes.io/os=linux" --debug`
`kubectl retina capture create --name capture-test --host-path /mnt/capture --namespace capture --node-selectors "kubernetes.io/os=linux" --max-size=50`

## Retina capture delete
### Capture Delete

`kubectl retina capture delete` deletes a Kubernetes Jobs with the specified Capture name.
Deleting the capture job before either of the terminating conditions have been met will stop the capture.

### Examples
`kubectl retina capture delete --name <string>` deletes a Kubernetes Jobs with the specified Capture name.

`kubectl retina capture delete --name retina-capture-zlx5v`
**Example:**

## Retina capture list
`kubectl retina capture delete --name retina-capture-zlx5v`

`kubectl retina capture list` lists Captures in a namespace or all namespaces.
#### Get Capture List

### Examples
To get a list of the captures you can run `kubectl retina capture list` to get the captures in a specific namespace or in all namespaces.

- list Captures under namespace `capture`
**Example (namespace):**

`kubectl retina capture list --namespace capture`

- list Captures under in all namespaces
**Example (all namespaces):**

`kubectl retina capture list --all-namespaces`

## Obtain the output
---

## Obtaining the output

After downloading or copying the tarball from the location specified, extract the tarball through the `tar` command in either Linux shell or Windows Powershell, for example,

Expand Down Expand Up @@ -281,9 +337,26 @@ the tarball take such name pattern, `$(capturename)-$(hostname)-$(date +%Y%m%d%H
- kernel networking configuration (/proc/sys/net)

- Windows
- reference: [Microsoft SDN Debug tool
](https://github.com/microsoft/SDN/blob/master/Kubernetes/windows/debug/collectlogs.ps1)
- reference: [Microsoft SDN Debug tool](https://github.com/microsoft/SDN/blob/master/Kubernetes/windows/debug/collectlogs.ps1)

## Debug mode

With debug mode, when `--debug` is specified, you can overwrite the capture job's pod image.

**Example:**

Use `ghcr.io` image in default debug mode.

`kubectl retina capture create --name capture-test --host-path /mnt/test --namespace capture --node-selectors "kubernetes.io/os=linux" --debug`

**Example:**

Use custom retina-agent image by specifying it before the command.

`RETINA_AGENT_IMAGE=<YOUR RETINA AGENT IMAGE> kubectl retina capture create --name capture-test --host-path /mnt/test --namespace capture --node-selectors "kubernetes.io/os=linux" --debug`

## Cleanup

When creating a capture, we can specify `--no-wait` to clean up the jobs after the Capture is completed. Otherwise, after creating a Capture, a random Capture name is returned, with which we can delete the jobs by `kubectl retina capture delete` command.
When creating a capture, you can specify `--no-wait` to clean up the jobs after the Capture is completed.

Otherwise, after creating a Capture, a random Capture name is returned, with which you can delete the jobs by running the `kubectl retina capture delete` command.
Loading

0 comments on commit 42a4324

Please sign in to comment.