Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance: add new testing section and refactor the structure #630

Merged
merged 1 commit into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 65 additions & 4 deletions crowdsec-docs/docs/notification_plugins/elasticsearch.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,20 @@ id: elastic
title: Elasticsearch
---

Elasticsearch can be integrated with CrowdSec by using the HTTP plugin. Enable it by following these [instructions](/notification_plugins/http.md) .
CrowdSec can forward Alerts to Elasticsearch using the HTTP plugin. This guide will show you how to configure the plugin to send alerts to your Elasticsearch instance.

## Configuring the plugin

By default the configuration for HTTP plugin is located at these default location per OS:

- **Linux** `/etc/crowdsec/notifications/http.yaml`
- **FreeBSD** `/usr/local/etc/crowdsec/notifications/http.yaml`
- **Windows** `C:\ProgramData\CrowdSec\config\notifications\http.yaml`

Then replace the `url` and the `format` of the plugin's config so that it posts the events to your Elasticsearch instance.

### Base configuration

An example configuration:

```yaml
Expand All @@ -29,7 +39,7 @@ headers:
```


## Authentication
### Authentication

If you have enabled security on your elasticsearch cluster, you will have to add a custom `Authorization` header to be able to insert the events.

Expand Down Expand Up @@ -61,7 +71,7 @@ headers:
```


## Self-Signed certificate
### Self-Signed certificate

If your elasticsearch cluster uses a self-signed certificate, you must set `skip_tls_verification` to `true` in your configuration:
```yaml
Expand All @@ -84,7 +94,7 @@ headers:

```

## Potential mapping issues
### Potential mapping issues

If you are facing errors because mapper complains about field types inference, ie:

Expand Down Expand Up @@ -355,6 +365,57 @@ headers:
Authorization: "Basic [redacted]"
```

## Testing the plugin

Before enabling the plugin it is best to test the configuration so the configuration is validated and you can see the output of the plugin.

```bash
cscli notifications test http_default
```

:::note
If you have changed the `name` property in the configuration file, you should replace `http_default` with the new name.
:::

## Enabling the plugin

In your profiles you will need to uncomment the `notifications` key and the `http_default` plugin list item.

```
#notifications:
# - http_default
```

:::note
If you have changed the `name` property in the configuration file, you should replace `http_default` with the new name.
:::

:::warning
Ensure your YAML is properly formatted the `notifications` key should be at the top level of the profile.
:::

<details>

<summary>Example profile with http plugin enabled</summary>

```yaml
name: default_ip_remediation
#debug: true
filters:
- Alert.Remediation == true && Alert.GetScope() == "Ip"
decisions:
- type: ban
duration: 4h
#duration_expr: Sprintf('%dh', (GetDecisionsCount(Alert.GetValue()) + 1) * 4)
#highlight-next-line
notifications:
#highlight-next-line
- http_default
on_success: break
```

</details>

## Final Steps:

Let's restart crowdsec
Expand Down
93 changes: 75 additions & 18 deletions crowdsec-docs/docs/notification_plugins/email.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,27 +3,19 @@ id: email
title: Email Plugin
---

The Email plugin is shipped by default with CrowdSec. This guide shows how to enable it.
The Email plugin is shipped by default with CrowdSec. The following guide shows how to configure, test and enable it.

## Enabling the plugin:
## Configuring the plugin

In the profile configuration (by default `/etc/crowdsec/profiles.yaml`) , uncomment the section:
By default the configuration for Email plugin is located at these default location per OS:

```
#notifications:
# - email_default
```

Every alert that passes the profile's filter will be dispatched to the `email_default` plugin.
- **Linux** `/etc/crowdsec/notifications/email.yaml`
- **FreeBSD** `/usr/local/etc/crowdsec/notifications/email.yaml`
- **Windows** `C:\ProgramData\CrowdSec\config\notifications\email.yaml`

## Configuring the plugin:
### Base configuration

The default configuration for the email plugin is located at `/etc/crowdsec/notifications/email.yaml`.
You need to provide the credentials for the SMTP server here.

### Example configuration for Gmail

Here's an example configuration that sends alerts to `receiver@gmail.com`:
Here is the base configuration for the Email plugin:

```yaml
type: email # Don't change
Expand Down Expand Up @@ -84,12 +76,77 @@ encryption_type: "ssltls"

The `format` configuration directive is a [go template](https://pkg.go.dev/text/template), which receives a list of [Alert](https://pkg.go.dev/github.com/crowdsecurity/crowdsec@master/pkg/models#Alert) objects.

Typical port and TLS/SSL settings

| Port | Encryption Type |
|------|-----------------|
| 25 | none |
| 465 | ssltls |
| 587 | starttls |

:::warning
Port 25 should be avoided at all costs as it is commonly blocked by ISPs and email providers and is insecure as it sends in plain text.
:::

:::info
Port settings above are common, but may vary depending on your email provider. Please refer to your email provider's documentation for the correct settings.
:::

## Testing the plugin

Before enabling the plugin it is best to test the configuration so the configuration is validated and you can see the output of the plugin.

```bash
cscli notifications test email_default
```

:::note
If you have changed the `name` property in the configuration file, you should replace `email_default` with the new name.
:::

## Enabling the plugin

In your profiles you will need to uncomment the `notifications` key and the `email_default` plugin list item.

```
#notifications:
# - email_default
```

:::note
If you have changed the `name` property in the configuration file, you should replace `email_default` with the new name.
:::

:::warning
Ensure your YAML is properly formatted the `notifications` key should be at the top level of the profile.
:::

<details>

<summary>Example profile with email plugin enabled</summary>

```yaml
name: default_ip_remediation
#debug: true
filters:
- Alert.Remediation == true && Alert.GetScope() == "Ip"
decisions:
- type: ban
duration: 4h
#duration_expr: Sprintf('%dh', (GetDecisionsCount(Alert.GetValue()) + 1) * 4)
#highlight-next-line
notifications:
#highlight-next-line
- email_default
on_success: break
```

</details>

## Final Steps:

Restart CrowdSec with the following command:

```bash
sudo systemctl restart crowdsec
```

To verify if the plugin is functioning correctly, you can trigger scenarios using tools like wapiti, nikto etc.
80 changes: 61 additions & 19 deletions crowdsec-docs/docs/notification_plugins/file.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,19 @@ id: file
title: File Plugin
---

The File plugin is by default shipped with your CrowdSec installation. The following guide shows how to enable it.
The File plugin is by default shipped with your CrowdSec installation and allows you to write Alerts to an external file that can be monitored by external applications. The following guide shows how to configure, test and enable it.

## Enabling the plugin:
## Configuring the plugin

In your profile file (by default `/etc/crowdsec/profiles.yaml`) , uncomment the section
By default the configuration for Email plugin is located at these default location per OS:

```
#notifications:
# - file_default
```

Every alert which would pass the profile's filter would be dispatched to `file_default` plugin.
- **Linux** `/etc/crowdsec/notifications/file.yaml`
- **FreeBSD** `/usr/local/etc/crowdsec/notifications/file.yaml`
- **Windows** `C:\ProgramData\CrowdSec\config\notifications\file.yaml`

## Configuring the plugin:
### Base configuration

By default the configuration for File plugin is located at `/etc/crowdsec/notifications/file.yaml`.

### Adding the plugin configuration

Example config which writes a ndjson file to `/tmp/crowdsec_alerts.json`.
Example config which writes Alerts to a file using NDJson (**N**ewline **D**elimiter **J**ava**S**cript **O**bject **N**otation) format to `/tmp/crowdsec_alerts.json`.

```yaml
# Don't change this
Expand Down Expand Up @@ -56,9 +49,9 @@ rotate:
Some SIEM agents may not support some top level keys we define in the default ndjson format. Please make sure to adjust the format to match your SIEM agent's requirements.
:::

## SIEM Integration
### SIEM Integration

### Filebeat
#### Filebeat

Filebeat has a set of reserved top level keys and should not be used in the ndjson format. The following format can be used to be compatible with Filebeat:

Expand All @@ -69,12 +62,61 @@ format: |
{{ end -}}
```

## Testing the plugin

Before enabling the plugin it is best to test the configuration so the configuration is validated and you can see the output of the plugin.

```bash
cscli notifications test file_default
```

:::note
If you have changed the `name` property in the configuration file, you should replace `file_default` with the new name.
:::

## Enabling the plugin

In your profiles you will need to uncomment the `notifications` key and the `file_default` plugin list item.

```
#notifications:
# - file_default
```

:::note
If you have changed the `name` property in the configuration file, you should replace `file_default` with the new name.
:::

:::warning
Ensure your YAML is properly formatted the `notifications` key should be at the top level of the profile.
:::

<details>

<summary>Example profile with file plugin enabled</summary>

```yaml
name: default_ip_remediation
#debug: true
filters:
- Alert.Remediation == true && Alert.GetScope() == "Ip"
decisions:
- type: ban
duration: 4h
#duration_expr: Sprintf('%dh', (GetDecisionsCount(Alert.GetValue()) + 1) * 4)
#highlight-next-line
notifications:
#highlight-next-line
- file_default
on_success: break
```

</details>

## Final Steps:

Let's restart crowdsec

```bash
sudo systemctl restart crowdsec
```

You can verify whether the plugin is properly working by triggering scenarios using tools like wapiti, nikto etc.
Loading
Loading