Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Nginx] Add SLO's with Nginx Package #10269

Closed
wants to merge 12 commits into from
Closed

[Nginx] Add SLO's with Nginx Package #10269

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
fc23f39
add slos to nginx integration
muthu-mps Jun 26, 2024
68f1f18
skipping validation errors
muthu-mps Jun 27, 2024
ac5e390
replace package-spec version
muthu-mps Jun 27, 2024
30161fe
fix CI error
muthu-mps Jun 27, 2024
029e7e5
revert blob store error
muthu-mps Jun 27, 2024
d84e352
Update packages/nginx/changelog.yml
muthu-mps Jun 27, 2024
031df18
Add SLO section in the README
muthu-mps Jun 27, 2024
0603c97
Merge branch 'add_slo' of github.com:muthu-mps/integrations into add_slo
muthu-mps Jun 27, 2024
9c32379
address review comments
muthu-mps Jun 27, 2024
2f311c8
update dataview
muthu-mps Jul 24, 2024
4ecfa70
resolve conflicts
muthu-mps Jul 24, 2024
d1ba630
filter nginx log data
muthu-mps Jul 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions packages/nginx/_dev/build/docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,3 +130,13 @@ Find unusual activity in HTTP access logs.
| `source_ip_request_rate_nginx` | HTTP Access Logs: Detect unusual source IPs - high request rates |
| `low_request_rate_nginx` | HTTP Access Logs: Detect low request rates |

## SLOs

SLOs are usually defined in terms of metrics such as availability, response time, and throughput, and they are used to ensure that the service meets the needs of its users.
You can use the Nginx integration to measure the availability of the Nginx service. The SLOs gets created automatically during the installation of the integration. The user can view the created SLOs in the Observability -> SLOs page.
>Note: To create and manage SLOs you need an appropriate [license](https://www.elastic.co/subscriptions).

| Events | Query |
|-------------|-----------------------------------|
| Good Query | `http.response.status_code < 500` |
| Total Query | `http.response.status_code : *` |
5 changes: 5 additions & 0 deletions packages/nginx/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.24.0"
changes:
- description: Add SLO to the Nginx package.
type: enhancement
link: https://github.com/elastic/integrations/pull/10269
- version: "1.23.0"
changes:
- description: Add processor support for stubstatus data stream.
Expand Down
10 changes: 10 additions & 0 deletions packages/nginx/docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -478,3 +478,13 @@ Find unusual activity in HTTP access logs.
| `source_ip_request_rate_nginx` | HTTP Access Logs: Detect unusual source IPs - high request rates |
| `low_request_rate_nginx` | HTTP Access Logs: Detect low request rates |

## SLOs

SLOs are usually defined in terms of metrics such as availability, response time, and throughput, and they are used to ensure that the service meets the needs of its users.
You can use the Nginx integration to measure the availability of the Nginx service. The SLOs gets created automatically during the installation of the integration. The user can view the created SLOs in the Observability -> SLOs page.
>Note: To create and manage SLOs you need an appropriate [license](https://www.elastic.co/subscriptions).

| Events | Query |
|-------------|-----------------------------------|
| Good Query | `http.response.status_code < 500` |
| Total Query | `http.response.status_code : *` |
28 changes: 28 additions & 0 deletions packages/nginx/kibana/slo/nginx-9eb25600-a1f0-11e7-928f-5dbe6f6f5510.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"attributes": {
"name": "Nginx Availability",
"description": "",
"indicator": {
"type": "sli.kql.custom",
"params": {
"index": "logs-*",
"filter": "data_stream.dataset :\"nginx.access\" ",
"good": "http.response.status_code < 500",
"total": "http.response.status_code : *",
"timestampField": "@timestamp"
}
},
"budgetingMethod": "occurrences",
"timeWindow": {
"duration": "7d",
"type": "rolling"
},
"objective": {
"target": 0.99
},
"tags": []
},
"id": "nginx-9eb25600-a1f0-11e7-928f-5dbe6f6f5510",
"references": [],
"type": "slo"
}
4 changes: 2 additions & 2 deletions packages/nginx/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
format_version: "3.0.2"
format_version: "3.3.0"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsoriano - I have updated the version to 3.3.0 to test the behaviour and I am getting this error.

Error: building package failed: invalid content found in built zip package: could not load specification for version [3.3.0]: spec version "3.3.0" not found

Can you suggest on how to test this change with the pre-release version?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No released version of package spec contains this spec version, you will need to add a replace in the meantime, something like this:

go mod edit -replace github.com/elastic/package-spec/v3=github.com/elastic/package-spec/v3@main
go mod tidy

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the format_version that we could use for testing the package?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the format_version that we could use for testing the package?

format_version: 3.3.0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsoriano - Tried the above step and with the format_version:3.3.0 we are still getting the same error.

Copy link
Member

@jsoriano jsoriano Jun 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, yes, this should be enough for CI.

If you want to use elastic-package locally with this dependency you can run it as go run github.com/elastic/elastic-package from the integrations repository.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jsoriano!

  • We are getting few validation errors in different package, I am skipping those for now as those are not related.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, after updating go.mod all tests are executed, but you can ignore the ones not related to your package.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No released version of package spec contains this spec version

@jsoriano When is the next release of the package spec?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsoriano When is the next release of the package spec?

We release more or less on demand, and we have just released 3.2.1. Maybe we can release 3.2.2 soon with a couple of bugfixes. This release would also include 3.3.0 prerelease, what will allow to use SLOs in prerelease packages.

name: nginx
title: Nginx
version: "1.23.0"
version: "1.24.0"
description: Collect logs and metrics from Nginx HTTP servers with Elastic Agent.
type: integration
categories:
Expand Down
4 changes: 4 additions & 0 deletions packages/nginx/validation.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
errors:
exclude_checks:
- PSR00001 # References in dashboards.
- JSE00001 # Kibana version for saved tags.