Skip to content
This repository has been archived by the owner on Mar 21, 2024. It is now read-only.

Commit

Permalink
Document the latest metrics improvements (#242)
Browse files Browse the repository at this point in the history 
* document the latest metrics improvements

* Update text/0174-metrics-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0174-metrics-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* adds some details on the meilisearch_db_size metrics

* update the open api metrics smaple

* update a bad copy paste

* Update open-api.yaml

* prefix prometheus metrics by meilisearch_

* Introduce two new metrics

---------

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
  • Loading branch information
@irevoire @gmourier @Kerollmops
3 people authored Jul 31, 2023
1 parent 0e79bef commit b67f0d8
Show file tree
Hide file tree
Showing 2 changed files with 141 additions and 42 deletions.
94 changes: 69 additions & 25 deletions open-api.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4151,37 +4151,81 @@ paths:
type: string
examples:
sampleResponse: '
# HELP http_requests_total HTTP requests total
# TYPE http_requests_total counter
http_requests_total{method="GET",path="/metrics"} 3
# HELP http_response_time_seconds HTTP response times
# TYPE http_response_time_seconds histogram
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.0005"} 0
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.0008"} 1
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.00085"} 1
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.0009"} 1
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.00095"} 1
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.001"} 1
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.00105"} 1
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.0011"} 1
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.00115"} 1
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.0012"} 1
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.0015"} 1
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.002"} 1
http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.003"} 1
http_response_time_seconds_bucket{method="GET",path="/metrics",le="1"} 2
http_response_time_seconds_bucket{method="GET",path="/metrics",le="+Inf"} 2
http_response_time_seconds_sum{method="GET",path="/metrics"} 0.0056515409999999995
http_response_time_seconds_count{method="GET",path="/metrics"} 2
# HELP meilisearch_db_size_bytes Meilisearch Db Size In Bytes
# HELP meilisearch_http_response_time_seconds HTTP response times
# TYPE meilisearch_http_response_time_seconds histogram
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.001"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.002"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.003"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.004"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.005"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.006"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.007"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.008"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.009"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.01"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.02"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.03"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.04"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.05"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.06"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.07"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.08"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.09"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.1"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.2"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.3"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.4"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.5"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.6"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.7"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.8"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="0.9"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="1"} 0
meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="+Inf"} 0
meilisearch_http_response_time_seconds_sum{method="GET",path="/metrics"} 0
meilisearch_http_response_time_seconds_count{method="GET",path="/metrics"} 0
# HELP meilisearch_db_size_bytes Meilisearch DB Size In Bytes
# TYPE meilisearch_db_size_bytes gauge
meilisearch_db_size_bytes 155648
meilisearch_db_size_bytes 983040
# HELP meilisearch_http_requests_total Meilisearch HTTP requests total
# TYPE meilisearch_http_requests_total counter
meilisearch_http_requests_total{method="GET",path="/metrics"} 1
# HELP meilisearch_index_count Meilisearch Index Count
# TYPE meilisearch_index_count gauge
meilisearch_index_count 1
# HELP meilisearch_index_docs_count Meilisearch Index Docs Count
# TYPE meilisearch_index_docs_count gauge
meilisearch_index_docs_count{index="movies"} 0'
meilisearch_index_docs_count{index="mieli"} 1
# HELP meilisearch_nb_tasks Meilisearch Number of tasks
# TYPE meilisearch_nb_tasks gauge
meilisearch_nb_tasks{kind="indexes",value="mieli"} 6
meilisearch_nb_tasks{kind="statuses",value="canceled"} 0
meilisearch_nb_tasks{kind="statuses",value="enqueued"} 0
meilisearch_nb_tasks{kind="statuses",value="failed"} 2
meilisearch_nb_tasks{kind="statuses",value="processing"} 0
meilisearch_nb_tasks{kind="statuses",value="succeeded"} 4
meilisearch_nb_tasks{kind="types",value="documentAdditionOrUpdate"} 2
meilisearch_nb_tasks{kind="types",value="documentDeletion"} 2
meilisearch_nb_tasks{kind="types",value="documentDeletionByFilter"} 0
meilisearch_nb_tasks{kind="types",value="dumpCreation"} 0
meilisearch_nb_tasks{kind="types",value="indexCreation"} 2
meilisearch_nb_tasks{kind="types",value="indexDeletion"} 0
meilisearch_nb_tasks{kind="types",value="indexSwap"} 0
meilisearch_nb_tasks{kind="types",value="indexUpdate"} 0
meilisearch_nb_tasks{kind="types",value="settingsUpdate"} 0
meilisearch_nb_tasks{kind="types",value="snapshotCreation"} 0
meilisearch_nb_tasks{kind="types",value="taskCancelation"} 0
meilisearch_nb_tasks{kind="types",value="taskDeletion"} 0
# HELP meilisearch_last_update Meilisearch Last Update
# TYPE meilisearch_last_update gauge
meilisearch_last_update 1689768676
# HELP meilisearch_is_indexing Meilisearch Is Indexing
# TYPE meilisearch_is_indexing gauge
meilisearch_is_indexing 1
# HELP meilisearch_used_db_size_bytes Meilisearch Used DB Size In Bytes
# TYPE meilisearch_used_db_size_bytes gauge
meilisearch_used_db_size_bytes 344064'
operationId: metrics.get
parameters: []
security:
Expand Down
89 changes: 72 additions & 17 deletions text/0174-metrics-api.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,25 +33,29 @@ A metric is composed by several fields:

Meilisearch returns the metrics specified in the table below.

| Name | Type |
|---------------------------------------------------------------------------|-----------|
| [`http_requests_total`](#321-http_requests_total) | counter |
| [`http_response_time_seconds`](#322-http_response_time_seconds) | histogram |
| [`meilisearch_database_size_bytes`](#323-meilisearch_database_size_bytes) | gauge |
| [`meilisearch_index_docs_count`](#324-meilisearch_index_docs_count) | gauge |
| [`meilisearch_index_count`](#325-meilisearch_index_count) | gauge |

#### 3.2.1 `http_requests_total`
| Name | Type |
|----------------------------------------------------------------------------------------|-----------|
| [`meilisearch_http_requests_total`](#321-meilisearch_http_requests_total) | counter |
| [`meilisearch_http_response_time_seconds`](#322-meilisearch_http_response_time_seconds)| histogram |
| [`meilisearch_db_size_bytes`](#323-meilisearch_db_size_bytes) | gauge |
| [`meilisearch_used_db_size_bytes`](#324-meilisearch_used_db_size_bytes) | gauge |
| [`meilisearch_index_docs_count`](#325-meilisearch_index_docs_count) | gauge |
| [`meilisearch_index_count`](#326-meilisearch_index_count) | gauge |
| [`meilisearch_nb_tasks`](#327-meilisearch_nb_tasks) | counter |
| [`meilisearch_last_update`](#328-meilisearch_last_update) | gauge |
| [`meilisearch_is_indexing`](#329-meilisearch_is_indexing) | gauge |

#### 3.2.1 `meilisearch_http_requests_total`

Returns the number of times an API resource is accessed.

```
# HELP http_requests_total HTTP requests total
# TYPE http_requests_total counter
http_requests_total{method=":httpMethod",path=":resourcePath"} :numberOfRequest
meilisearch_http_requests_total{method=":httpMethod",path=":resourcePath"} :numberOfRequest
```

#### 3.2.2. `http_responses_time_seconds`
#### 3.2.2. `meilisearch_http_response_time_seconds`

Returns a time histogram showing the number of times an API resource call goes into a time bucket (expressed in second).

Expand All @@ -74,20 +78,33 @@ http_response_time_seconds_bucket{method=":httpMethod",path=":resourcePath",le="
http_response_time_seconds_bucket{method=":httpMethod",path=":resourcePath",le="1"} :numberOfRequest
http_response_time_seconds_bucket{method=":httpMethod",path=":resourcePath",le="+Inf"} :numberOfRequest
http_response_time_seconds_sum{method=":httpMethod",path=":resourcePath"} :numberOfRequest
http_response_time_seconds_count{method=":httpMethod",path=":resourcePath"} :numberOfRequest
meilisearch_http_response_time_seconds_count{method=":httpMethod",path=":resourcePath"} :numberOfRequest
```

#### 3.2.3. `meilisearch_database_size_bytes`
#### 3.2.3. `meilisearch_db_size_bytes`

Returns the size of the database in bytes.
Returns the “real” size of the database on disk in bytes.
It includes all the lmdb memory mapped files plus all the files contained in the `data.ms` directory (mainly the updates files that were not processed yet).

```
# HELP meilisearch_db_size_bytes Meilisearch Db Size In Bytes
# TYPE meilisearch_db_size_bytes gauge
meilisearch_db_size_bytes :databaseSizeInBytes
```

#### 3.2.4. `meilisearch_index_docs_count`
#### 3.2.4. `meilisearch_used_db_size_bytes`

Returns the size of the database actually used by meilisearch in bytes.
Include all the same files as `meilisearch_db_size_bytes` except that when it comes to an LMDB database, we only count the pages used by meilisearch.
This means if you see a large gap between both metrics, adding documents will probably re-use freed pages instead of growing `meilisearch_db_size_bytes`.

```
# HELP meilisearch_used_db_size_bytes Meilisearch Used DB Size In Bytes
# TYPE meilisearch_used_db_size_bytes gauge
meilisearch_used_db_size_bytes :databaseSizeInBytes
```

#### 3.2.5. `meilisearch_index_docs_count`

Returns the number of documents for an index.

Expand All @@ -97,7 +114,7 @@ Returns the number of documents for an index.
meilisearch_index_docs_count{index=":indexUid"} :numberOfDocuments
```

#### 3.2.5. `meilisearch_index_count`
#### 3.2.6. `meilisearch_index_count`

Returns the total number of index for the Meilisearch instance.

Expand All @@ -107,6 +124,44 @@ Returns the total number of index for the Meilisearch instance.
meilisearch_index_count :numberOfIndex
````
#### 3.2.7. `meilisearch_nb_tasks`
Returns the total number of tasks for the Meilisearch instance parametrized by the kind of task and its value (see the table below).
```
# HELP meilisearch_nb_tasks Meilisearch Number of tasks
# TYPE meilisearch_nb_tasks gauge
meilisearch_nb_tasks{kind=":kind",value=":value"} :number
````
Here is a list of available kind and associated values:
| Kind | values |
|-----------|------------------------------------------------------------------------|
| indexes | Any created indexes |
| statuses | Any task statuses (i.e.: succeeded, failed, enqueued, etc...) |
| types | Any task types (i.e.: documentAdditionOrUpdate, settingsUpdate, etc...)|
#### 3.2.8. `meilisearch_last_update`
Returns the timestamp of the last update.
```
# HELP meilisearch_last_update Meilisearch Last Update
# TYPE meilisearch_last_update gauge
meilisearch_last_update :unixTimestamp
````

#### 3.2.8. `meilisearch_is_indexing`

Returns `1` if Meilisearch is indexing or `0` if not.

```
# HELP meilisearch_is_indexing Meilisearch Is Indexing
# TYPE meilisearch_is_indexing gauge
meilisearch_is_indexing :isIndexing
````
### 3.3. API Endpoints Definition
#### 3.3.1. `GET` - `/metrics`
Expand Down Expand Up @@ -145,4 +200,4 @@ N/A
## 5. Future Possibilities
- Merge `/stats` with `/metrics`. A header could specify the prefered format. e.g `application/json` (similar to actual `stats` resource) or `text/plain` (prometheus)
- Merge `/stats` with `/metrics`. A header could specify the prefered format. e.g `application/json` (similar to actual `stats` resource) or `text/plain` (prometheus)

0 comments on commit b67f0d8

Please sign in to comment.