Activate metrics endpoint on aggregator #2022

dlachaume · 2024-10-18T15:23:51Z

Content

This PR includes:

Optional activation of a metrics (Prometheus) endpoint on the aggregator node.
Computation of the following metrics:
- certificate_detail_total_served_since_startup (Counter)
- artifact_detail_cardano_db_total_served_since_startup (Counter)
- artifact_detail_mithril_stake_distribution_total_served_since_startup (Counter)
- artifact_detail_cardano_stake_distribution_total_served_since_startup (Counter)
- artifact_detail_cardano_transaction_total_served_since_startup (Counter)
- proof_cardano_transaction_total_proofs_served_since_startup (Counter)
- proof_cardano_transaction_total_transactions_served_since_startup (Counter)
- signer_registration_total_received_since_startup (Counter)
- signature_registration_total_received_since_startup (Counter)
- certificate_total_produced_since_startup (Counter)
- artifact_cardano_db_total_produced_since_startup (Counter)
- artifact_mithril_stake_distribution_total_produced_since_startup (Counter)
- artifact_cardano_stake_distribution_total_produced_since_startup (Counter)
- artifact_cardano_transaction_total_produced_since_startup (Counter)
- runtime_cycle_success_since_startup (Counter)
- runtime_cycle_total_since_startup (Counter)
Deployment in the Mithril networks infrastructure.

`ab` tests on `testing-preview` BEFORE merging the PR

Number of requests: 1000

ab -n 1000 -c 50 https://aggregator.testing-preview.api.mithril.network/aggregator/certificate/62018d3b945ae75fe2c9749b0bb89f24e51e40c187c4d629edb2d5ce99d006f

Run 1

Requests per second:    298.81 [#/sec] (mean)
Time per request:       167.331 [ms] (mean)
Time per request:       3.347 [ms] (mean, across all concurrent requests)

Run 2

Requests per second:    295.42 [#/sec] (mean)
Time per request:       169.250 [ms] (mean)
Time per request:       3.385 [ms] (mean, across all concurrent requests)

Run 3

Requests per second:    307.03 [#/sec] (mean)
Time per request:       162.853 [ms] (mean)
Time per request:       3.257 [ms] (mean, across all concurrent requests)

Number of requests: 10 000

ab -n 10000 -c 50 https://aggregator.testing-preview.api.mithril.network/aggregator/certificate/62018d3b945ae75fe2c9749b0bb89f24e51e40c187c4d629edb2d5ce99d006f

Run 1

Requests per second:    309.23 [#/sec] (mean)
Time per request:       161.694 [ms] (mean)
Time per request:       3.234 [ms] (mean, across all concurrent requests)

Run 2

Requests per second:    299.44 [#/sec] (mean)
Time per request:       166.979 [ms] (mean)
Time per request:       3.340 [ms] (mean, across all concurrent requests)

Run 3

Requests per second:    312.36 [#/sec] (mean)
Time per request:       160.070 [ms] (mean)
Time per request:       3.201 [ms] (mean, across all concurrent requests)

`ab` tests on `testing-preview` AFTER merging the PR

Number of requests: 1000

ab -n 1000 -c 50 https://aggregator.testing-preview.api.mithril.network/aggregator/certificate/62018d3b945ae75fe2c9749b0bb89f24e51e40c187c4d629edb2d5ce99d006f

Run 1

Requests per second:    291.38 [#/sec] (mean)
Time per request:       171.595 [ms] (mean)
Time per request:       3.432 [ms] (mean, across all concurrent requests)

Run 2

Requests per second:    289.75 [#/sec] (mean)
Time per request:       172.563 [ms] (mean)
Time per request:       3.451 [ms] (mean, across all concurrent requests)

Run 3

Requests per second:    291.73 [#/sec] (mean)
Time per request:       171.392 [ms] (mean)
Time per request:       3.428 [ms] (mean, across all concurrent requests)

Number of requests: 10 000

ab -n 10000 -c 50 https://aggregator.testing-preview.api.mithril.network/aggregator/certificate/62018d3b945ae75fe2c9749b0bb89f24e51e40c187c4d629edb2d5ce99d006f

Run 1

Requests per second:    285.65 [#/sec] (mean)
Time per request:       175.039 [ms] (mean)
Time per request:       3.501 [ms] (mean, across all concurrent requests)

Run 2

Requests per second:    300.41 [#/sec] (mean)
Time per request:       166.438 [ms] (mean)
Time per request:       3.329 [ms] (mean, across all concurrent requests)

Run 3

Requests per second:    300.62 [#/sec] (mean)
Time per request:       166.320 [ms] (mean)
Time per request:       3.326 [ms] (mean, across all concurrent requests)

Pre-submit checklist

Branch
- Tests are provided (if possible)
- Crates versions are updated (if relevant)
- CHANGELOG file is updated (if relevant)
- Commit sequence broadly makes sense
- Key commits have useful messages
PR
- No clippy warnings in the CI
- Self-reviewed the diff
- Useful pull request description
- Reviewer requested
Documentation
- Update README file (if relevant)
- Update documentation website (if relevant)
- Add dev blog post (if relevant)

Issue(s)

Closes #1980

github-actions · 2024-10-18T15:31:21Z

Test Results

4 files ±0 55 suites ±0 10m 23s ⏱️ +2s
1 379 tests +9 1 379 ✅ +9 0 💤 ±0 0 ❌ ±0
1 593 runs +9 1 593 ✅ +9 0 💤 ±0 0 ❌ ±0

Results for commit 7352335. ± Comparison against base commit 6101238.

♻️ This comment has been updated with latest results.

jpraynaud

LGTM 👍

I left few comments and suggestions below.

I have also pushed a commit with the missing configuration of the scraping Prometheus endpoint for the aggregator metrics in the infrastructure: a67d64a.

CHANGELOG.md

mithril-aggregator/src/commands/serve_command.rs

mithril-aggregator/src/http_server/routes/proof_routes.rs

mithril-aggregator/src/metrics/service.rs

mithril-aggregator/src/runtime/runner.rs

mithril-aggregator/tests/prove_transactions.rs

mithril-aggregator/tests/test_extensions/mod.rs

Alenar

LGTM, just two minor issues.

mithril-aggregator/src/commands/serve_command.rs

mithril-aggregator/tests/test_extensions/runtime_tester.rs

Co-authored-by: Sébastien Fauvel <sfauvel@users.noreply.github.com>

…onfiguration

… JoinSet completes

…e parameters

…kAggregatorRunner` during state machine tests

…point

…endpoint

…er, fix typos and renaming

Co-authored-by: Sébastien Fauvel <sfauvel@users.noreply.github.com> Co-authored-by: DJO <Alenar@users.noreply.github.com>

…d for proof - Implement `increment_by` function for `MetricCounter` to increment the counter by another value than '1'. - Renaming of `proof_cardano_transaction_total_served_since_startup` to `proof_cardano_transaction_total_proofs_served_since_startup`.

* mithril-metric from `0.1.0` to `0.1.1` * mithril-aggregator from `0.5.84` to `0.5.85`

dlachaume self-assigned this Oct 18, 2024

dlachaume temporarily deployed to testing-preview October 21, 2024 10:51 — with GitHub Actions Inactive

dlachaume temporarily deployed to testing-sanchonet October 21, 2024 10:51 — with GitHub Actions Inactive

jpraynaud approved these changes Oct 21, 2024

View reviewed changes

dlachaume force-pushed the ensemble/1980/expose_prometheus_metrics_for_aggregator branch from d2fe79f to eefc17b Compare October 21, 2024 14:54

dlachaume marked this pull request as ready for review October 21, 2024 14:58

dlachaume requested review from Alenar and sfauvel October 21, 2024 14:58

Alenar approved these changes Oct 21, 2024

View reviewed changes

mithril-aggregator/src/commands/serve_command.rs Show resolved Hide resolved

mithril-aggregator/tests/test_extensions/runtime_tester.rs Outdated Show resolved Hide resolved

sfauvel and others added 11 commits October 21, 2024 18:25

Implementation of MetricService for the aggregator

6aa699d

Add MetricsService to the dependency_builder and start MetricsServer

9c70c72

test: rename metrics unit test

3ee52b7

Co-authored-by: Sébastien Fauvel <sfauvel@users.noreply.github.com>

feat: update metrics help text

20ed3a1

feat: compute metrics in HTTP server routes

2b5b4be

feat: compute metrics when certificates and artifacts are produced

627bc47

feat: compute runtime cycle metrics

35f12e6

feat: add metrics clap arguments and enable_metrics_server in the c…

0963008

…onfiguration

fix: send signal to shut down metrics server when another task in the…

3f47cc4

… JoinSet completes

docs: update aggregator node documentation with metrics server featur…

63f49fb

…e parameters

feat: deploy metrics endpoints for aggregator node in infra

88eb71e

dlachaume force-pushed the ensemble/1980/expose_prometheus_metrics_for_aggregator branch from eefc17b to 90d7374 Compare October 21, 2024 16:25

dlachaume temporarily deployed to testing-preview October 21, 2024 16:41 — with GitHub Actions Inactive

dlachaume temporarily deployed to testing-sanchonet October 21, 2024 16:41 — with GitHub Actions Inactive

dlachaume force-pushed the ensemble/1980/expose_prometheus_metrics_for_aggregator branch from 90d7374 to e35d60a Compare October 22, 2024 08:15

dlachaume and others added 5 commits October 22, 2024 10:37

test: specify behavior of increment_runtime_cycle functions in `Moc…

d627d39

…kAggregatorRunner` during state machine tests

feat: scrape the aggregator metrics endpoint in the VM prometheus end…

9dce660

…point

fix: computation of aggegator metrics endpoints in the VM prometheus …

18efa2d

…endpoint

fix: add missing configuration file for the VM prometheus endpoint

7baea0f

refactor: enhance calls to the metrics service in the aggregator runn…

c7525be

…er, fix typos and renaming

dlachaume and others added 5 commits October 22, 2024 10:37

fix: add component name as 'src' for MetricService logger

89a21b6

Co-authored-by: Sébastien Fauvel <sfauvel@users.noreply.github.com> Co-authored-by: DJO <Alenar@users.noreply.github.com>

fix: allow use of clap parameters that where missing

a05781c

chore: upgrade crate versions

77321b3

* mithril-metric from `0.1.0` to `0.1.1` * mithril-aggregator from `0.5.84` to `0.5.85`

chore: reference the feature in the CHANGELOG

7352335

dlachaume force-pushed the ensemble/1980/expose_prometheus_metrics_for_aggregator branch from e35d60a to 7352335 Compare October 22, 2024 08:37

dlachaume temporarily deployed to testing-preview October 22, 2024 08:45 — with GitHub Actions Inactive

dlachaume temporarily deployed to testing-sanchonet October 22, 2024 08:45 — with GitHub Actions Inactive

dlachaume merged commit ef8c283 into main Oct 22, 2024
45 checks passed

dlachaume deleted the ensemble/1980/expose_prometheus_metrics_for_aggregator branch October 22, 2024 08:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Activate metrics endpoint on aggregator #2022

Activate metrics endpoint on aggregator #2022

dlachaume commented Oct 18, 2024 •

edited

Loading

github-actions bot commented Oct 18, 2024 •

edited

Loading

jpraynaud left a comment •

edited

Loading

Alenar left a comment

Activate metrics endpoint on aggregator #2022

Activate metrics endpoint on aggregator #2022

Conversation

dlachaume commented Oct 18, 2024 • edited Loading

Content

ab tests on testing-preview BEFORE merging the PR

Number of requests: 1000

Number of requests: 10 000

ab tests on testing-preview AFTER merging the PR

Number of requests: 1000

Number of requests: 10 000

Pre-submit checklist

Issue(s)

github-actions bot commented Oct 18, 2024 • edited Loading

Test Results

jpraynaud left a comment • edited Loading

Choose a reason for hiding this comment

Alenar left a comment

Choose a reason for hiding this comment

dlachaume commented Oct 18, 2024 •

edited

Loading

`ab` tests on `testing-preview` BEFORE merging the PR

`ab` tests on `testing-preview` AFTER merging the PR

github-actions bot commented Oct 18, 2024 •

edited

Loading

jpraynaud left a comment •

edited

Loading