Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor(rpc-server): Split error metrics to disambiguate the error reasons #98

Merged
merged 3 commits into from
Aug 10, 2023
Merged

refactor(rpc-server): Split error metrics to disambiguate the error reasons #98

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e0f82aa
refactor(rpc-server): Split error metrics to disambiguate the error r…
khorolets Aug 9, 2023
f5cedd0
fix(rpc-server): README metric error codes fix grammar
khorolets Aug 9, 2023
e0c1f9b
Update rpc-server/README.md
khorolets Aug 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ services:
<<: *common-variables
AWS_BUCKET_NAME: near-lake-data-mainnet
SERVER_PORT: 8080
RUST_LOG: "read_rpc_server=debug,is_data_consistency=debug"
RUST_LOG: "read_rpc_server=debug,shadow_data_consistency=debug"
restart: on-failure
ports:
- 8080:8080
Expand Down
1 change: 1 addition & 0 deletions rpc-server/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ lazy_static = "1.4.0"
lru = "0.8.1"
num-bigint = "0.3"
num-traits = "0.2.15"
paste = "1.0.14"
prometheus = "0.13.1"
scylla = "0.9.0"
serde = { version = "1.0.145", features = ["derive"] }
Expand Down
26 changes: 26 additions & 0 deletions rpc-server/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,3 +79,29 @@ This feature flag enables the shadow data consistency checks. With this feature
### `account_access_keys` (default: `false`)

We encountered a problem with the design of the table `account_access_keys` and overall design of the logic around it. We had to disable the table and proxy the calls of the `query.access_key_list` method to the real NEAR RPC. However, we still aren't ready to get rid of the code and that's why we hid it under the feature flag. We are planning to remove the code in the future and remove the feature flag.

## Metrics (Prometheus)

The read-rpc-server exposes Prometheus-compatible metrics at the `/metrics` endpoint.

All the metrics are defined in `src/metrics.rs` file. Two main categories of metrics are:

- Total number of requests for a specific method
- Number of errors for a specific method (works only if the `shadow_data_consistency` feature flag is enabled)

The latter is split by the suffix code to differentiate the reason for the error when possible:

- **0** - ReadRPC returns a Success result, and NEAR RPC returns a Success result, but the results don't match
- **1** - ReadRPC returns Success result, and NEAR RPC returns an Error result
- **2** - ReadRPC returns an Error result, and NEAR RPC returns Success result
- **3** - ReadRPC returns an Error result, and NEAR RPC returns an Error result, but the results don't match
- **4** - Could not perform consistency check because of the error (either network or parsing the results)

For example, method `block` will have these metrics:

- `BLOCK_REQUESTS_TOTAL`
- `BLOCK_ERROR_0`
- `BLOCK_ERROR_1`
- `BLOCK_ERROR_2`
- `BLOCK_ERROR_3`
- `BLOCK_ERROR_4`
Loading
Loading