-
Notifications
You must be signed in to change notification settings - Fork 511
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
services/horizon: Add prometheus metrics to database queries #3597
Conversation
Just want to confirm: are we getting raw numbers from these metrics? It would be great to be able to derive e.g. max and min as well as average for these calls. |
Yeah they look good to me. |
Averages we'll get out of the box - in prometheus we'll be able to divide total of all queries by number of queries to get average. |
e878915
to
baffc85
Compare
baffc85
to
2dcfb5c
Compare
session, err := db.Open("postgres", databaseURL) | ||
if err != nil { | ||
log.Fatalf("cannot open Horizon DB: %v", err) | ||
} | ||
|
||
session.DB.SetMaxIdleConns(maxIdle) | ||
session.DB.SetMaxOpenConns(maxOpen) | ||
return session | ||
return db.RegisterMetrics(session, "horizon", subsystem, registry) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it changes metrics' names. Can we revert it? It will break Grafana dashboard.
PR Checklist
PR Structure
otherwise).
services/friendbot
, orall
ordoc
if the changes are broad or impact manypackages.
Thoroughness
.md
files, etc... affected by this change). Take a look in the
docs
folder for a given service,like this one.
Release planning
needed with deprecations, added features, breaking changes, and DB schema changes.
semver, or if it's mainly a patch change. The PR is targeted at the next
release branch if it's not a patch change.
What
Add prometheus metrics for query count and duration to horizon database calls. Turns out the metrics we had before were actually missing the ingest and core database clients as well, so I've added the same metrics to those too.
The big code changes here are:
db.SessionInterface
, instead of just using*db.Session
everywhere. Nice cleanup anyway.StateMiddleware
andHistoryMiddleware
into each route individually, as they have to happen after routing. Big OOF.Why
So we can verify and really understand which endpoints are causing db load and why.
Fixes #3593
Known limitations