Skip to content
This repository was archived by the owner on Mar 24, 2025. It is now read-only.

feat: Add ABCIMethodLatency histogram [BLO-791] #26

Merged
merged 7 commits into from
Jan 23, 2024

Conversation

nivasan1
Copy link
Contributor

@nivasan1 nivasan1 commented Jan 22, 2024

In This PR

  • I introduce the ABCIMethodLatency histogram in service/metrics/metrics.go , see the RFC
  • I instrument Prepare/ProcessProposal
  • I add a README section for metrics, and a README for service/metrics
  • I fix the e2e tests to expose the app-metrics port
  • misc changes are documented (all bug-fixes)

Example output

Screen Shot 2024-01-22 at 3 19 42 PM

}),
Name: "abci_method_latency",
Help: "The time it took for an ABCI method to execute slinky specific logic (in seconds)",
Buckets: []float64{.0001, .0004, .002, .009, .02, .1, .65, 2, 6, 25},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buckets chosen are in seconds, these are the same buckets used by cometBFTs, abci_method_latency histogram

@nivasan1 nivasan1 force-pushed the nv/abci-method-latency branch from d628a89 to 0fc81c7 Compare January 22, 2024 23:24
@@ -77,7 +77,7 @@ func DefaultOracleConfig(node *cosmos.ChainNode) (

// Create the metrics config
metricsConfig := oracleconfig.MetricsConfig{
PrometheusServerAddress: fmt.Sprintf("%s:%s", oracle.HostName(), "8081"),
PrometheusServerAddress: fmt.Sprintf("%s:%s", "0.0.0.0", "8081"),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expose app-metrics to docker-host, as opposed to oracle-host in docker-network

@nivasan1 nivasan1 force-pushed the nv/abci-method-latency branch from 36e3ec6 to 68883c3 Compare January 23, 2024 00:09
@@ -1,2 +1 @@
go.work*
Copy link
Contributor Author

@nivasan1 nivasan1 Jan 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

un ignore go.sum so dockerfiles can build

Comment on lines 273 to 286
// record time spent in slinky-specific ProcesssProposal
processDuration := time.Since(start)

// call the wrapped process-proposal
resp, err := h.processProposalHandler(ctx, req)
if err == nil {
// record time spent in slinky-specific ProcesssProposal on successful process-proposals
h.logger.Info(
"recording process proposal time",
"duration (seconds)", processDuration.Seconds(),
)
h.metrics.ObserveABCIMethodLatency(servicemetrics.ProcessProposal, processDuration)
}
return resp, err
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I had metrics observation in a defer was to guarantee that we reported a metric no matter what our response was.

The way you have it above will only report in the happy path and not any other result. I think we want to report metrics no matter whether the response is Accept or Reject.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya, I intentionally wanted to only collect latencies for successful requests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'd prefer to report in all cases?

@nivasan1 nivasan1 force-pushed the nv/abci-method-latency branch from 68883c3 to 9294abc Compare January 23, 2024 00:14
@nivasan1 nivasan1 force-pushed the nv/abci-method-latency branch from 19c2e36 to 26cf8ac Compare January 23, 2024 00:38
Comment on lines -60 to +89
// ObserveProcessProposalTime records the time it took for the oracle-specific parts of process proposal
ObserveProcessProposalTime(duration time.Duration)

// ObservePrepareProposalTime records the time it took for the oracle-specific parts of prepare proposal
ObservePrepareProposalTime(duration time.Duration)
// ObserveABCIMethodLatency reports the given latency (as a duration), for the given ABCIMethod, and updates the ABCIMethodLatency histogram w/ that value.
ObserveABCIMethodLatency(method ABCIMethod, duration time.Duration)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 👍

@nivasan1 nivasan1 merged commit 2b0af7d into main Jan 23, 2024
@zrbecker zrbecker deleted the nv/abci-method-latency branch November 5, 2024 21:03
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants