Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[docdb] Expose RPC queue sizes in master and tserver #2389

Closed
rao-vasireddy opened this issue Sep 20, 2019 · 1 comment
Closed

[docdb] Expose RPC queue sizes in master and tserver #2389

rao-vasireddy opened this issue Sep 20, 2019 · 1 comment
Assignees
@rajukumaryb
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature

Comments

@rao-vasireddy
Copy link
Contributor

Even if this data is not collected in normal run of the database, one should be able to enable this and collect the data for debugging.
@rao-vasireddy rao-vasireddy added kind/enhancement This is an enhancement of an existing feature area/docdb YugabyteDB core features labels Sep 20, 2019
@bmatican bmatican changed the title Provide a way to look at queue sizes in master and tserver for debugging purposes [docdb] Expose RPC queue sizes in master and tserver Oct 30, 2019
@bmatican bmatican added good first issue This is a good issue to start contributing! help wanted We welcome your contributions for this issue! labels Oct 30, 2019
@bmatican bmatican removed their assignment Oct 30, 2019
@bmatican bmatican removed good first issue This is a good issue to start contributing! help wanted We welcome your contributions for this issue! labels Nov 15, 2019
@bmatican
Copy link
Contributor

Passing this to @rajukumaryb as I think this would be useful to start plotting for back pressure!

Let's also, while putting this in, figure out which queue belongs to which service and how to expose stats per-queue, if possible! cc @spolitov

rajukumaryb added a commit that referenced this issue Dec 17, 2019
@rajukumaryb
#2389: Per service gauge counter for number of enqueued rpcs
d21147a 
Summary:
  Example Output: curl -s 127.0.0.1:9000/prometheus-metrics | grep rpcs_in_queue
			rpcs_in_queue_yb_tserver_TabletServerBackupService{exported_instance="DEFAULT_NODE_NAME",metric_id="yb.tabletserver",metric_type="server"} 0 1575514867781
			rpcs_in_queue_yb_tserver_TabletServerService{exported_instance="DEFAULT_NODE_NAME",metric_id="yb.tabletserver",metric_type="server"} 0 1575514867781
			rpcs_in_queue_yb_tserver_TabletServerAdminService{exported_instance="DEFAULT_NODE_NAME",metric_id="yb.tabletserver",metric_type="server"} 0 1575514867781
			rpcs_in_queue_yb_consensus_ConsensusService{exported_instance="DEFAULT_NODE_NAME",metric_id="yb.tabletserver",metric_type="server"} 1 1575514867781
			rpcs_in_queue_yb_tserver_RemoteBootstrapService{exported_instance="DEFAULT_NODE_NAME",metric_id="yb.tabletserver",metric_type="server"} 0 1575514867781
			rpcs_in_queue_yb_cdc_CDCService{exported_instance="DEFAULT_NODE_NAME",metric_id="yb.tabletserver",metric_type="server"} 0 1575514867781
			rpcs_in_queue_yb_server_GenericService{exported_instance="DEFAULT_NODE_NAME",metric_id="yb.tabletserver",metric_type="server"} 0 1575514867781

Test Plan: Jenkins

Reviewers: bogdan, mikhail, sergei

Reviewed By: sergei

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D7658
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rajukumaryb rajukumaryb

Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@bmatican @rao-vasireddy @rajukumaryb