Reduce some unnecessary prometheus metrics. #5006

mengxin9014 · 2022-05-26T09:55:45Z

What problem does this PR solve?

Issue Number: close #5080

Problem Summary:

What is changed and how it works?

reduce some unnecessary prometheus metrics.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

reduce some unnecessary prometheus metrics.

ti-chi-bot · 2022-05-26T09:55:47Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

JaySon-Huang
windtalker

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

dbms/src/Storages/MarkCache.h

dbms/src/Storages/BackgroundProcessingPool.cpp

dbms/src/Interpreters/AsynchronousMetrics.cpp

dbms/src/Common/LRUCache.h

mengxin9014 · 2022-06-07T09:31:45Z

/run-all-tests

sre-bot · 2022-06-07T10:04:03Z

Coverage for changed files

too many lines from llvm-cov, please refer to full report instead

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18277      9726             46.79%    204970  97453        52.45%

full coverage report (for internal network access only)

mengxin9014 · 2022-06-07T11:46:34Z

/run-all-tests

sre-bot · 2022-06-07T11:57:18Z

Coverage for changed files

too many lines from llvm-cov, please refer to full report instead

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18277      9726             46.79%    204970  97447        52.46%

full coverage report (for internal network access only)

windtalker

LGTM

mengxin9014 · 2022-06-20T06:28:54Z

/merge

ti-chi-bot · 2022-06-20T06:28:54Z

@mengxin9014: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2022-06-20T06:28:56Z

@mengxin9014: /merge in this pull request requires 2 approval(s).

In response to this:

/merge

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

sre-bot · 2022-06-20T07:05:00Z

Coverage for changed files

too many lines from llvm-cov, please refer to full report instead

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18331      9649             47.36%    205961  96534        53.13%

full coverage report (for internal network access only)

JaySon-Huang · 2022-06-20T07:47:28Z

dbms/src/Common/TiFlashMetrics.h

    M(tiflash_tmt_merge_count, "Total number of TMT engine merge", Counter)                                                               \
    M(tiflash_tmt_merge_duration_seconds, "Bucketed histogram of TMT engine merge duration", Histogram,                                   \
-        F(type_tmt_merge_duration, {{"type", "tmt_merge_duration"}}, ExpBuckets{0.0005, 2, 20}))                                          \
+        F(type_tmt_merge_duration, {{"type", "tmt_merge_duration"}}, ExpBuckets{0.001, 2, 20}))                                           \
    M(tiflash_tmt_write_parts_count, "Total number of TMT engine write parts", Counter)                                                   \
    M(tiflash_tmt_write_parts_duration_seconds, "Bucketed histogram of TMT engine write parts duration", Histogram,                       \
-        F(type_tmt_write_duration, {{"type", "tmt_write_parts_duration"}}, ExpBuckets{0.0005, 2, 20}))                                    \
+        F(type_tmt_write_duration, {{"type", "tmt_write_parts_duration"}}, ExpBuckets{0.001, 2, 20}))                                     \
    M(tiflash_tmt_read_parts_count, "Total number of TMT engine read parts", Gauge)                                                       \


We can remove these metrics of tmt engine

We can remove these metrics of tmt engine

done

JaySon-Huang · 2022-06-20T08:15:35Z

The panels shown in Grafana by "heatmap" are strange and misleading after we changed the bucket boundary for the following metrics:

tiflash_storage_page_gc_duration_seconds
tiflash_raft_apply_write_command_duration_seconds
tiflash_raft_write_data_to_storage_duration_seconds
tiflash_raft_upstream_latency

Maybe leave those metrics drawn by "heatmap" unchanged?

When the time range contains both before upgrade and after upgrade

When the time range contains only before upgrade

When the time range contains only after upgrade

mengxin9014 · 2022-06-20T08:42:42Z

The panels shown in Grafana by "heatmap" are strange and misleading after we changed the bucket boundary for the following metrics:

tiflash_storage_page_gc_duration_seconds

tiflash_raft_apply_write_command_duration_seconds

tiflash_raft_write_data_to_storage_duration_seconds

tiflash_raft_upstream_latency

Maybe leave those metrics drawn by "heatmap" unchanged?

When the time range contains both before upgrade and after upgrade

When the time range contains only before upgrade

When the time range contains only after upgrade

done.

mengxin9014 · 2022-06-20T08:45:30Z

/run-all-tests

JaySon-Huang

LGTM with minor comment

JaySon-Huang · 2022-06-20T08:50:02Z

dbms/src/Common/TiFlashMetrics.h

    M(tiflash_tmt_merge_count, "Total number of TMT engine merge", Counter)                                                               \
    M(tiflash_tmt_merge_duration_seconds, "Bucketed histogram of TMT engine merge duration", Histogram,                                   \
-        F(type_tmt_merge_duration, {{"type", "tmt_merge_duration"}}, ExpBuckets{0.0005, 2, 20}))                                          \
+        F(type_tmt_merge_duration, {{"type", "tmt_merge_duration"}}, ExpBuckets{0.001, 2, 20}))                                           \
    M(tiflash_tmt_write_parts_count, "Total number of TMT engine write parts", Counter)                                                   \
    M(tiflash_tmt_write_parts_duration_seconds, "Bucketed histogram of TMT engine write parts duration", Histogram,                       \
-        F(type_tmt_write_duration, {{"type", "tmt_write_parts_duration"}}, ExpBuckets{0.0005, 2, 20}))                                    \
-    M(tiflash_tmt_read_parts_count, "Total number of TMT engine read parts", Gauge)                                                       \
+        F(type_tmt_write_duration, {{"type", "tmt_write_parts_duration"}}, ExpBuckets{0.001, 2, 20}))                                     \


these metrics can also be removed

tiflash_tmt_merge_count

tiflash_tmt_merge_duration_seconds

tiflash_tmt_write_parts_count

tiflash_tmt_write_parts_duration_seconds
* tiflash_tmt_read_parts_count

mengxin9014 · 2022-06-20T09:05:33Z

/merge

ti-chi-bot · 2022-06-20T09:05:33Z

@mengxin9014: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2022-06-20T09:05:35Z

This pull request has been accepted and is ready to merge.

Commit hash: 3b46ee5

ti-chi-bot · 2022-06-20T09:20:46Z

@mengxin9014: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

sre-bot · 2022-06-20T09:56:01Z

Coverage for changed files

too many lines from llvm-cov, please refer to full report instead

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18335      9649             47.37%    206177  96568        53.16%

full coverage report (for internal network access only)

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2022-06-20T10:03:15Z

In response to a cherrypick label: new pull request created: #5173.

ti-chi-bot · 2022-06-20T10:03:52Z

In response to a cherrypick label: new pull request created: #5174.

delete useless metrics

7f3feaf

JaySon-Huang reviewed May 26, 2022

View reviewed changes

dbms/src/Storages/MarkCache.h Outdated Show resolved Hide resolved

dbms/src/Storages/BackgroundProcessingPool.cpp Outdated Show resolved Hide resolved

dbms/src/Interpreters/AsynchronousMetrics.cpp Outdated Show resolved Hide resolved

JaySon-Huang mentioned this pull request Jun 1, 2022

Show metrics about resident memory usage #5042

Closed

reduce some histogram bucket size

aa65360

JaySon-Huang reviewed Jun 6, 2022

View reviewed changes

dbms/src/Common/LRUCache.h Outdated Show resolved Hide resolved

mengxin9014 added 4 commits June 6, 2022 16:06

Merge branch 'master' into mx/deleteUselessMetrics

1162532

update metrics

1be260c

change exp bucket size to 20

ce5a902

change size

ef51f63

ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. labels Jun 7, 2022

mengxin9014 changed the title ~~WIP: delete useless metrics in TiFlash~~ Reduce some unnecessary prometheus metrics. Jun 7, 2022

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 7, 2022

mengxin9014 self-assigned this Jun 7, 2022

mengxin9014 requested a review from windtalker June 7, 2022 09:29

Merge branch 'master' into mx/deleteUselessMetrics

85084d0

ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 11, 2022

windtalker approved these changes Jun 20, 2022

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 20, 2022

Merge branch 'master' into mx/deleteUselessMetrics

1fb1d27

mengxin9014 added affects-6.0 affects-6.1 labels Jun 20, 2022

JaySon-Huang reviewed Jun 20, 2022

View reviewed changes

modify according to comments

b6fbd1c

ti-chi-bot added needs-cherry-pick-release-6.0 Type: Need cherry pick to release-6.0 needs-cherry-pick-release-6.1 Should cherry pick this PR to release-6.1 branch. labels Jun 20, 2022

JaySon-Huang approved these changes Jun 20, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 20, 2022

modify according to comments

3b46ee5

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 20, 2022

Merge branch 'master' into mx/deleteUselessMetrics

69a9453

ti-chi-bot merged commit 40baeca into pingcap:master Jun 20, 2022

ti-chi-bot pushed a commit to ti-chi-bot/tiflash that referenced this pull request Jun 20, 2022

This is an automated cherry-pick of pingcap#5006

f4c39a2

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Jun 20, 2022

Reduce some unnecessary prometheus metrics. (#5006) #5173

Closed

12 tasks

ti-chi-bot mentioned this pull request Jun 20, 2022

Reduce some unnecessary prometheus metrics. (#5006) #5174

Closed

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reduce some unnecessary prometheus metrics. #5006

Reduce some unnecessary prometheus metrics. #5006

mengxin9014 commented May 26, 2022 •

edited

Loading

ti-chi-bot commented May 26, 2022 •

edited

Loading

mengxin9014 commented Jun 7, 2022

sre-bot commented Jun 7, 2022

mengxin9014 commented Jun 7, 2022

sre-bot commented Jun 7, 2022

windtalker left a comment

mengxin9014 commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

sre-bot commented Jun 20, 2022

JaySon-Huang Jun 20, 2022

mengxin9014 Jun 20, 2022

JaySon-Huang commented Jun 20, 2022

mengxin9014 commented Jun 20, 2022

When the time range contains both before upgrade and after upgrade

When the time range contains only before upgrade

When the time range contains only after upgrade

mengxin9014 commented Jun 20, 2022

JaySon-Huang left a comment

JaySon-Huang Jun 20, 2022 •

edited

Loading

mengxin9014 commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

sre-bot commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

Reduce some unnecessary prometheus metrics. #5006

Reduce some unnecessary prometheus metrics. #5006

Conversation

mengxin9014 commented May 26, 2022 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented May 26, 2022 • edited Loading

mengxin9014 commented Jun 7, 2022

sre-bot commented Jun 7, 2022

Coverage for changed files

Coverage summary

mengxin9014 commented Jun 7, 2022

sre-bot commented Jun 7, 2022

Coverage for changed files

Coverage summary

windtalker left a comment

Choose a reason for hiding this comment

mengxin9014 commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

sre-bot commented Jun 20, 2022

Coverage for changed files

Coverage summary

JaySon-Huang Jun 20, 2022

Choose a reason for hiding this comment

mengxin9014 Jun 20, 2022

Choose a reason for hiding this comment

JaySon-Huang commented Jun 20, 2022

When the time range contains both before upgrade and after upgrade

When the time range contains only before upgrade

When the time range contains only after upgrade

mengxin9014 commented Jun 20, 2022

When the time range contains both before upgrade and after upgrade

When the time range contains only before upgrade

When the time range contains only after upgrade

mengxin9014 commented Jun 20, 2022

JaySon-Huang left a comment

Choose a reason for hiding this comment

JaySon-Huang Jun 20, 2022 • edited Loading

Choose a reason for hiding this comment

mengxin9014 commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

sre-bot commented Jun 20, 2022

Coverage for changed files

Coverage summary

ti-chi-bot commented Jun 20, 2022

ti-chi-bot commented Jun 20, 2022

mengxin9014 commented May 26, 2022 •

edited

Loading

ti-chi-bot commented May 26, 2022 •

edited

Loading

JaySon-Huang Jun 20, 2022 •

edited

Loading