From ec17181c6ac76369b5e65d6679aaf855fae08663 Mon Sep 17 00:00:00 2001 From: JaySon-Huang Date: Thu, 23 Sep 2021 18:23:32 +0800 Subject: [PATCH 1/5] Add desc of "Storage Write Stall" row in the TiFlash grafana Signed-off-by: JaySon-Huang --- tiflash/monitor-tiflash.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tiflash/monitor-tiflash.md b/tiflash/monitor-tiflash.md index 90d28c38238ed..88eb502f05c05 100644 --- a/tiflash/monitor-tiflash.md +++ b/tiflash/monitor-tiflash.md @@ -73,6 +73,13 @@ The following sections introduce the default monitoring information of **TiFlash > > These metrics only cover the monitoring information of the TiFlash storage layer and do not cover that in TiFlash-Proxy. +## Storage Write Stall + +- Write & Delta Management Throughput: The throughput of write and delta's background management for all instances. `throughput_write` means the throughput of data sync through Raft. `throughput_delta-management` means the throughput of data compaction. `total_write` means the total bytes written since last starts. `total_delta-management` means the total bytes of data compaction since last starts. +- Write Stall Duration: The stall duration of write and delete range. +- Write Throughput By Instance: The throughput of write by instance. It includes the throughput by applying the Raft write commands and Raft snapshots. +- Write Command OPS By Instance: The total count of different kinds of commands received by instance. `write block` means some data sync through Raft. `delete_range` means some Regions are removed from or move to this instance. `ingest` means some Region snapshot are applied to this instance. + ## Raft - Read Index OPS: The number of times that each TiFlash instance triggers the `read_index` request per second, which equals to the number of Regions triggered. From db2abe5d6f33e07278bc313f314151730fd7f0f3 Mon Sep 17 00:00:00 2001 From: JaySon-Huang Date: Thu, 23 Sep 2021 18:40:03 +0800 Subject: [PATCH 2/5] Fix some wording Signed-off-by: JaySon-Huang --- tiflash/monitor-tiflash.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tiflash/monitor-tiflash.md b/tiflash/monitor-tiflash.md index 88eb502f05c05..9370de3bb6a97 100644 --- a/tiflash/monitor-tiflash.md +++ b/tiflash/monitor-tiflash.md @@ -76,9 +76,9 @@ The following sections introduce the default monitoring information of **TiFlash ## Storage Write Stall - Write & Delta Management Throughput: The throughput of write and delta's background management for all instances. `throughput_write` means the throughput of data sync through Raft. `throughput_delta-management` means the throughput of data compaction. `total_write` means the total bytes written since last starts. `total_delta-management` means the total bytes of data compaction since last starts. -- Write Stall Duration: The stall duration of write and delete range. +- Write Stall Duration: The stall duration of write and removing Region data (delete range) by instance. - Write Throughput By Instance: The throughput of write by instance. It includes the throughput by applying the Raft write commands and Raft snapshots. -- Write Command OPS By Instance: The total count of different kinds of commands received by instance. `write block` means some data sync through Raft. `delete_range` means some Regions are removed from or move to this instance. `ingest` means some Region snapshot are applied to this instance. +- Write Command OPS By Instance: The total count of different kinds of commands received by instance. `write block` means the data logs synced through Raft. `delete_range` means some Regions are removed from or move to this instance. `ingest` means some Region snapshot are applied to this instance. ## Raft From 101a7dbd71902f1e9e9d570e67737928ac877e16 Mon Sep 17 00:00:00 2001 From: JaySon-Huang Date: Fri, 24 Sep 2021 16:57:32 +0800 Subject: [PATCH 3/5] Align with the Chinese version Signed-off-by: JaySon-Huang --- tiflash/monitor-tiflash.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/tiflash/monitor-tiflash.md b/tiflash/monitor-tiflash.md index 9370de3bb6a97..069c415023a30 100644 --- a/tiflash/monitor-tiflash.md +++ b/tiflash/monitor-tiflash.md @@ -75,10 +75,17 @@ The following sections introduce the default monitoring information of **TiFlash ## Storage Write Stall -- Write & Delta Management Throughput: The throughput of write and delta's background management for all instances. `throughput_write` means the throughput of data sync through Raft. `throughput_delta-management` means the throughput of data compaction. `total_write` means the total bytes written since last starts. `total_delta-management` means the total bytes of data compaction since last starts. +- Write & Delta Management Throughput: The throughput of write and delta's background management for all instances. + - `throughput_write` means the throughput of data sync through Raft. + - `throughput_delta-management` means the throughput of data compaction. + - `total_write` means the total bytes written since last starts. + - `total_delta-management` means the total bytes of data compaction since last starts. - Write Stall Duration: The stall duration of write and removing Region data (delete range) by instance. - Write Throughput By Instance: The throughput of write by instance. It includes the throughput by applying the Raft write commands and Raft snapshots. -- Write Command OPS By Instance: The total count of different kinds of commands received by instance. `write block` means the data logs synced through Raft. `delete_range` means some Regions are removed from or move to this instance. `ingest` means some Region snapshot are applied to this instance. +- Write Command OPS By Instance: The total count of different kinds of commands received by instance. + - `write block` means the data logs synced through Raft. + - `delete_range` means some Regions are removed from or move to this instance. + - `ingest` means some Region snapshot are applied to this instance. ## Raft From 3b6c14a8b6df06c25c52e8bd0d10329a7772819e Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Sun, 26 Sep 2021 15:52:52 +0800 Subject: [PATCH 4/5] Apply suggestions from code review --- tiflash/monitor-tiflash.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tiflash/monitor-tiflash.md b/tiflash/monitor-tiflash.md index 069c415023a30..f09f034c16c44 100644 --- a/tiflash/monitor-tiflash.md +++ b/tiflash/monitor-tiflash.md @@ -76,16 +76,16 @@ The following sections introduce the default monitoring information of **TiFlash ## Storage Write Stall - Write & Delta Management Throughput: The throughput of write and delta's background management for all instances. - - `throughput_write` means the throughput of data sync through Raft. + - `throughput_write` means the throughput of data synchronization through Raft. - `throughput_delta-management` means the throughput of data compaction. - - `total_write` means the total bytes written since last starts. - - `total_delta-management` means the total bytes of data compaction since last starts. + - `total_write` means the total bytes written since the last start. + - `total_delta-management` means the total bytes of data compaction since the last start. - Write Stall Duration: The stall duration of write and removing Region data (delete range) by instance. - Write Throughput By Instance: The throughput of write by instance. It includes the throughput by applying the Raft write commands and Raft snapshots. - Write Command OPS By Instance: The total count of different kinds of commands received by instance. - - `write block` means the data logs synced through Raft. - - `delete_range` means some Regions are removed from or move to this instance. - - `ingest` means some Region snapshot are applied to this instance. + - `write block` means the data logs synchronized through Raft. + - `delete_range` means some Regions are removed from or moved to this instance. + - `ingest` means some Region snapshots are applied to this instance. ## Raft From e9b02000f1eebaa4efa4676427cbcc5b45ba501e Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Sun, 26 Sep 2021 15:56:48 +0800 Subject: [PATCH 5/5] Update monitor-tiflash.md --- tiflash/monitor-tiflash.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tiflash/monitor-tiflash.md b/tiflash/monitor-tiflash.md index f09f034c16c44..bdf3c0d1f51fb 100644 --- a/tiflash/monitor-tiflash.md +++ b/tiflash/monitor-tiflash.md @@ -75,16 +75,16 @@ The following sections introduce the default monitoring information of **TiFlash ## Storage Write Stall -- Write & Delta Management Throughput: The throughput of write and delta's background management for all instances. +- Write & Delta Management Throughput: The throughput of write and data compaction for all instances. - `throughput_write` means the throughput of data synchronization through Raft. - `throughput_delta-management` means the throughput of data compaction. - `total_write` means the total bytes written since the last start. - - `total_delta-management` means the total bytes of data compaction since the last start. -- Write Stall Duration: The stall duration of write and removing Region data (delete range) by instance. + - `total_delta-management` means the total bytes of data compacted since the last start. +- Write Stall Duration: The stall duration of write and removing Region data (deleting ranges) by instance. - Write Throughput By Instance: The throughput of write by instance. It includes the throughput by applying the Raft write commands and Raft snapshots. - Write Command OPS By Instance: The total count of different kinds of commands received by instance. - `write block` means the data logs synchronized through Raft. - - `delete_range` means some Regions are removed from or moved to this instance. + - `delete_range` means that some Regions are removed from or moved to this instance. - `ingest` means some Region snapshots are applied to this instance. ## Raft