add explicit request source type to label the external request like lightning/br #868

nolouch · 2023-07-04T07:07:52Z

if use

set @tidb_request_source_type='lightling'
select count(*) from t;

the request source field will like:

external_Select_lightning

util/request_source.go

…ightning/br Signed-off-by: nolouch <nolouch@gmail.com>

util/request_source.go

Signed-off-by: nolouch <nolouch@gmail.com>

util/request_source.go

util/request_source_test.go

Signed-off-by: nolouch <nolouch@gmail.com>

…urce

nolouch · 2023-07-06T04:51:05Z

ptal @HuSharp @glorv

glorv · 2023-07-06T06:17:25Z

util/request_source.go

 	}
-	return ExternalRequest + r.RequestSourceType
+	labels := []string{ExternalRequest, r.RequestSourceType, r.ExplicitRequestSourceType}


Currently, tikv uses the last segments of resquest_source.split("_") as the explicit_request_source(task_names) value, so if it is empty, the logic won't be correct. Because there are some request_source_types such as ddl_refill, we can't depend on the number of segments either. So I suggest always make ExplicitRequestSourceType not empty.

changed,ptal @glorv

Signed-off-by: nolouch <nolouch@gmail.com>

glorv

LGTM

glorv · 2023-07-06T06:51:17Z

util/request_source.go

+	}
+	explicitSourceType := r.ExplicitRequestSourceType
+	if len(explicitSourceType) == 0 {
+		explicitSourceType = ExplicitTypeDB


NIT: not sure if db is a good name for the default value. Maybe default is more conventional?

HuSharp

LGTM!

Signed-off-by: nolouch <nolouch@gmail.com>

util/request_source.go

Signed-off-by: nolouch <nolouch@gmail.com>

overvenus · 2023-07-07T06:27:46Z

Some tests fail.

HuSharp · 2023-07-07T06:54:59Z

@overvenus Can you leave a merge label? thx!

overvenus · 2023-07-07T07:01:37Z

/merge

…ightning/br (tikv#868) Signed-off-by: nolouch <nolouch@gmail.com>

* client-go: add some key range info to error when PD returned no region (#862) Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * *: refine non-global stale-read request retry logic (#863) Signed-off-by: crazycs520 <crazycs520@gmail.com> * Fix the issue that primary pessimistic lock may be left not cleared after GC (#866) * Fix the issue that primary pessimistic lock may be left not cleared after GC Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> * Fix mysteriously shown up thing that makes compilation failed Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> * Fix test effectiveness (forgot to set txn2 to pessimistic txn); add more strict checks Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> * Address comments Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> --------- Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com> * add explicit request source type to label the external request like lightning/br (#868) Signed-off-by: nolouch <nolouch@gmail.com> * use '%d' instead of '%q' for some int values in error message (#875) Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * format key in error message in method `scanRegions` (#876) Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * make cop request timeout a config paramter (#865) * update Signed-off-by: Spade A <u6748471@anu.edu.au> * update Signed-off-by: Spade A <u6748471@anu.edu.au> * update Signed-off-by: Spade A <u6748471@anu.edu.au> * update Signed-off-by: Spade A <u6748471@anu.edu.au> --------- Signed-off-by: Spade A <u6748471@anu.edu.au> * region_cache: support check pending tiflash peer (#821) Signed-off-by: guo-shaoge <shaoge1994@163.com> Co-authored-by: disksing <i@disksing.com> * *: add `SnapshotIterReverse` and make `iterReverse` supports `lowerBound` (#883) Signed-off-by: Jason Mo <mohangjie1995@gmail.com> * *: fix stale read ops metric (#878) (#889) Signed-off-by: crazycs520 <crazycs520@gmail.com> Co-authored-by: disksing <i@disksing.com> * add gc options (#828) Signed-off-by: weedge <weege007@gmail.com> Co-authored-by: disksing <i@disksing.com> * reload region cache when store is resolved from invalid status (#843) Signed-off-by: you06 <you1474600@gmail.com> Co-authored-by: disksing <i@disksing.com> * ci: update setup-go action (#904) Signed-off-by: disksing <i@disksing.com> * fix unexpected slow query during GC running after stop 1 tikv-server (#899) (#909) * fix unexpected slow query during GC running after stop 1 tikv-server Signed-off-by: crazycs520 <crazycs520@gmail.com> * fix test Signed-off-by: crazycs520 <crazycs520@gmail.com> --------- Signed-off-by: crazycs520 <crazycs520@gmail.com> * resource_manager: ignore ru metrics for background request (#872) Signed-off-by: husharp <jinhao.hu@pingcap.com> Co-authored-by: disksing <i@disksing.com> * add more log for diagnose (#915) * add more log for diagnose Signed-off-by: crazycs520 <crazycs520@gmail.com> * fix Signed-off-by: crazycs520 <crazycs520@gmail.com> * add more log for diagnose Signed-off-by: crazycs520 <crazycs520@gmail.com> * add more log Signed-off-by: crazycs520 <crazycs520@gmail.com> * address comment Signed-off-by: crazycs520 <crazycs520@gmail.com> --------- Signed-off-by: crazycs520 <crazycs520@gmail.com> * use context logger as much as possible (#908) * use context logger as much as possible Signed-off-by: crazycs520 <crazycs520@gmail.com> * refine Signed-off-by: crazycs520 <crazycs520@gmail.com> --------- Signed-off-by: crazycs520 <crazycs520@gmail.com> * Resume max retry time check for stale read retry with leader option(#903) (#911) * Resume max retry time check for stale read retry with leader option Signed-off-by: cfzjywxk <lsswxrxr@163.com> * add cancel Signed-off-by: cfzjywxk <lsswxrxr@163.com> --------- Signed-off-by: cfzjywxk <lsswxrxr@163.com> * request_source: remove default label (#890) * request_source: remove default label Signed-off-by: nolouch <nolouch@gmail.com> * add a function to set request source task type (#925) * add a function to set request source task type Signed-off-by: glorv <glorvs@163.com> * ci: update go version (#936) * ci: update go version Signed-off-by: crazycs520 <crazycs520@gmail.com> * fix test Signed-off-by: crazycs520 <crazycs520@gmail.com> --------- Signed-off-by: crazycs520 <crazycs520@gmail.com> * use tidb_kv_read_timeout as first kv request timeout (#919) * support tidb_kv_read_timeout as first round kv request timeout Signed-off-by: crazycs520 <crazycs520@gmail.com> * fix ci Signed-off-by: crazycs520 <crazycs520@gmail.com> * fix ci Signed-off-by: crazycs520 <crazycs520@gmail.com> * fix ci Signed-off-by: crazycs520 <crazycs520@gmail.com> * fix ci Signed-off-by: crazycs520 <crazycs520@gmail.com> * fix ci Signed-off-by: crazycs520 <crazycs520@gmail.com> * update comment Signed-off-by: crazycs520 <crazycs520@gmail.com> * refine test Signed-off-by: crazycs520 <crazycs520@gmail.com> --------- Signed-off-by: crazycs520 <crazycs520@gmail.com> * [pick] resource_control: bypass some internal urgent request (#938) * resource_control: bypass some internal urgent request (#884) Signed-off-by: nolouch <nolouch@gmail.com> * resourcecontrol: fix nil pointer (#900) Signed-off-by: nolouch <nolouch@gmail.com> --------- Signed-off-by: nolouch <nolouch@gmail.com> --------- Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> Signed-off-by: crazycs520 <crazycs520@gmail.com> Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> Signed-off-by: nolouch <nolouch@gmail.com> Signed-off-by: Spade A <u6748471@anu.edu.au> Signed-off-by: guo-shaoge <shaoge1994@163.com> Signed-off-by: Jason Mo <mohangjie1995@gmail.com> Signed-off-by: weedge <weege007@gmail.com> Signed-off-by: you06 <you1474600@gmail.com> Signed-off-by: disksing <i@disksing.com> Signed-off-by: husharp <jinhao.hu@pingcap.com> Signed-off-by: cfzjywxk <lsswxrxr@163.com> Signed-off-by: glorv <glorvs@163.com> Signed-off-by: iosmanthus <myosmanthustree@gmail.com> Co-authored-by: 王超 <cclcwangchao@hotmail.com> Co-authored-by: crazycs <crazycs520@gmail.com> Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com> Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com> Co-authored-by: ShuNing <nolouch@gmail.com> Co-authored-by: Spade A <71589810+SpadeA-Tang@users.noreply.github.com> Co-authored-by: guo-shaoge <shaoge1994@163.com> Co-authored-by: disksing <i@disksing.com> Co-authored-by: Hangjie Mo <mohangjie1995@gmail.com> Co-authored-by: weedge <weege007@gmail.com> Co-authored-by: you06 <you1474600@gmail.com> Co-authored-by: Hu# <jinhao.hu@pingcap.com> Co-authored-by: cfzjywxk <lsswxrxr@163.com> Co-authored-by: glorv <glorvs@163.com>

HuSharp mentioned this pull request Jul 4, 2023

lightning: mark background and resource group name when in local mode pingcap/tidb#44950

Merged

1 task

HuSharp reviewed Jul 5, 2023

View reviewed changes

util/request_source.go Outdated Show resolved Hide resolved

HuSharp reviewed Jul 5, 2023

View reviewed changes

util/request_source.go Outdated Show resolved Hide resolved

nolouch force-pushed the explicit-request-source branch from 16dabbc to e7c5520 Compare July 5, 2023 07:42

add explicit request source type to label the external request like l…

b87ad72

…ightning/br Signed-off-by: nolouch <nolouch@gmail.com>

nolouch force-pushed the explicit-request-source branch 2 times, most recently from 7e79174 to 157d072 Compare July 5, 2023 07:51

HuSharp reviewed Jul 5, 2023

View reviewed changes

util/request_source.go Outdated Show resolved Hide resolved

address

2596ac9

Signed-off-by: nolouch <nolouch@gmail.com>

nolouch force-pushed the explicit-request-source branch 2 times, most recently from 49f8e28 to 0f935d8 Compare July 5, 2023 09:52

address

b2b79e4

Signed-off-by: nolouch <nolouch@gmail.com>

nolouch force-pushed the explicit-request-source branch from 0f935d8 to b2b79e4 Compare July 5, 2023 09:54

HuSharp reviewed Jul 5, 2023

View reviewed changes

util/request_source.go Outdated Show resolved Hide resolved

nolouch force-pushed the explicit-request-source branch from 245b9e2 to 69d736f Compare July 6, 2023 02:55

HuSharp reviewed Jul 6, 2023

View reviewed changes

util/request_source_test.go Outdated Show resolved Hide resolved

nolouch force-pushed the explicit-request-source branch from 69d736f to b2b79e4 Compare July 6, 2023 03:11

nolouch added 2 commits July 6, 2023 11:13

address

1c12e80

Signed-off-by: nolouch <nolouch@gmail.com>

Merge remote-tracking branch 'origin/master' into explicit-request-so…

6992649

…urce

glorv reviewed Jul 6, 2023

View reviewed changes

address comments

ba1066b

Signed-off-by: nolouch <nolouch@gmail.com>

glorv approved these changes Jul 6, 2023

View reviewed changes

glorv mentioned this pull request Jul 6, 2023

*: label external the request source via sessionvar pingcap/tidb#44770

Merged

4 tasks

HuSharp approved these changes Jul 6, 2023

View reviewed changes

use default

bedfada

Signed-off-by: nolouch <nolouch@gmail.com>

nolouch force-pushed the explicit-request-source branch from c3bf88b to bedfada Compare July 6, 2023 08:56

HuSharp reviewed Jul 6, 2023

View reviewed changes

util/request_source.go Outdated Show resolved Hide resolved

HuSharp reviewed Jul 6, 2023

View reviewed changes

util/request_source.go Outdated Show resolved Hide resolved

address comments

3d194d0

Signed-off-by: nolouch <nolouch@gmail.com>

nolouch force-pushed the explicit-request-source branch 2 times, most recently from 558a4af to 144302b Compare July 7, 2023 05:49

overvenus approved these changes Jul 7, 2023

View reviewed changes

nolouch force-pushed the explicit-request-source branch from 144302b to 3d194d0 Compare July 7, 2023 06:34

overvenus merged commit 178f6fa into tikv:master Jul 7, 2023
17 of 18 checks passed

nolouch deleted the explicit-request-source branch July 7, 2023 07:37

nolouch added a commit to nolouch/client-go that referenced this pull request Jul 14, 2023

add explicit request source type to label the external request like l…

7b5da57

…ightning/br (tikv#868) Signed-off-by: nolouch <nolouch@gmail.com>

nolouch mentioned this pull request Jul 14, 2023

resource_control: bypass some internal urgent request #884

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add explicit request source type to label the external request like lightning/br #868

add explicit request source type to label the external request like lightning/br #868

nolouch commented Jul 4, 2023

nolouch commented Jul 6, 2023

glorv Jul 6, 2023

nolouch Jul 6, 2023

glorv left a comment

glorv Jul 6, 2023

HuSharp left a comment

overvenus commented Jul 7, 2023 •

edited

Loading

HuSharp commented Jul 7, 2023

overvenus commented Jul 7, 2023

add explicit request source type to label the external request like lightning/br #868

add explicit request source type to label the external request like lightning/br #868

Conversation

nolouch commented Jul 4, 2023

nolouch commented Jul 6, 2023

glorv Jul 6, 2023

Choose a reason for hiding this comment

nolouch Jul 6, 2023

Choose a reason for hiding this comment

glorv left a comment

Choose a reason for hiding this comment

glorv Jul 6, 2023

Choose a reason for hiding this comment

HuSharp left a comment

Choose a reason for hiding this comment

overvenus commented Jul 7, 2023 • edited Loading

HuSharp commented Jul 7, 2023

overvenus commented Jul 7, 2023

overvenus commented Jul 7, 2023 •

edited

Loading