Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

workload: teach interleavedpartitioned about retryable transactions #30944

Merged
merged 1 commit into from
Oct 4, 2018

Conversation

BramGruneir
Copy link
Member

Also some other speedups and cleanups while I was in there.

Fixes #28567.

Release note: None

Also some other speedups and cleanups while I was in there.

Fixes cockroachdb#28567.

Release note: None
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@BramGruneir
Copy link
Member Author

I'll run this few a few roachtests tomorrow to ensure it does fix it.

Copy link
Member

@tbg tbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @BramGruneir!

Copy link
Collaborator

@petermattis petermattis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale)


pkg/workload/interleavedpartitioned/interleavedpartitioned.go, line 413 at r1 (raw file):

		if opRand < w.insertPercent {
			start := timeutil.Now()
			insertStatement, err := db.Prepare(insertQuery)

Are these database-level prepared statements a speedup? @nvanbenschoten discovered recently that there is an internal mutex (in database/sql) for database-level prepared statements that can become a contention point.

Copy link
Member Author

@BramGruneir BramGruneir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained


pkg/workload/interleavedpartitioned/interleavedpartitioned.go, line 413 at r1 (raw file):

Previously, petermattis (Peter Mattis) wrote…

Are these database-level prepared statements a speedup? @nvanbenschoten discovered recently that there is an internal mutex (in database/sql) for database-level prepared statements that can become a contention point.

I don't know if they're a speedup. The original goal of this workload was to emulate the customer's workload as closely as possible. So they do indeed prepare every statement. I'll do a run without them to see if they make a significant difference. But I'm going to merge this change first to get the test passing again.

Copy link
Member Author

@BramGruneir BramGruneir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTRs!

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained

@BramGruneir
Copy link
Member Author

bors r+

craig bot pushed a commit that referenced this pull request Oct 4, 2018
30482: workload/kv: print the highest sequence number r=andreimatei a=andreimatei

At the end of the kv workload, print the sequnence number of the highest
row written so that it can be used in a next run: for example, you might
want to prepopulate the db with --read_percent=0 and then perform only
reads with --read_percent=100.
If you don't pass --write-seq (or you don't know what to put in it), all
the reads from --read_percent=100 would try to read a non-exitent row
with sentinel sequence number 0.

Release note: None

30882: workload: actually prepare writeStmt with Conn r=nvanbenschoten a=nvanbenschoten

See #30811 (comment).

Release note: None

30944: workload: teach interleavedpartitioned about retryable transactions r=BramGruneir a=BramGruneir

Also some other speedups and cleanups while I was in there.

Fixes #28567.

Release note: None

Co-authored-by: Andrei Matei <andrei@cockroachlabs.com>
Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Co-authored-by: Bram Gruneir <bram@cockroachlabs.com>
@craig
Copy link
Contributor

craig bot commented Oct 4, 2018

Build succeeded

@craig craig bot merged commit 4a4113a into cockroachdb:master Oct 4, 2018
@BramGruneir
Copy link
Member Author

@petermattis

To answer your question as to if the prepare statements speed things up, in this case, they clearly do:

east no prepare
_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  600.0s        0           2455            4.1    978.8    939.5   1409.3   1610.6   3892.3  insert
  600.0s        0            329            0.5   1927.0    318.8   2080.4 103079.2 103079.2  retrieve
  600.0s        0            285            0.5   2013.0    268.4   1946.2 103079.2 103079.2  updates
east prepare
_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  600.0s        0           2937            4.9   1093.1   1040.2   1543.5   1879.0   5905.6  insert
  600.0s        0            380            0.6   1912.4    209.7   2080.4  73014.4 103079.2  retrieve
  600.0s        0            330            0.5   2532.6    226.5   2952.8  90194.3 103079.2  updates

west no prepare
_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  600.0s        0           1369            2.3   1858.9   1879.0   2281.7   2415.9   3355.4  insert
  600.0s        0            153            0.3   1098.7    771.8   2281.7   2952.8   4563.4  retrieve
  600.0s        0            177            0.3   1282.9    872.4   2684.4   3758.1   6979.3  updates
west prepare
_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  600.0s        0           1762            2.9   1843.9   1879.0   2281.7   2550.1   3355.4  insert
  600.0s        0            223            0.4   3058.6    771.8   2550.1 103079.2 103079.2  retrieve
  600.0s        0            215            0.4   2677.8    872.4   8321.5  27917.3 103079.2  updates

central no
_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  600.0s        0           2204            3.7   1146.1   1476.4   2684.4   2952.8  38654.7  delete
central yes
 _elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  600.0s        0           3422            5.7    849.7     37.7   2684.4   3221.2 103079.2  delete

@petermattis
Copy link
Collaborator

Ack. Thanks for checking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants