Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

workload: change the way we issue SQL for TPCC [DNM] #31079

Closed
wants to merge 3 commits into from

Conversation

RaduBerinde
Copy link
Member

This PR changes the SQL issue method for the TPCC workload (for the queries that don't use variable-length tuples).

Before, we were putting the entire query in a single string (no parameters) and it was issued as a "simple query". After, we issue with parameters and support two methods. With --method=prepare, we prepare once per session and re-execute the same query. With --method=noprepare, we don't prepare explicitly, which internally results in each statement being prepared and executed every time.

I made some test runs and I'm baffled at the results (link below). I expected prepare to be the fastest but it is the slowest. I can't understand how it can be slower than noprepare.. These were interleaved runs on a single-node gceworker (which ran both the workload and the server). Maybe I'm missing some subtlety of lib/pq and Tx.Stmt? (CC @mjibson)

https://docs.google.com/spreadsheets/d/14TrUwte2QMN6L3y_H6RRtrYt1q9xJkm5yF5CNX9U3X0/edit?usp=sharing

workload: add SQLRunner

A common facility for issuing SQL queries. It supports multiple
issuing methods, speciifed the --method flag:

  • prepare: each statement is prepared once per session and then
    reused.
  • noprepare: each statement is executed directly (which internally
    is equivalent to preparing and executing each statement
    separately).

workload: refactor tpccTx

Refactoring tpccTx to allow per-worker state. This will be necessary
for preparing statements in advance.

workload: use SQLRunner for queries without tuples

Switch TPCC to use SQLRunner for queries that have a fixed number of
arguments.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

A common facility for issuing SQL queries. It supports multiple
issuing methods, speciifed the `--method` flag:
 - prepare: each statement is prepared once per session and then
   reused.
 - noprepare: each statement is executed directly (which internally
   is equivalent to preparing and executing each statement
   separately).

Release note: None
Refactoring tpccTx to allow per-worker state. This will be necessary
for preparing statements in advance.

Release note: None
Switch TPCC to use SQLRunner for queries that have a fixed number of
arguments.

Release note: None
@maddyblue
Copy link
Contributor

These results are also surprising to me. I'm not aware of anything in lib/pq or Tx.Stmt that would cause this (but it's been a while since I looked at those).

@RaduBerinde
Copy link
Member Author

I may be hitting the same problem as #30811. I will rework the code to prepare on separate connections and try again.

@maddyblue
Copy link
Contributor

Serious suggestion: if performance is what you want and you have a benchmark script handy, try porting to https://github.com/jackc/pgx and see if it's faster (and don't use it via database/sql).

@RaduBerinde
Copy link
Member Author

Closing for now; opened a new PR #31159 and will revisit the TPCC side of things separately.

@RaduBerinde RaduBerinde closed this Oct 9, 2018
@RaduBerinde
Copy link
Member Author

RaduBerinde commented Oct 9, 2018

@mjibson I plan to look at that at some point. For now I'm mainly interested in investigating the server-side code paths that these different ways of issuing requests take.

@RaduBerinde RaduBerinde deleted the tpcc-prepare branch October 18, 2018 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants