New table partitions code doesn't appear to be concurrent safe.

[joel-rieke]

Please see below for the concurrency issue.

This exception happened intermittently while performing a combinations of SELECT/INSERT/UPDATES on a single table with many rows...

Looking at the code however, it is clear to me why this happened.

I tried fixing this by applying a table level lock around the partition access, but this seems heavy handed. I think there could be a more elegant approach to solving this.

============================

"concurrent map read and map write"
Stack:
2 0x000000000117e905 in github.com/dolthub/go-mysql-server/memory.partitionssort.Less
at /go/pkg/mod/github.com/trimble-oss/go-mysql-server@v0.12.0-1.4/memory/table.go:1523
3 0x000000000118fc3c in github.com/dolthub/go-mysql-server/memory.(*partitionssort).Less
at :1
4 0x00000000004d4318 in sort.partialInsertionSort
at /usr/local/go/src/sort/zsortinterface.go:202
5 0x00000000004d3b29 in sort.pdqsort
at /usr/local/go/src/sort/zsortinterface.go:101
6 0x00000000004d070e in sort.Sort
at /usr/local/go/src/sort/sort.go:48
7 0x000000000117e23c in github.com/dolthub/go-mysql-server/memory.(*Table).sortRows
at /go/pkg/mod/github.com/trimble-oss/go-mysql-server@v0.12.0-1.4/memory/table.go:1502
8 0x00000000011859d1 in github.com/dolthub/go-mysql-server/memory.(*pkTableEditAccumulator).ApplyEdits
at /go/pkg/mod/github.com/trimble-oss/go-mysql-server@v0.12.0-1.4/memory/table_editor.go:460
9 0x00000000011819e2 in github.com/dolthub/go-mysql-server/memory.(*tableEditor).StatementComplete
at /go/pkg/mod/github.com/trimble-oss/go-mysql-server@v0.12.0-1.4/memory/table_editor.go:101
10 0x0000000001181493 in github.com/dolthub/go-mysql-server/memory.(*tableEditor).Close
at /go/pkg/mod/github.com/trimble-oss/go-mysql-server@v0.12.0-1.4/memory/table_editor.go:76
11 0x00000000013cf20b in github.com/dolthub/go-mysql-server/sql/plan.(*insertIter).Close
at /go/pkg/mod/github.com/trimble-oss/go-mysql-server@v0.12.0-1.4/sql/plan/insert.go:508
12 0x000000000143d28a in github.com/dolthub/go-mysql-server/sql/plan.checkpointingTableEditorIter.Close
at /go/pkg/mod/github.com/trimble-oss/go-mysql-server@v0.12.0-1.4/sql/plan/table_editor.go:118
13 0x000000000148e99d in github.com/dolthub/go-mysql-server/sql/plan.(*checkpointingTableEditorIter).Close
at :1
14 0x0000000001409f4e in github.com/dolthub/go-mysql-server/sql/plan.(*accumulatorIter).Next.func2
at /go/pkg/mod/github.com/trimble-oss/go-mysql-server@v0.12.0-1.4/sql/plan/row_update_accumulator.go:322
16 0x0000000001409bc6 in github.com/dolthub/go-mysql-server/sql/plan.(*accumulatorIter).Next
at /go/pkg/mod/github.com/trimble-oss/go-mysql-server@v0.12.0-1.4/sql/plan/row_update_accumulator.go:360
17 0x00000000014437c8 in github.com/dolthub/go-mysql-server/sql/plan.transactionCommittingIter.Next
at /go/pkg/mod/github.com/trimble-oss/go-mysql-server@v0.12.0-1.4/sql/plan/transaction_committing_iter.go:106
18 0x000000000148fa0d in github.com/dolthub/go-mysql-server/sql/plan.(*transactionCommittingIter).Next
at :1
19 0x00000000013f3502 in github.com/dolthub/go-mysql-server/sql/plan.(*trackedRowIter).Next
at /go/pkg/mod/github.com/trimble-oss/go-mysql-server@v0.12.0-1.4/sql/plan/process.go:381

[timsehn]

Thanks for the bug report!

You mind adding the SQL you are running concurrently along with the stack trace?

[joel-rieke]

Classic insert, but many simultaneous inserts:
Something like:
INSERT INTO tableA (columnP, columnQ, columnR, columnS) VALUES (?, ?, ?, ?)

tablename and column names and counts changed obviously.

[timsehn]

@zachmu Can you take a look here?

[zachmu]

Hi @joel-rieke,

You filed this against the dolt repo, but unless I'm mistaken you're using the built-in in-memory database implementation that ships with go-mysql-server, right?

Unfortunately that implementation isn't thread safe, as you're discovering. It doesn't implement transaction semantics, and it's not safe to do parallel write operations on it.

The fix we want to do eventually is implement a binlog like MySQL uses, but that's probably a long way off. A shorter term solution would be for us to build simple mutex locks for all table edit operations in the in-memory implementation.

In the meantime, do you mind if we move this issue over the go-mysql-server repository, since it's not a dolt bug?

Zach

[joel-rieke]

@zachmu , Doh! Uh yes, please do! I meant to put it there!

[joel-rieke]

Moving this over: dolthub/go-mysql-server#1306