Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

util/uuid: move to a non-deprecated randomness source #110597

Open
rafiss opened this issue Sep 13, 2023 · 5 comments
Open

util/uuid: move to a non-deprecated randomness source #110597

rafiss opened this issue Sep 13, 2023 · 5 comments
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team

Comments

@rafiss
Copy link
Collaborator

rafiss commented Sep 13, 2023

This usage of math/rand is deprecated as of go1.20:

// defaultRandReader is an io.Reader that calls through to "math/rand".Read
// which is safe for concurrent use.
type defaultRandReader struct{}
func (r defaultRandReader) Read(p []byte) (n int, err error) {
return rand.Read(p)
}

https://pkg.go.dev/math/rand#Read

Read generates len(p) random bytes from the default Source and writes them into p. It always returns len(p) and a nil error. Read, unlike the Rand.Read method, is safe for concurrent use.

Deprecated: For almost all use cases, crypto/rand.Read is more appropriate.

However, using crypto/rand has previously been shown to be a performance issue. See #30236 (comment) for that investigation.

For now, we will stick with the deprecated usage, but eventually we will have to update it.

Also note that go 1.21 (which we do not use as of 2023-09-13) includes this change, which improves the performance of math/rand: golang/go@0b9974d

We may want to use something similar to that, since UUID generation is something that can happen with high concurrency. Before the aforementioned go1.21 change, math/rand.Read was synchronized around a single mutex.

Jira issue: CRDB-31518

@rafiss rafiss added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-shared-systems Shared Systems Team labels Sep 13, 2023
@bdarnell
Copy link
Contributor

This is subtle and poorly documented. math/rand is not deprecated; only math/rand.Read is deprecated (and the main concern appears to be that it's too easy for goimports to pick the wrong package when it sees rand.Read()). Generating a pair of int64s from math/rand is not deprecated. Much more detail about the math/rand interfaces is available in golang/go#61716.

Also note that the uuid package already supports both crypto/rand and math/rand. uuid.MakeV4 uses crypto/rand, while uuid.FastMakeV4 uses math/rand.

@bdarnell
Copy link
Contributor

We may also want to reevaluate the use of crypto/rand. We saw the performance issue in 2018; but there have been improvements in the meantime.

@rafiss
Copy link
Collaborator Author

rafiss commented Sep 13, 2023

@srosenberg saw worse performance with crypto/rand (due to waiting for the entropy pool) in August 2023: https://cockroachlabs.slack.com/archives/C4X2J0RH6/p1692725321047249

@rafiss
Copy link
Collaborator Author

rafiss commented Sep 13, 2023

Also note that the uuid package already supports both crypto/rand and math/rand. uuid.MakeV4 uses crypto/rand, while uuid.FastMakeV4 uses math/rand.

I'm aware, but this issue is just about deciding what to do with the current math/rand usage.

Generating a pair of int64s from math/rand is not deprecated.

That's a good point, and it looks like we can update our code to generate ints instead of bytes, with a fair bit of refactoring. I can take that on for a Friday project, and if I have trouble, I'll leave this issue to KV as per https://cockroachlabs.slack.com/archives/C05B1BPMJLE/p1694633033861299?thread_ts=1694632185.354129&cid=C05B1BPMJLE

craig bot pushed a commit that referenced this issue Sep 13, 2023
110561: *: add back non-test usages of math/rand r=rafiss a=rafiss

78fee99 moved usages of math/rand to
crypto/rand. This has performance implications, as described in
#30236 (comment).

Rather than forcing a change, we mark the existing non-test usages of
math/rand as ignored by the linter so the appropriate engineering teams
can decide what to do.

informs #110597
informs #110599
Release note: None

110601:     changefeedccl: Fix setting propagation flake r=miretskiy a=miretskiy

Setting changes are asynchronous, particularly in tenants.
Set the feature setting directly instead.

Fixes #110300

Release note: None

Co-authored-by: Rafi Shamim <rafi@cockroachlabs.com>
Co-authored-by: Yevgeniy Miretskiy <yevgeniy@cockroachlabs.com>
@rafiss
Copy link
Collaborator Author

rafiss commented Sep 14, 2023

I tried implementing this by generating random uint64s, and there was a ~28% performance hit. See #110656

I am unassigning myself for now and leaving this with the KV team as per cockroachlabs.slack.com/archives/C05B1BPMJLE/p1694633033861299?thread_ts=1694632185.354129&cid=C05B1BPMJLE

There isn't any particular urgency to this - rand.Read will likely be around for a while.

@rafiss rafiss added T-kv KV Team and removed T-shared-systems Shared Systems Team labels Sep 14, 2023
craig bot pushed a commit that referenced this issue Sep 15, 2023
110670: uuid: fix Int63 call; rationalize naming and comments r=rafiss a=rafiss

### uuid: fix NewPopulatedUUID int generation

Using Int63 as before is a mistake, since that only generates positive
numbers, meaning the sign bit is never random.

### uuid: rationalize naming and comments

The word default was abused in this package to mean opposite things:
sometimes it meant crypto/rand and sometimes it meant math/rand.

Now this is improved and some comments are adjusted.

informs #110597
Release note: None

Co-authored-by: Rafi Shamim <rafi@cockroachlabs.com>
@github-project-automation github-project-automation bot moved this to Incoming in KV Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team
Projects
No open projects
Status: Incoming
Development

Successfully merging a pull request may close this issue.

2 participants