Fix rocsp unit test flakes

[aarongable]

D161252 rocsp-tool.test 88b-5wc SQL:  insert into `certificateStatus` (`id`,`serial`,`status`,`ocspLastUpdated`,`revokedDate`,`revokedReason`,`lastExpirationNagSent`,`ocspResponse`,`notAfter`,`isExpired`,`IssuerID`) values (null,?,?,?,?,?,?,?,?,?,?); [1:"1337" 2:"" 3:0001-01-01 00:00:00 +0000 UTC 4:0001-01-01 00:00:00 +0000 UTC 5:0 6:0001-01-01 00:00:00 +0000 UTC 7:[] 8:1970-01-01 12:00:00 +0000 UTC 9:false 10:0] (1.433177ms)
D161252 rocsp-tool.test qYyKowg SQL:  insert into `certificateStatus` (`id`,`serial`,`status`,`ocspLastUpdated`,`revokedDate`,`revokedReason`,`lastExpirationNagSent`,`ocspResponse`,`notAfter`,`isExpired`,`IssuerID`) values (null,?,?,?,?,?,?,?,?,?,?); [1:"1338" 2:"" 3:0001-01-01 00:00:00 +0000 UTC 4:0001-01-01 00:00:00 +0000 UTC 5:0 6:0001-01-01 00:00:00 +0000 UTC 7:[] 8:1970-01-02 12:00:00 +0000 UTC 9:false 10:0] (988.953µs)
--- FAIL: TestStoreResponse (0.12s)
    client_test.go:115: storing response: storing response: transaction failed: got 4 elements in cluster info address, expected 2 or 3
FAIL
FAIL	github.com/letsencrypt/boulder/cmd/rocsp-tool	0.543s

and

--- FAIL: TestSetAndGet (0.01s)
    rocsp_test.go:50: storing response: transaction failed: got 4 elements in cluster info address, expected 2 or 3
FAIL
FAIL	github.com/letsencrypt/boulder/rocsp	0.042s

[aarongable]

It's failing this way in multiple PRs:
https://github.com/letsencrypt/boulder/runs/6232644947?check_suite_focus=true
https://github.com/letsencrypt/boulder/runs/6231447684?check_suite_focus=true

But I cannot reproduce it locally, neither on main nor with either of those PRs checked out.

[aarongable]

The lowest error message comes from here:

boulder/vendor/github.com/go-redis/redis/v8/command.go

Line 2704 in 0243b54

err := fmt.Errorf("got %d elements in cluster info address, expected 2 or 3", n)

[aarongable]

I updated my local redis docker image from aea9b698d7d1 (4 months old) to a10f849e1540 (2 days old, and the same version being run in github workflows and codespaces). I now reliably reproduce the failure. So it's something to do with the redis docker image version, not anything to do with our Boulder (or dependencies) code changing.

There is no more recent version of the go-redis/redis dependency available that might fix this.

[aarongable]

It's a major version update. We've always been specifying "redis:latest" in our docker-compose.yml:

boulder/docker-compose.yml

Line 69 in 0243b54

image: redis:latest

As of two days ago, the official "redis" docker image has released version 7.0.0, and updated their "latest" tag to point at it:

Because github workflows (and github codespaces) rebuild the environment from scratch every time, they get the true latest version every time. Most of the rest of us, whose local docker installations cache images, stick with the same version until we explicitly run a docker image pull redis:<version> to update it. This explains why we couldn't reproduce the error locally.

I ran docker image pull redis:7.0.0, and updated our docker-compose.yml to specify the same version, and was immediately able to reproduce the failures locally.

I then ran docker image pull redis:6.2.7 (the most recent non-v7 version) and did the same, and the errors went away.

I intend to fix this by pinning our redis to 6.2.7, and filing a bug against the go-redis/redis library to properly support talking to a v7 redis image.

[aarongable]

Here are the Redis 7.0.0 release notes: https://raw.githubusercontent.com/redis/redis/7.0/00-RELEASENOTES

Of particular interest, they mention:

Command replies that have been extended

• CLUSTER SLOTS hostname support (#9530)

That's the failing command, and "extending" it sounds exactly like giving it a fourth field, as the error message indicates. The referenced PR does exactly that.

[aarongable]

Upstream issue filed: redis/go-redis#2082