aws-secretsmanager: Secret rotation fails in China if using Postgresql > 13

[holomekc]

Describe the bug

With Postgresql 14+ the password encryption was changed from md5 to scram-sha-256.
The used semantic version from application repository is very, very old and uses a libpq.so.5 version, which does not support scram. Please update to the latest available version: 1.1.212 (aws-cn).

aws-cdk/packages/aws-cdk-lib/aws-secretsmanager/lib/secret-rotation.ts

Lines 161 to 172 in 58b90c4

	public semanticVersionForPartition(partition: string) {
	if (partition === 'aws') {
	return this.semanticVersion;
	} else if (partition === 'aws-cn') {
	return '1.1.37';
	} else if (partition === 'aws-us-gov') {
	return '1.1.93';
	} else {
	throw new Error(`unsupported partition: ${partition}`);
	}
	}
	}

Expected Behavior

If secret rotation is configured for Postgresql it should also support versions > 13.

Current Behavior

Lambda fails with:
setSecret: Unable to log into database with previous, current, or pending secret of secret arn

After updating the code manually and add more logging we could see:
SCRAM authentication requires libpq version 10 or above

Reproduction Steps

• Create an RDS with postgresql engine version > 13 in China (cn-north-1)
• Setup password rotation
• Rotate the password

Possible Solution

As mentioned above update to 1.1.212 for aws-cn partition. Not sure about aws-us-gov.

Additional Information/Context

CDK CLI Version

2.120.0

Framework Version

No response

Node.js Version

20.11.0

OS

Mac OSX

Language

Java

Language Version

11

Other information

No response

[laurelmay]

It looks like the newest version for aws-us-gov is 1.1.164.

$ aws serverlessrepo get-application --application-id arn:aws-us-gov:serverlessrepo:us-gov-west-1:023102451235:applications/SecretsManagerRDSPostgreSQLRotationSingleUser --query Version.SemanticVersion --output text
1.1.164

It's sad that this class doesn't get a reference to the construct scope to be able to use the RegionInfo db easier.

[pahud]

I can confirm the version in BJS region is 1.1.212 but I am not sure if it's safe to just bump the version but looks like there's no way for users to customize the version number from the construct.

% aws --profile bjs serverlessrepo get-application --application-id arn:aws-cn:serverlessrepo:cn-north-1:193023089310:applications/SecretsManagerRDSPostgreSQLRotationSingleUser --query Version.SemanticVersion --output text
1.1.212

[holomekc]

Hi @pahud ,
This is true. We were forced to override the complete application definition. This works ok-ish, but it would be great if this wouldn‘t be necessary.

[pahud]

@holomekc I think we'll need a way to allow users to define their preferred versions and that requires a separate PR. Bumping the version to the latest could be dangerous I believe.

[pahud]

I probably was wrong as #26884 just bumped the version for aws partition.

I guess we can have separate PR for that.

[holomekc]

I guess it depends if it is backwards compatible. We use Postgres 13 and 14. it works for both. We also compared the code from the Lambdas in aws and aws-cn. They match. So in aws it is kind of already updated.

Although there are sometimes differences between aws and aws-cn the code is super similar. The important part of the new version is the updated libpq.so.5, which supports scram now. The connection approach slightly changes and contains some SSL adjustments. I did not check in detail. But I also cannot imagine that AWS updates the lambda in the application repo, which then doesn't support all available Postgres versions.

We plan to update our DB versions asap because md5... well you know.

There is also the topic of changed certificates, because the 2019 rds ones expire in August. Maybe the ssl adjustments are for that. I would need to check what the code actually does there.

Best regards,
Chris

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.