Use regex cache to save memory

[YakDriver]

Description

Saves ~18% memory use by introducing a cache layer, regexache, between the code and regexp.MustCompile(). This de-duplicates regular expressions in memory. Althought regexache can expired cache entries, testing so far seems to show the greatest reduction in memory use when the cache is not cleaned up (no expiration).

To save memory, there may be a small performance hit from cache contention. This should be monitored to ensure it's not noticeable.

These are the results of profiling memory with and without regexache.

• Not using regexache
  • Single VPC acc test: 6.76GB
  • 2 AppRunner acc tests: 17.89GB
• Using regexache, no expiration
  • Single VPC acc test: 5.54GB (18% less than w/o)
  • 2 AppRunner acc tests: 14.81GB (17.2% less than w/o)
• Using regexache, 1s expiration, expiration protection at 2 uses, 5s interval, 0.5s clean per interval
  • Single VPC acc test: 5.62GB (16.9% less than w/o)
  • 2 AppRunner acc tests: 15.00GB (16.2% less than w/o)
• Using regexache, 0.5s expiration, expiration protection at 10 uses, 1s interval, 0.1s clean per interval
  • Single VPC acc test: 5.62GB (16.9% less than w/o)
  • 2 AppRunner acc tests: 15.01GB (16.1% less than w/o)
• Using regexache, 2s expiration, expiration protection at 3 uses, 2s interval, 0.1s clean per interval
  • Single VPC acc test: 5.67GB (16.1% less than w/o)
  • 2 AppRunner acc tests: 15.05GB (15.9% less than w/o)

Relations

Relates #32289

References

• https://github.com/YakDriver/regexache

Output from Acceptance Testing

% make sane
==> Sane Check (48 tests of Top 30 resources)
==> Like 'sanity' except full output and stops soon after 1st error
==> NOTE: NOT an exhaustive set of tests! Finds big problems only.
=== RUN   TestAccIAMInstanceProfile_basic
=== PAUSE TestAccIAMInstanceProfile_basic
=== RUN   TestAccIAMInstanceProfile_tags
=== PAUSE TestAccIAMInstanceProfile_tags
=== RUN   TestAccIAMPolicyDocumentDataSource_basic
=== PAUSE TestAccIAMPolicyDocumentDataSource_basic
=== RUN   TestAccIAMPolicyDocumentDataSource_sourceConflicting
=== PAUSE TestAccIAMPolicyDocumentDataSource_sourceConflicting
=== RUN   TestAccIAMPolicy_basic
=== PAUSE TestAccIAMPolicy_basic
=== RUN   TestAccIAMPolicy_tags
=== PAUSE TestAccIAMPolicy_tags
=== RUN   TestAccIAMPolicy_policy
=== PAUSE TestAccIAMPolicy_policy
=== RUN   TestAccIAMRolePolicyAttachment_basic
=== PAUSE TestAccIAMRolePolicyAttachment_basic
=== RUN   TestAccIAMRolePolicyAttachment_disappears
=== PAUSE TestAccIAMRolePolicyAttachment_disappears
=== RUN   TestAccIAMRolePolicyAttachment_Disappears_role
=== PAUSE TestAccIAMRolePolicyAttachment_Disappears_role
=== RUN   TestAccIAMRolePolicy_basic
=== PAUSE TestAccIAMRolePolicy_basic
=== RUN   TestAccIAMRolePolicy_unknownsInPolicy
=== PAUSE TestAccIAMRolePolicy_unknownsInPolicy
=== RUN   TestAccIAMRole_basic
=== PAUSE TestAccIAMRole_basic
=== RUN   TestAccIAMRole_namePrefix
=== PAUSE TestAccIAMRole_namePrefix
=== RUN   TestAccIAMRole_disappears
=== PAUSE TestAccIAMRole_disappears
=== RUN   TestAccIAMRole_InlinePolicy_basic
=== PAUSE TestAccIAMRole_InlinePolicy_basic
=== CONT  TestAccIAMInstanceProfile_basic
=== CONT  TestAccIAMRolePolicyAttachment_disappears
=== CONT  TestAccIAMRole_basic
=== CONT  TestAccIAMRolePolicy_unknownsInPolicy
=== CONT  TestAccIAMPolicy_basic
=== CONT  TestAccIAMRolePolicy_basic
=== CONT  TestAccIAMRolePolicyAttachment_basic
=== CONT  TestAccIAMRole_namePrefix
=== CONT  TestAccIAMRolePolicyAttachment_Disappears_role
=== CONT  TestAccIAMPolicyDocumentDataSource_basic
=== CONT  TestAccIAMInstanceProfile_tags
=== CONT  TestAccIAMRole_disappears
=== CONT  TestAccIAMPolicyDocumentDataSource_sourceConflicting
=== CONT  TestAccIAMPolicy_policy
=== CONT  TestAccIAMRole_InlinePolicy_basic
=== CONT  TestAccIAMPolicy_tags
--- PASS: TestAccIAMPolicyDocumentDataSource_sourceConflicting (71.17s)
--- PASS: TestAccIAMPolicyDocumentDataSource_basic (72.43s)
--- PASS: TestAccIAMRole_disappears (77.27s)
--- PASS: TestAccIAMRolePolicyAttachment_Disappears_role (80.29s)
--- PASS: TestAccIAMRolePolicyAttachment_disappears (81.18s)
--- PASS: TestAccIAMRolePolicy_unknownsInPolicy (83.53s)
--- PASS: TestAccIAMPolicy_basic (89.75s)
--- PASS: TestAccIAMRole_namePrefix (90.12s)
--- PASS: TestAccIAMRole_basic (90.87s)
--- PASS: TestAccIAMInstanceProfile_basic (91.17s)
--- PASS: TestAccIAMRolePolicy_basic (126.25s)
--- PASS: TestAccIAMRolePolicyAttachment_basic (127.09s)
--- PASS: TestAccIAMPolicy_policy (128.47s)
--- PASS: TestAccIAMPolicy_tags (157.70s)
--- PASS: TestAccIAMRole_InlinePolicy_basic (158.58s)
--- PASS: TestAccIAMInstanceProfile_tags (159.07s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/iam	161.478s
=== RUN   TestAccLogsGroup_basic
=== PAUSE TestAccLogsGroup_basic
=== RUN   TestAccLogsGroup_multiple
=== PAUSE TestAccLogsGroup_multiple
=== CONT  TestAccLogsGroup_basic
=== CONT  TestAccLogsGroup_multiple
--- PASS: TestAccLogsGroup_multiple (67.47s)
--- PASS: TestAccLogsGroup_basic (81.52s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/logs	84.429s
=== RUN   TestAccVPCDataSource_basic
=== PAUSE TestAccVPCDataSource_basic
=== RUN   TestAccVPCRouteTableAssociation_Subnet_basic
=== PAUSE TestAccVPCRouteTableAssociation_Subnet_basic
=== RUN   TestAccVPCRouteTable_basic
=== PAUSE TestAccVPCRouteTable_basic
=== RUN   TestAccVPCSecurityGroupRule_race
=== PAUSE TestAccVPCSecurityGroupRule_race
=== RUN   TestAccVPCSecurityGroupRule_protocolChange
=== PAUSE TestAccVPCSecurityGroupRule_protocolChange
=== RUN   TestAccVPCSecurityGroup_basic
=== PAUSE TestAccVPCSecurityGroup_basic
=== RUN   TestAccVPCSecurityGroup_ipRangesWithSameRules
=== PAUSE TestAccVPCSecurityGroup_ipRangesWithSameRules
=== RUN   TestAccVPCSecurityGroup_vpcAllEgress
=== PAUSE TestAccVPCSecurityGroup_vpcAllEgress
=== RUN   TestAccVPCSubnet_basic
=== PAUSE TestAccVPCSubnet_basic
=== RUN   TestAccVPC_tenancy
=== PAUSE TestAccVPC_tenancy
=== CONT  TestAccVPCDataSource_basic
=== CONT  TestAccVPCSecurityGroup_basic
=== CONT  TestAccVPCSecurityGroupRule_race
=== CONT  TestAccVPC_tenancy
=== CONT  TestAccVPCSecurityGroupRule_protocolChange
=== CONT  TestAccVPCSubnet_basic
=== CONT  TestAccVPCSecurityGroup_vpcAllEgress
=== CONT  TestAccVPCRouteTable_basic
=== CONT  TestAccVPCSecurityGroup_ipRangesWithSameRules
=== CONT  TestAccVPCRouteTableAssociation_Subnet_basic
--- PASS: TestAccVPCRouteTable_basic (102.50s)
--- PASS: TestAccVPCDataSource_basic (102.50s)
--- PASS: TestAccVPCSubnet_basic (102.66s)
--- PASS: TestAccVPCSecurityGroup_basic (104.06s)
--- PASS: TestAccVPCSecurityGroup_vpcAllEgress (104.53s)
--- PASS: TestAccVPCRouteTableAssociation_Subnet_basic (105.91s)
--- PASS: TestAccVPCSecurityGroup_ipRangesWithSameRules (106.86s)
--- PASS: TestAccVPCSecurityGroupRule_protocolChange (175.35s)
--- PASS: TestAccVPC_tenancy (176.35s)
--- PASS: TestAccVPCSecurityGroupRule_race (194.35s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/ec2	200.060s
=== RUN   TestAccECSService_basic
=== PAUSE TestAccECSService_basic
=== RUN   TestAccECSService_basicImport
=== PAUSE TestAccECSService_basicImport
=== RUN   TestAccECSTaskDefinition_basic
=== PAUSE TestAccECSTaskDefinition_basic
=== CONT  TestAccECSService_basic
=== CONT  TestAccECSTaskDefinition_basic
=== CONT  TestAccECSService_basicImport
--- PASS: TestAccECSTaskDefinition_basic (125.55s)
--- PASS: TestAccECSService_basic (176.12s)
--- PASS: TestAccECSService_basicImport (259.34s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/ecs	262.046s
=== RUN   TestAccELBV2TargetGroup_basic
=== PAUSE TestAccELBV2TargetGroup_basic
=== CONT  TestAccELBV2TargetGroup_basic
--- PASS: TestAccELBV2TargetGroup_basic (73.80s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/elbv2	78.266s
=== RUN   TestAccKMSKey_basic
=== PAUSE TestAccKMSKey_basic
=== CONT  TestAccKMSKey_basic
--- PASS: TestAccKMSKey_basic (119.52s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/kms	125.110s
=== RUN   TestAccLambdaFunction_basic
=== PAUSE TestAccLambdaFunction_basic
=== RUN   TestAccLambdaPermission_basic
=== PAUSE TestAccLambdaPermission_basic
=== CONT  TestAccLambdaFunction_basic
=== CONT  TestAccLambdaPermission_basic
--- PASS: TestAccLambdaPermission_basic (101.96s)
--- PASS: TestAccLambdaFunction_basic (107.04s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/lambda	109.866s
=== RUN   TestAccMetaPartitionDataSource_basic
=== PAUSE TestAccMetaPartitionDataSource_basic
=== RUN   TestAccMetaRegionDataSource_basic
=== PAUSE TestAccMetaRegionDataSource_basic
=== RUN   TestAccMetaRegionDataSource_endpoint
=== PAUSE TestAccMetaRegionDataSource_endpoint
=== RUN   TestAccMetaRegionDataSource_endpointAndName
=== PAUSE TestAccMetaRegionDataSource_endpointAndName
=== CONT  TestAccMetaPartitionDataSource_basic
=== CONT  TestAccMetaRegionDataSource_endpoint
=== CONT  TestAccMetaRegionDataSource_endpointAndName
=== CONT  TestAccMetaRegionDataSource_basic
--- PASS: TestAccMetaRegionDataSource_basic (69.50s)
--- PASS: TestAccMetaPartitionDataSource_basic (70.56s)
--- PASS: TestAccMetaRegionDataSource_endpoint (71.75s)
--- PASS: TestAccMetaRegionDataSource_endpointAndName (71.98s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/meta	76.843s
=== RUN   TestAccRoute53Record_basic
=== PAUSE TestAccRoute53Record_basic
=== RUN   TestAccRoute53Record_Latency_basic
=== PAUSE TestAccRoute53Record_Latency_basic
=== RUN   TestAccRoute53ZoneDataSource_name
=== PAUSE TestAccRoute53ZoneDataSource_name
=== CONT  TestAccRoute53Record_basic
=== CONT  TestAccRoute53ZoneDataSource_name
=== CONT  TestAccRoute53Record_Latency_basic
--- PASS: TestAccRoute53ZoneDataSource_name (111.54s)
--- PASS: TestAccRoute53Record_basic (167.45s)
--- PASS: TestAccRoute53Record_Latency_basic (188.90s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/route53	194.273s
=== RUN   TestAccS3BucketACL_updateACL
=== PAUSE TestAccS3BucketACL_updateACL
=== RUN   TestAccS3BucketPolicy_basic
=== PAUSE TestAccS3BucketPolicy_basic
=== RUN   TestAccS3BucketPublicAccessBlock_basic
=== PAUSE TestAccS3BucketPublicAccessBlock_basic
=== RUN   TestAccS3Bucket_Basic_basic
=== PAUSE TestAccS3Bucket_Basic_basic
=== RUN   TestAccS3Bucket_Security_corsUpdate
=== PAUSE TestAccS3Bucket_Security_corsUpdate
=== CONT  TestAccS3BucketACL_updateACL
=== CONT  TestAccS3Bucket_Basic_basic
=== CONT  TestAccS3BucketPublicAccessBlock_basic
=== CONT  TestAccS3Bucket_Security_corsUpdate
=== CONT  TestAccS3BucketPolicy_basic
--- PASS: TestAccS3Bucket_Basic_basic (94.10s)
--- PASS: TestAccS3BucketPolicy_basic (95.13s)
--- PASS: TestAccS3BucketPublicAccessBlock_basic (95.26s)
--- PASS: TestAccS3BucketACL_updateACL (134.71s)
--- PASS: TestAccS3Bucket_Security_corsUpdate (134.95s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/s3	139.151s
=== RUN   TestAccSecretsManagerSecret_basic
=== PAUSE TestAccSecretsManagerSecret_basic
=== RUN   TestAccSecretsManagerSecret_basicReplica
=== PAUSE TestAccSecretsManagerSecret_basicReplica
=== CONT  TestAccSecretsManagerSecret_basic
=== CONT  TestAccSecretsManagerSecret_basicReplica
--- PASS: TestAccSecretsManagerSecret_basic (81.41s)
--- PASS: TestAccSecretsManagerSecret_basicReplica (106.01s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/secretsmanager	111.525s
=== RUN   TestAccSTSCallerIdentityDataSource_basic
=== PAUSE TestAccSTSCallerIdentityDataSource_basic
=== CONT  TestAccSTSCallerIdentityDataSource_basic
--- PASS: TestAccSTSCallerIdentityDataSource_basic (61.62s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/sts	67.149s

[github-actions]

This functionality has been released in v5.14.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.