Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for S3 Object Lock legal holds, retention modes and retention periods #9942

Merged
merged 10 commits into from
Sep 18, 2019
Merged

Add support for S3 Object Lock legal holds, retention modes and retention periods #9942

merged 10 commits into from
Sep 18, 2019

Conversation

ewbankkit
Copy link
Contributor

@ewbankkit ewbankkit commented Aug 30, 2019 •
edited
Loading

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Closes #7158.
Closes #7159.
Replaces #7179.

Add support for S3 Object Lock legal holds, retention modes and retention periods.
.
Release note for CHANGELOG:

data-source/aws_s3_bucket_object: Add `object_lock_legal_hold_status`, `object_lock_mode` and `object_lock_retain_until_date` attributes
resource/aws_s3_bucket_object: Add `force_destroy`, `object_lock_legal_hold_status`, `object_lock_mode` and `object_lock_retain_until_date` attributes

Output from acceptance testing:

aws_s3_bucket_object

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccDataSourceAWSS3BucketObject_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 20 -run=TestAccDataSourceAWSS3BucketObject_ -timeout 120m
=== RUN   TestAccDataSourceAWSS3BucketObject_basic
=== PAUSE TestAccDataSourceAWSS3BucketObject_basic
=== RUN   TestAccDataSourceAWSS3BucketObject_readableBody
=== PAUSE TestAccDataSourceAWSS3BucketObject_readableBody
=== RUN   TestAccDataSourceAWSS3BucketObject_kmsEncrypted
=== PAUSE TestAccDataSourceAWSS3BucketObject_kmsEncrypted
=== RUN   TestAccDataSourceAWSS3BucketObject_allParams
=== PAUSE TestAccDataSourceAWSS3BucketObject_allParams
=== RUN   TestAccDataSourceAWSS3BucketObject_ObjectLockLegalHoldOff
=== PAUSE TestAccDataSourceAWSS3BucketObject_ObjectLockLegalHoldOff
=== RUN   TestAccDataSourceAWSS3BucketObject_ObjectLockLegalHoldOn
=== PAUSE TestAccDataSourceAWSS3BucketObject_ObjectLockLegalHoldOn
=== CONT  TestAccDataSourceAWSS3BucketObject_basic
=== CONT  TestAccDataSourceAWSS3BucketObject_ObjectLockLegalHoldOn
=== CONT  TestAccDataSourceAWSS3BucketObject_ObjectLockLegalHoldOff
=== CONT  TestAccDataSourceAWSS3BucketObject_allParams
=== CONT  TestAccDataSourceAWSS3BucketObject_kmsEncrypted
=== CONT  TestAccDataSourceAWSS3BucketObject_readableBody
--- PASS: TestAccDataSourceAWSS3BucketObject_readableBody (57.06s)
--- PASS: TestAccDataSourceAWSS3BucketObject_basic (57.09s)
--- PASS: TestAccDataSourceAWSS3BucketObject_allParams (57.60s)
--- PASS: TestAccDataSourceAWSS3BucketObject_ObjectLockLegalHoldOff (58.27s)
--- PASS: TestAccDataSourceAWSS3BucketObject_ObjectLockLegalHoldOn (59.89s)
--- PASS: TestAccDataSourceAWSS3BucketObject_kmsEncrypted (80.70s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	80.768s
$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSS3BucketObject_'==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 20 -run=TestAccAWSS3BucketObject_ -timeout 120m
=== RUN   TestAccAWSS3BucketObject_noNameNoKey
=== PAUSE TestAccAWSS3BucketObject_noNameNoKey
=== RUN   TestAccAWSS3BucketObject_empty
=== PAUSE TestAccAWSS3BucketObject_empty
=== RUN   TestAccAWSS3BucketObject_source
=== PAUSE TestAccAWSS3BucketObject_source
=== RUN   TestAccAWSS3BucketObject_content
=== PAUSE TestAccAWSS3BucketObject_content
=== RUN   TestAccAWSS3BucketObject_etagEncryption
=== PAUSE TestAccAWSS3BucketObject_etagEncryption
=== RUN   TestAccAWSS3BucketObject_contentBase64
=== PAUSE TestAccAWSS3BucketObject_contentBase64
=== RUN   TestAccAWSS3BucketObject_withContentCharacteristics
=== PAUSE TestAccAWSS3BucketObject_withContentCharacteristics
=== RUN   TestAccAWSS3BucketObject_updates
=== PAUSE TestAccAWSS3BucketObject_updates
=== RUN   TestAccAWSS3BucketObject_updateSameFile
=== PAUSE TestAccAWSS3BucketObject_updateSameFile
=== RUN   TestAccAWSS3BucketObject_updatesWithVersioning
=== PAUSE TestAccAWSS3BucketObject_updatesWithVersioning
=== RUN   TestAccAWSS3BucketObject_kms
=== PAUSE TestAccAWSS3BucketObject_kms
=== RUN   TestAccAWSS3BucketObject_sse
=== PAUSE TestAccAWSS3BucketObject_sse
=== RUN   TestAccAWSS3BucketObject_acl
=== PAUSE TestAccAWSS3BucketObject_acl
=== RUN   TestAccAWSS3BucketObject_metadata
=== PAUSE TestAccAWSS3BucketObject_metadata
=== RUN   TestAccAWSS3BucketObject_storageClass
=== PAUSE TestAccAWSS3BucketObject_storageClass
=== RUN   TestAccAWSS3BucketObject_tags
=== PAUSE TestAccAWSS3BucketObject_tags
=== RUN   TestAccAWSS3BucketObject_tagsLeadingSlash
=== PAUSE TestAccAWSS3BucketObject_tagsLeadingSlash
=== RUN   TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithNone
=== PAUSE TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithNone
=== RUN   TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithOn
=== PAUSE TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithOn
=== RUN   TestAccAWSS3BucketObject_ObjectLockRetentionStartWithNone
=== PAUSE TestAccAWSS3BucketObject_ObjectLockRetentionStartWithNone
=== RUN   TestAccAWSS3BucketObject_ObjectLockRetentionStartWithSet
=== PAUSE TestAccAWSS3BucketObject_ObjectLockRetentionStartWithSet
=== CONT  TestAccAWSS3BucketObject_noNameNoKey
=== CONT  TestAccAWSS3BucketObject_ObjectLockRetentionStartWithSet
=== CONT  TestAccAWSS3BucketObject_ObjectLockRetentionStartWithNone
=== CONT  TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithOn
=== CONT  TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithNone
=== CONT  TestAccAWSS3BucketObject_tagsLeadingSlash
=== CONT  TestAccAWSS3BucketObject_tags
=== CONT  TestAccAWSS3BucketObject_storageClass
=== CONT  TestAccAWSS3BucketObject_metadata
=== CONT  TestAccAWSS3BucketObject_acl
=== CONT  TestAccAWSS3BucketObject_sse
=== CONT  TestAccAWSS3BucketObject_kms
=== CONT  TestAccAWSS3BucketObject_updatesWithVersioning
=== CONT  TestAccAWSS3BucketObject_updateSameFile
=== CONT  TestAccAWSS3BucketObject_updates
=== CONT  TestAccAWSS3BucketObject_withContentCharacteristics
=== CONT  TestAccAWSS3BucketObject_contentBase64
=== CONT  TestAccAWSS3BucketObject_etagEncryption
=== CONT  TestAccAWSS3BucketObject_content
=== CONT  TestAccAWSS3BucketObject_source
--- PASS: TestAccAWSS3BucketObject_noNameNoKey (7.34s)
=== CONT  TestAccAWSS3BucketObject_empty
--- PASS: TestAccAWSS3BucketObject_etagEncryption (47.60s)
--- PASS: TestAccAWSS3BucketObject_source (47.78s)
--- PASS: TestAccAWSS3BucketObject_withContentCharacteristics (47.85s)
--- PASS: TestAccAWSS3BucketObject_content (47.98s)
--- PASS: TestAccAWSS3BucketObject_contentBase64 (48.08s)
--- PASS: TestAccAWSS3BucketObject_sse (48.59s)
--- PASS: TestAccAWSS3BucketObject_empty (41.82s)
--- PASS: TestAccAWSS3BucketObject_kms (72.71s)
--- PASS: TestAccAWSS3BucketObject_updates (74.49s)
--- PASS: TestAccAWSS3BucketObject_updatesWithVersioning (75.76s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithOn (76.17s)
--- PASS: TestAccAWSS3BucketObject_updateSameFile (76.36s)
--- PASS: TestAccAWSS3BucketObject_metadata (100.03s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockRetentionStartWithNone (101.45s)
--- PASS: TestAccAWSS3BucketObject_acl (101.77s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithNone (102.23s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockRetentionStartWithSet (121.94s)
--- PASS: TestAccAWSS3BucketObject_tagsLeadingSlash (122.56s)
--- PASS: TestAccAWSS3BucketObject_tags (122.82s)
--- PASS: TestAccAWSS3BucketObject_storageClass (143.16s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	143.193s
$ go test ./aws -v -sweep=us-east-1,us-west-2 -sweep-run=aws_s3_bucket_object
2019/09/02 15:04:33 [DEBUG] Running Sweepers for region (us-east-1):
2019/09/02 15:04:33 [INFO] Building AWS auth structure
2019/09/02 15:04:33 [INFO] Setting AWS metadata API timeout to 100ms
2019/09/02 15:04:34 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2019/09/02 15:04:34 [INFO] AWS Auth provider used: "EnvProvider"
2019/09/02 15:04:34 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2019/09/02 15:04:34 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2019/09/02 15:04:35 [INFO] Skipping S3 Bucket: 346386234494-awsmacietrail-dataevent
2019/09/02 15:04:35 [INFO] Skipping S3 Bucket: ewbankkit-object-lock-enabled-compliance
2019/09/02 15:04:35 [INFO] Skipping S3 Bucket: ewbankkit-object-lock-enabled-governance
2019/09/02 15:04:35 [INFO] Skipping S3 Bucket: ewbankkit-object-lock-enabled-no-mode
2019/09/02 15:04:35 [INFO] Skipping S3 Bucket (tf-test-bucket-5621812836250344014) in different region: us-west-2
2019/09/02 15:04:35 [INFO] Deleting S3 Bucket (tf-test-bucket-5621812836250344015) Object (it.pdf) Version: z2Wpp3t1SMvP4xrbO83oOuEt8HynXJDL
2019/09/02 15:04:35 [WARN] Error deleting S3 Bucket (tf-test-bucket-5621812836250344015) Object (it.pdf) Version (z2Wpp3t1SMvP4xrbO83oOuEt8HynXJDL): AccessDenied: Access Denied
	status code: 403, request id: A44BEE7F6482C2C1, host id: TsFLNnBowL7SJmvQs+CVshwXD8zuSTOe4f2yYxDyBJCLF5iZ2ad/pm4x6q0m+BJ/DvbIMzzCHio=
2019/09/02 15:04:35 [INFO] Deleting S3 Bucket (tf-test-bucket-5621812836250344015) Object (it.pdf) Version: z2Wpp3t1SMvP4xrbO83oOuEt8HynXJDL
2019/09/02 15:04:35 Sweeper Tests ran:
	- aws_s3_bucket_object
2019/09/02 15:04:35 [DEBUG] Running Sweepers for region (us-west-2):
2019/09/02 15:04:35 [INFO] Building AWS auth structure
2019/09/02 15:04:35 [INFO] Setting AWS metadata API timeout to 100ms
2019/09/02 15:04:36 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2019/09/02 15:04:36 [INFO] AWS Auth provider used: "EnvProvider"
2019/09/02 15:04:36 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2019/09/02 15:04:36 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2019/09/02 15:04:37 [INFO] Skipping S3 Bucket: 346386234494-awsmacietrail-dataevent
2019/09/02 15:04:37 [INFO] Skipping S3 Bucket: ewbankkit-object-lock-enabled-compliance
2019/09/02 15:04:37 [INFO] Skipping S3 Bucket: ewbankkit-object-lock-enabled-governance
2019/09/02 15:04:37 [INFO] Skipping S3 Bucket: ewbankkit-object-lock-enabled-no-mode
2019/09/02 15:04:38 [INFO] Deleting S3 Bucket (tf-test-bucket-5621812836250344014) Object (it.pdf) Version: null
2019/09/02 15:04:39 [INFO] Deleting S3 Bucket (tf-test-bucket-5621812836250344014) Object (link.pdf) Version: null
2019/09/02 15:04:39 [INFO] Deleting S3 Bucket (tf-test-bucket-5621812836250344014) Object (tpu.pdf) Version: null
2019/09/02 15:04:40 [INFO] Skipping S3 Bucket (tf-test-bucket-5621812836250344015) in different region: us-east-1
2019/09/02 15:04:40 Sweeper Tests ran:
	- aws_s3_bucket_object
ok  	github.com/terraform-providers/terraform-provider-aws/aws	6.634s

aws_s3_bucket

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSS3Bucket_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 20 -run=TestAccAWSS3Bucket_ -timeout 120m
=== RUN   TestAccAWSS3Bucket_importBasic
=== PAUSE TestAccAWSS3Bucket_importBasic
=== RUN   TestAccAWSS3Bucket_importWithPolicy
=== PAUSE TestAccAWSS3Bucket_importWithPolicy
=== RUN   TestAccAWSS3Bucket_basic
=== PAUSE TestAccAWSS3Bucket_basic
=== RUN   TestAccAWSS3Bucket_Bucket_EmptyString
=== PAUSE TestAccAWSS3Bucket_Bucket_EmptyString
=== RUN   TestAccAWSS3Bucket_tagsWithNoSystemTags
=== PAUSE TestAccAWSS3Bucket_tagsWithNoSystemTags
=== RUN   TestAccAWSS3Bucket_tagsWithSystemTags
=== PAUSE TestAccAWSS3Bucket_tagsWithSystemTags
=== RUN   TestAccAWSS3Bucket_namePrefix
=== PAUSE TestAccAWSS3Bucket_namePrefix
=== RUN   TestAccAWSS3Bucket_generatedName
=== PAUSE TestAccAWSS3Bucket_generatedName
=== RUN   TestAccAWSS3Bucket_region
=== PAUSE TestAccAWSS3Bucket_region
=== RUN   TestAccAWSS3Bucket_acceleration
=== PAUSE TestAccAWSS3Bucket_acceleration
=== RUN   TestAccAWSS3Bucket_RequestPayer
=== PAUSE TestAccAWSS3Bucket_RequestPayer
=== RUN   TestAccAWSS3Bucket_Policy
=== PAUSE TestAccAWSS3Bucket_Policy
=== RUN   TestAccAWSS3Bucket_UpdateAcl
=== PAUSE TestAccAWSS3Bucket_UpdateAcl
=== RUN   TestAccAWSS3Bucket_Website_Simple
=== PAUSE TestAccAWSS3Bucket_Website_Simple
=== RUN   TestAccAWSS3Bucket_WebsiteRedirect
=== PAUSE TestAccAWSS3Bucket_WebsiteRedirect
=== RUN   TestAccAWSS3Bucket_WebsiteRoutingRules
=== PAUSE TestAccAWSS3Bucket_WebsiteRoutingRules
=== RUN   TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical
=== PAUSE TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical
=== RUN   TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed
=== PAUSE TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed
=== RUN   TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled
=== PAUSE TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled
=== RUN   TestAccAWSS3Bucket_shouldFailNotFound
=== PAUSE TestAccAWSS3Bucket_shouldFailNotFound
=== RUN   TestAccAWSS3Bucket_Versioning
=== PAUSE TestAccAWSS3Bucket_Versioning
=== RUN   TestAccAWSS3Bucket_Cors_Update
=== PAUSE TestAccAWSS3Bucket_Cors_Update
=== RUN   TestAccAWSS3Bucket_Cors_Delete
=== PAUSE TestAccAWSS3Bucket_Cors_Delete
=== RUN   TestAccAWSS3Bucket_Cors_EmptyOrigin
=== PAUSE TestAccAWSS3Bucket_Cors_EmptyOrigin
=== RUN   TestAccAWSS3Bucket_Logging
=== PAUSE TestAccAWSS3Bucket_Logging
=== RUN   TestAccAWSS3Bucket_LifecycleBasic
=== PAUSE TestAccAWSS3Bucket_LifecycleBasic
=== RUN   TestAccAWSS3Bucket_LifecycleExpireMarkerOnly
=== PAUSE TestAccAWSS3Bucket_LifecycleExpireMarkerOnly
=== RUN   TestAccAWSS3Bucket_Replication
=== PAUSE TestAccAWSS3Bucket_Replication
=== RUN   TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation
=== PAUSE TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation
=== RUN   TestAccAWSS3Bucket_ReplicationWithoutStorageClass
=== PAUSE TestAccAWSS3Bucket_ReplicationWithoutStorageClass
=== RUN   TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError
=== PAUSE TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError
=== RUN   TestAccAWSS3Bucket_ReplicationWithoutPrefix
=== PAUSE TestAccAWSS3Bucket_ReplicationWithoutPrefix
=== RUN   TestAccAWSS3Bucket_ReplicationSchemaV2
=== PAUSE TestAccAWSS3Bucket_ReplicationSchemaV2
=== RUN   TestAccAWSS3Bucket_objectLock
=== PAUSE TestAccAWSS3Bucket_objectLock
=== RUN   TestAccAWSS3Bucket_forceDestroy
=== PAUSE TestAccAWSS3Bucket_forceDestroy
=== RUN   TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled
=== PAUSE TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled
=== CONT  TestAccAWSS3Bucket_importBasic
=== CONT  TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled
=== CONT  TestAccAWSS3Bucket_forceDestroy
=== CONT  TestAccAWSS3Bucket_objectLock
=== CONT  TestAccAWSS3Bucket_ReplicationSchemaV2
=== CONT  TestAccAWSS3Bucket_ReplicationWithoutPrefix
=== CONT  TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError
=== CONT  TestAccAWSS3Bucket_ReplicationWithoutStorageClass
=== CONT  TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation
=== CONT  TestAccAWSS3Bucket_Replication
=== CONT  TestAccAWSS3Bucket_LifecycleExpireMarkerOnly
=== CONT  TestAccAWSS3Bucket_LifecycleBasic
=== CONT  TestAccAWSS3Bucket_Logging
=== CONT  TestAccAWSS3Bucket_Cors_EmptyOrigin
=== CONT  TestAccAWSS3Bucket_Cors_Delete
=== CONT  TestAccAWSS3Bucket_Cors_Update
=== CONT  TestAccAWSS3Bucket_Versioning
=== CONT  TestAccAWSS3Bucket_shouldFailNotFound
=== CONT  TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled
=== CONT  TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed
--- PASS: TestAccAWSS3Bucket_shouldFailNotFound (28.69s)
=== CONT  TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical
--- PASS: TestAccAWSS3Bucket_Cors_EmptyOrigin (45.70s)
=== CONT  TestAccAWSS3Bucket_WebsiteRoutingRules
--- PASS: TestAccAWSS3Bucket_Cors_Delete (45.90s)
=== CONT  TestAccAWSS3Bucket_WebsiteRedirect
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed (45.94s)
=== CONT  TestAccAWSS3Bucket_Website_Simple
--- PASS: TestAccAWSS3Bucket_forceDestroy (47.74s)
=== CONT  TestAccAWSS3Bucket_UpdateAcl
--- PASS: TestAccAWSS3Bucket_importBasic (51.02s)
=== CONT  TestAccAWSS3Bucket_Policy
--- PASS: TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled (51.62s)
=== CONT  TestAccAWSS3Bucket_RequestPayer
--- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (57.06s)
=== CONT  TestAccAWSS3Bucket_acceleration
--- FAIL: TestAccAWSS3Bucket_LifecycleExpireMarkerOnly (67.60s)
    testing.go:568: Step 1 error: After applying this step, the plan was not empty:
        
        DIFF:
        
        UPDATE: aws_s3_bucket.bucket
          acceleration_status:                                        "" => ""
          acl:                                                        "public-read" => "public-read"
          arn:                                                        "arn:aws:s3:::tf-test-bucket-8342519326936755730" => "arn:aws:s3:::tf-test-bucket-8342519326936755730"
          bucket:                                                     "tf-test-bucket-8342519326936755730" => "tf-test-bucket-8342519326936755730"
          bucket_domain_name:                                         "tf-test-bucket-8342519326936755730.s3.amazonaws.com" => "tf-test-bucket-8342519326936755730.s3.amazonaws.com"
          bucket_regional_domain_name:                                "tf-test-bucket-8342519326936755730.s3.us-west-2.amazonaws.com" => "tf-test-bucket-8342519326936755730.s3.us-west-2.amazonaws.com"
          cors_rule.#:                                                "0" => "0"
          force_destroy:                                              "false" => "false"
          hosted_zone_id:                                             "Z3BJ6K6RIION7M" => "Z3BJ6K6RIION7M"
          id:                                                         "tf-test-bucket-8342519326936755730" => "tf-test-bucket-8342519326936755730"
          lifecycle_rule.#:                                           "1" => "0"
          lifecycle_rule.0.abort_incomplete_multipart_upload_days:    "0" => ""
          lifecycle_rule.0.enabled:                                   "true" => ""
          lifecycle_rule.0.expiration.#:                              "1" => ""
          lifecycle_rule.0.expiration.0.date:                         "" => ""
          lifecycle_rule.0.expiration.0.days:                         "0" => ""
          lifecycle_rule.0.expiration.0.expired_object_delete_marker: "true" => ""
          lifecycle_rule.0.id:                                        "id1" => ""
          lifecycle_rule.0.noncurrent_version_expiration.#:           "0" => ""
          lifecycle_rule.0.noncurrent_version_transition.#:           "0" => ""
          lifecycle_rule.0.prefix:                                    "path1/" => ""
          lifecycle_rule.0.transition.#:                              "0" => ""
          logging.#:                                                  "0" => "0"
          object_lock_configuration.#:                                "0" => "0"
          region:                                                     "us-west-2" => "us-west-2"
          replication_configuration.#:                                "0" => "0"
          request_payer:                                              "BucketOwner" => "BucketOwner"
          server_side_encryption_configuration.#:                     "0" => "0"
          versioning.#:                                               "1" => "1"
          versioning.0.enabled:                                       "false" => "false"
          versioning.0.mfa_delete:                                    "false" => "false"
          website.#:                                                  "0" => "0"
        
        
        
        STATE:
        
        aws_s3_bucket.bucket:
          ID = tf-test-bucket-8342519326936755730
          provider = provider.aws
          acceleration_status = 
          acl = public-read
          arn = arn:aws:s3:::tf-test-bucket-8342519326936755730
          bucket = tf-test-bucket-8342519326936755730
          bucket_domain_name = tf-test-bucket-8342519326936755730.s3.amazonaws.com
          bucket_regional_domain_name = tf-test-bucket-8342519326936755730.s3.us-west-2.amazonaws.com
          force_destroy = false
          hosted_zone_id = Z3BJ6K6RIION7M
          lifecycle_rule.# = 1
          lifecycle_rule.0.abort_incomplete_multipart_upload_days = 0
          lifecycle_rule.0.enabled = true
          lifecycle_rule.0.expiration.# = 1
          lifecycle_rule.0.expiration.0.date = 
          lifecycle_rule.0.expiration.0.days = 0
          lifecycle_rule.0.expiration.0.expired_object_delete_marker = true
          lifecycle_rule.0.id = id1
          lifecycle_rule.0.prefix = path1/
          region = us-west-2
          request_payer = BucketOwner
          versioning.# = 1
          versioning.0.enabled = false
          versioning.0.mfa_delete = false
=== CONT  TestAccAWSS3Bucket_region
--- PASS: TestAccAWSS3Bucket_Logging (67.71s)
=== CONT  TestAccAWSS3Bucket_generatedName
--- PASS: TestAccAWSS3Bucket_objectLock (77.00s)
=== CONT  TestAccAWSS3Bucket_namePrefix
--- PASS: TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled (77.55s)
=== CONT  TestAccAWSS3Bucket_tagsWithSystemTags
--- PASS: TestAccAWSS3Bucket_Cors_Update (78.45s)
=== CONT  TestAccAWSS3Bucket_tagsWithNoSystemTags
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (93.81s)
=== CONT  TestAccAWSS3Bucket_Bucket_EmptyString
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutPrefix (94.49s)
=== CONT  TestAccAWSS3Bucket_basic
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical (71.10s)
=== CONT  TestAccAWSS3Bucket_importWithPolicy
--- PASS: TestAccAWSS3Bucket_generatedName (35.85s)
--- PASS: TestAccAWSS3Bucket_LifecycleBasic (106.34s)
--- PASS: TestAccAWSS3Bucket_Versioning (107.00s)
--- PASS: TestAccAWSS3Bucket_namePrefix (33.27s)
--- PASS: TestAccAWSS3Bucket_region (43.60s)
--- PASS: TestAccAWSS3Bucket_UpdateAcl (66.55s)
--- PASS: TestAccAWSS3Bucket_WebsiteRoutingRules (69.46s)
--- PASS: TestAccAWSS3Bucket_RequestPayer (67.33s)
--- PASS: TestAccAWSS3Bucket_acceleration (66.50s)
--- PASS: TestAccAWSS3Bucket_Bucket_EmptyString (33.01s)
--- PASS: TestAccAWSS3Bucket_basic (34.03s)
--- PASS: TestAccAWSS3Bucket_Policy (89.73s)
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (95.82s)
--- PASS: TestAccAWSS3Bucket_importWithPolicy (41.96s)
--- PASS: TestAccAWSS3Bucket_Website_Simple (96.06s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (180.86s)
--- PASS: TestAccAWSS3Bucket_tagsWithNoSystemTags (114.48s)
--- PASS: TestAccAWSS3Bucket_tagsWithSystemTags (147.98s)
--- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (261.99s)
--- PASS: TestAccAWSS3Bucket_Replication (270.09s)
FAIL
FAIL	github.com/terraform-providers/terraform-provider-aws/aws	270.184s
GNUmakefile:20: recipe for target 'testacc' failed
make: *** [testacc] Error 1
$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSS3Bucket_LifecycleExpireMarkerOnly'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 20 -run=TestAccAWSS3Bucket_LifecycleExpireMarkerOnly -timeout 120m
=== RUN   TestAccAWSS3Bucket_LifecycleExpireMarkerOnly
=== PAUSE TestAccAWSS3Bucket_LifecycleExpireMarkerOnly
=== CONT  TestAccAWSS3Bucket_LifecycleExpireMarkerOnly
--- PASS: TestAccAWSS3Bucket_LifecycleExpireMarkerOnly (54.82s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	54.863s

It looks like that failure is an S3 eventual consistency problem.

@ewbankkit ewbankkit requested a review from a team August 30, 2019 21:35
@ghost ghost added size/L Managed by automation to categorize the size of a PR. documentation Introduces or discusses updates to documentation. service/s3 Issues and PRs that pertain to the s3 service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Aug 30, 2019
@ewbankkit ewbankkit mentioned this pull request Aug 30, 2019
Closed
@bflad bflad added the enhancement Requests to existing resources that expand the functionality or scope. label Aug 31, 2019
@ghost ghost added size/XL Managed by automation to categorize the size of a PR. and removed size/L Managed by automation to categorize the size of a PR. labels Aug 31, 2019
@ewbankkit ewbankkit force-pushed the issue-7158-MkII branch 2 times, most recently from 9dcbb25 to 57f8c06 Compare September 2, 2019 22:23
@ewbankkit ewbankkit changed the title [WIP] Add support for S3 Object Lock legal holds Add support for S3 Object Lock legal holds, retention modes and retention periods Sep 2, 2019
ewbankkit
ewbankkit commented Sep 2, 2019
View reviewed changes
aws/resource_aws_s3_bucket.go
@@ -567,8 +567,8 @@ func resourceAwsS3Bucket() *schema.Resource {
Type: schema.TypeString,
Required: true,
ValidateFunc: validation.StringInSlice([]string{
s3.ObjectLockModeGovernance,
Copy link
Contributor Author

@ewbankkit ewbankkit Sep 2, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used the correct enumeration here (though the values are identical).

@ewbankkit
Copy link
Contributor Author

@bflad This is ready for review now.
Simplified from the original PR - #7179 - and also included changes for #7159.

@bflad bflad self-assigned this Sep 4, 2019
@ewbankkit
Copy link
Contributor Author

Re-ran acceptance tests:

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSS3BucketObject_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 20 -run=TestAccAWSS3BucketObject_ -timeout 120m
=== RUN   TestAccAWSS3BucketObject_noNameNoKey
=== PAUSE TestAccAWSS3BucketObject_noNameNoKey
=== RUN   TestAccAWSS3BucketObject_empty
=== PAUSE TestAccAWSS3BucketObject_empty
=== RUN   TestAccAWSS3BucketObject_source
=== PAUSE TestAccAWSS3BucketObject_source
=== RUN   TestAccAWSS3BucketObject_content
=== PAUSE TestAccAWSS3BucketObject_content
=== RUN   TestAccAWSS3BucketObject_etagEncryption
=== PAUSE TestAccAWSS3BucketObject_etagEncryption
=== RUN   TestAccAWSS3BucketObject_contentBase64
=== PAUSE TestAccAWSS3BucketObject_contentBase64
=== RUN   TestAccAWSS3BucketObject_withContentCharacteristics
=== PAUSE TestAccAWSS3BucketObject_withContentCharacteristics
=== RUN   TestAccAWSS3BucketObject_updates
=== PAUSE TestAccAWSS3BucketObject_updates
=== RUN   TestAccAWSS3BucketObject_updateSameFile
=== PAUSE TestAccAWSS3BucketObject_updateSameFile
=== RUN   TestAccAWSS3BucketObject_updatesWithVersioning
=== PAUSE TestAccAWSS3BucketObject_updatesWithVersioning
=== RUN   TestAccAWSS3BucketObject_kms
=== PAUSE TestAccAWSS3BucketObject_kms
=== RUN   TestAccAWSS3BucketObject_sse
=== PAUSE TestAccAWSS3BucketObject_sse
=== RUN   TestAccAWSS3BucketObject_acl
=== PAUSE TestAccAWSS3BucketObject_acl
=== RUN   TestAccAWSS3BucketObject_metadata
=== PAUSE TestAccAWSS3BucketObject_metadata
=== RUN   TestAccAWSS3BucketObject_storageClass
=== PAUSE TestAccAWSS3BucketObject_storageClass
=== RUN   TestAccAWSS3BucketObject_tags
=== PAUSE TestAccAWSS3BucketObject_tags
=== RUN   TestAccAWSS3BucketObject_tagsLeadingSlash
=== PAUSE TestAccAWSS3BucketObject_tagsLeadingSlash
=== RUN   TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithNone
=== PAUSE TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithNone
=== RUN   TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithOn
=== PAUSE TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithOn
=== RUN   TestAccAWSS3BucketObject_ObjectLockRetentionStartWithNone
=== PAUSE TestAccAWSS3BucketObject_ObjectLockRetentionStartWithNone
=== RUN   TestAccAWSS3BucketObject_ObjectLockRetentionStartWithSet
=== PAUSE TestAccAWSS3BucketObject_ObjectLockRetentionStartWithSet
=== CONT  TestAccAWSS3BucketObject_noNameNoKey
=== CONT  TestAccAWSS3BucketObject_ObjectLockRetentionStartWithSet
=== CONT  TestAccAWSS3BucketObject_ObjectLockRetentionStartWithNone
=== CONT  TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithOn
=== CONT  TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithNone
=== CONT  TestAccAWSS3BucketObject_tagsLeadingSlash
=== CONT  TestAccAWSS3BucketObject_tags
=== CONT  TestAccAWSS3BucketObject_storageClass
=== CONT  TestAccAWSS3BucketObject_metadata
=== CONT  TestAccAWSS3BucketObject_acl
=== CONT  TestAccAWSS3BucketObject_sse
=== CONT  TestAccAWSS3BucketObject_kms
=== CONT  TestAccAWSS3BucketObject_updatesWithVersioning
=== CONT  TestAccAWSS3BucketObject_updateSameFile
=== CONT  TestAccAWSS3BucketObject_updates
=== CONT  TestAccAWSS3BucketObject_withContentCharacteristics
=== CONT  TestAccAWSS3BucketObject_contentBase64
=== CONT  TestAccAWSS3BucketObject_etagEncryption
=== CONT  TestAccAWSS3BucketObject_content
=== CONT  TestAccAWSS3BucketObject_source
--- PASS: TestAccAWSS3BucketObject_noNameNoKey (7.37s)
=== CONT  TestAccAWSS3BucketObject_empty
--- PASS: TestAccAWSS3BucketObject_source (48.09s)
--- PASS: TestAccAWSS3BucketObject_contentBase64 (48.60s)
--- PASS: TestAccAWSS3BucketObject_content (48.60s)
--- PASS: TestAccAWSS3BucketObject_withContentCharacteristics (48.76s)
--- PASS: TestAccAWSS3BucketObject_sse (49.55s)
--- PASS: TestAccAWSS3BucketObject_etagEncryption (50.05s)
--- PASS: TestAccAWSS3BucketObject_empty (43.23s)
--- PASS: TestAccAWSS3BucketObject_kms (73.62s)
--- PASS: TestAccAWSS3BucketObject_updateSameFile (76.85s)
--- PASS: TestAccAWSS3BucketObject_updates (77.17s)
--- PASS: TestAccAWSS3BucketObject_updatesWithVersioning (77.49s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithOn (77.50s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockRetentionStartWithNone (99.82s)
--- PASS: TestAccAWSS3BucketObject_metadata (100.49s)
--- PASS: TestAccAWSS3BucketObject_acl (101.44s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithNone (103.39s)
--- PASS: TestAccAWSS3BucketObject_tagsLeadingSlash (123.34s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockRetentionStartWithSet (123.54s)
--- PASS: TestAccAWSS3BucketObject_tags (123.63s)
--- PASS: TestAccAWSS3BucketObject_storageClass (144.45s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	144.484s
$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSS3Bucket_forceDestroy'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 20 -run=TestAccAWSS3Bucket_forceDestroy -timeout 120m
=== RUN   TestAccAWSS3Bucket_forceDestroy
=== PAUSE TestAccAWSS3Bucket_forceDestroy
=== RUN   TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled
=== PAUSE TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled
=== CONT  TestAccAWSS3Bucket_forceDestroy
=== CONT  TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled
--- PASS: TestAccAWSS3Bucket_forceDestroy (32.79s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled (37.39s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	37.425s

@ghost ghost added size/XXL Managed by automation to categorize the size of a PR. and removed size/XL Managed by automation to categorize the size of a PR. labels Sep 6, 2019
@ewbankkit
Copy link
Contributor Author

@bflad OK, ready for re-review. Thanks.

nywilken
nywilken reviewed Sep 18, 2019
View reviewed changes
Copy link
Contributor

@nywilken nywilken left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ewbankkit this looks good to me. I have slight nit but nothing that would prevent a merge. I'm running the tests to verify everything is passing as expected. I'll take another look in the morning before approving and merging.

aws/resource_aws_s3_bucket.go Outdated Show resolved Hide resolved
aws/resource_aws_s3_bucket.go Outdated Show resolved Hide resolved
@ewbankkit
Copy link
Contributor Author

Re-ran aws_s3_bucket resource acceptance tests:

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSS3Bucket_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 5 -run=TestAccAWSS3Bucket_ -timeout 120m
=== RUN   TestAccAWSS3Bucket_basic
=== PAUSE TestAccAWSS3Bucket_basic
=== RUN   TestAccAWSS3Bucket_Bucket_EmptyString
=== PAUSE TestAccAWSS3Bucket_Bucket_EmptyString
=== RUN   TestAccAWSS3Bucket_tagsWithNoSystemTags
=== PAUSE TestAccAWSS3Bucket_tagsWithNoSystemTags
=== RUN   TestAccAWSS3Bucket_tagsWithSystemTags
=== PAUSE TestAccAWSS3Bucket_tagsWithSystemTags
=== RUN   TestAccAWSS3Bucket_namePrefix
=== PAUSE TestAccAWSS3Bucket_namePrefix
=== RUN   TestAccAWSS3Bucket_generatedName
=== PAUSE TestAccAWSS3Bucket_generatedName
=== RUN   TestAccAWSS3Bucket_region
=== PAUSE TestAccAWSS3Bucket_region
=== RUN   TestAccAWSS3Bucket_acceleration
=== PAUSE TestAccAWSS3Bucket_acceleration
=== RUN   TestAccAWSS3Bucket_RequestPayer
=== PAUSE TestAccAWSS3Bucket_RequestPayer
=== RUN   TestAccAWSS3Bucket_Policy
=== PAUSE TestAccAWSS3Bucket_Policy
=== RUN   TestAccAWSS3Bucket_UpdateAcl
=== PAUSE TestAccAWSS3Bucket_UpdateAcl
=== RUN   TestAccAWSS3Bucket_Website_Simple
=== PAUSE TestAccAWSS3Bucket_Website_Simple
=== RUN   TestAccAWSS3Bucket_WebsiteRedirect
=== PAUSE TestAccAWSS3Bucket_WebsiteRedirect
=== RUN   TestAccAWSS3Bucket_WebsiteRoutingRules
=== PAUSE TestAccAWSS3Bucket_WebsiteRoutingRules
=== RUN   TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical
=== PAUSE TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical
=== RUN   TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed
=== PAUSE TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed
=== RUN   TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled
=== PAUSE TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled
=== RUN   TestAccAWSS3Bucket_shouldFailNotFound
=== PAUSE TestAccAWSS3Bucket_shouldFailNotFound
=== RUN   TestAccAWSS3Bucket_Versioning
=== PAUSE TestAccAWSS3Bucket_Versioning
=== RUN   TestAccAWSS3Bucket_Cors_Update
=== PAUSE TestAccAWSS3Bucket_Cors_Update
=== RUN   TestAccAWSS3Bucket_Cors_Delete
=== PAUSE TestAccAWSS3Bucket_Cors_Delete
=== RUN   TestAccAWSS3Bucket_Cors_EmptyOrigin
=== PAUSE TestAccAWSS3Bucket_Cors_EmptyOrigin
=== RUN   TestAccAWSS3Bucket_Logging
=== PAUSE TestAccAWSS3Bucket_Logging
=== RUN   TestAccAWSS3Bucket_LifecycleBasic
=== PAUSE TestAccAWSS3Bucket_LifecycleBasic
=== RUN   TestAccAWSS3Bucket_LifecycleExpireMarkerOnly
=== PAUSE TestAccAWSS3Bucket_LifecycleExpireMarkerOnly
=== RUN   TestAccAWSS3Bucket_Replication
=== PAUSE TestAccAWSS3Bucket_Replication
=== RUN   TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation
=== PAUSE TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation
=== RUN   TestAccAWSS3Bucket_ReplicationWithoutStorageClass
=== PAUSE TestAccAWSS3Bucket_ReplicationWithoutStorageClass
=== RUN   TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError
=== PAUSE TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError
=== RUN   TestAccAWSS3Bucket_ReplicationWithoutPrefix
=== PAUSE TestAccAWSS3Bucket_ReplicationWithoutPrefix
=== RUN   TestAccAWSS3Bucket_ReplicationSchemaV2
=== PAUSE TestAccAWSS3Bucket_ReplicationSchemaV2
=== RUN   TestAccAWSS3Bucket_objectLock
=== PAUSE TestAccAWSS3Bucket_objectLock
=== RUN   TestAccAWSS3Bucket_forceDestroy
=== PAUSE TestAccAWSS3Bucket_forceDestroy
=== RUN   TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled
=== PAUSE TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled
=== CONT  TestAccAWSS3Bucket_basic
=== CONT  TestAccAWSS3Bucket_Versioning
=== CONT  TestAccAWSS3Bucket_shouldFailNotFound
=== CONT  TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled
=== CONT  TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed
=== CONT  TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical
--- PASS: TestAccAWSS3Bucket_shouldFailNotFound (22.35s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed (43.92s)
=== CONT  TestAccAWSS3Bucket_WebsiteRoutingRules
--- PASS: TestAccAWSS3Bucket_basic (44.02s)
=== CONT  TestAccAWSS3Bucket_WebsiteRedirect
--- PASS: TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled (73.46s)
=== CONT  TestAccAWSS3Bucket_Website_Simple
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical (75.11s)
=== CONT  TestAccAWSS3Bucket_UpdateAcl
--- PASS: TestAccAWSS3Bucket_Versioning (104.09s)
=== CONT  TestAccAWSS3Bucket_Policy
--- PASS: TestAccAWSS3Bucket_WebsiteRoutingRules (77.59s)
=== CONT  TestAccAWSS3Bucket_RequestPayer
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (107.69s)
=== CONT  TestAccAWSS3Bucket_acceleration
--- PASS: TestAccAWSS3Bucket_UpdateAcl (73.27s)
=== CONT  TestAccAWSS3Bucket_region
--- PASS: TestAccAWSS3Bucket_Website_Simple (106.88s)
=== CONT  TestAccAWSS3Bucket_generatedName
--- PASS: TestAccAWSS3Bucket_RequestPayer (76.33s)
=== CONT  TestAccAWSS3Bucket_tagsWithNoSystemTags
--- PASS: TestAccAWSS3Bucket_Policy (101.48s)
=== CONT  TestAccAWSS3Bucket_Bucket_EmptyString
--- PASS: TestAccAWSS3Bucket_region (50.15s)
=== CONT  TestAccAWSS3Bucket_namePrefix
--- PASS: TestAccAWSS3Bucket_generatedName (44.27s)
=== CONT  TestAccAWSS3Bucket_ReplicationWithoutStorageClass
--- PASS: TestAccAWSS3Bucket_acceleration (75.71s)
=== CONT  TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled
--- PASS: TestAccAWSS3Bucket_Bucket_EmptyString (43.83s)
=== CONT  TestAccAWSS3Bucket_forceDestroy
--- PASS: TestAccAWSS3Bucket_namePrefix (44.19s)
=== CONT  TestAccAWSS3Bucket_objectLock
--- PASS: TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled (43.70s)
=== CONT  TestAccAWSS3Bucket_ReplicationSchemaV2
--- PASS: TestAccAWSS3Bucket_forceDestroy (40.15s)
=== CONT  TestAccAWSS3Bucket_ReplicationWithoutPrefix
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (100.36s)
=== CONT  TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError
--- PASS: TestAccAWSS3Bucket_tagsWithNoSystemTags (138.19s)
=== CONT  TestAccAWSS3Bucket_LifecycleBasic
--- PASS: TestAccAWSS3Bucket_objectLock (78.92s)
=== CONT  TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation
--- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (52.93s)
=== CONT  TestAccAWSS3Bucket_tagsWithSystemTags
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutPrefix (99.91s)
=== CONT  TestAccAWSS3Bucket_Replication
--- PASS: TestAccAWSS3Bucket_LifecycleBasic (107.41s)
=== CONT  TestAccAWSS3Bucket_LifecycleExpireMarkerOnly
--- PASS: TestAccAWSS3Bucket_LifecycleExpireMarkerOnly (76.15s)
=== CONT  TestAccAWSS3Bucket_Logging
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (201.27s)
=== CONT  TestAccAWSS3Bucket_Cors_EmptyOrigin
--- PASS: TestAccAWSS3Bucket_tagsWithSystemTags (178.66s)
=== CONT  TestAccAWSS3Bucket_Cors_Delete
--- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (312.08s)
=== CONT  TestAccAWSS3Bucket_Cors_Update
--- PASS: TestAccAWSS3Bucket_Logging (65.15s)
--- PASS: TestAccAWSS3Bucket_Cors_EmptyOrigin (44.70s)
--- PASS: TestAccAWSS3Bucket_Cors_Delete (37.23s)
--- PASS: TestAccAWSS3Bucket_Cors_Update (79.13s)
--- PASS: TestAccAWSS3Bucket_Replication (324.03s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	713.733s

nywilken
nywilken approved these changes Sep 18, 2019
View reviewed changes
Copy link
Contributor

@nywilken nywilken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the pull request! This is ready to go. Nicely done, and thank you for the quick review turnaround. I left a comment about naming for the future.

Data source

--- PASS: TestAccDataSourceAWSS3BucketObjects_basic (62.24s)                                                                                                                                                                                                                                                                                                        
--- PASS: TestAccDataSourceAWSS3BucketObjects_startAfter (62.39s)                                                                                                                                                                                                                                                                                                   
--- PASS: TestAccDataSourceAWSS3BucketObjects_encoded (62.98s)                                                                                                                                                                                                                                                                                                      
--- PASS: TestAccDataSourceAWSS3BucketObject_readableBody (63.74s)                                                                                                                                                                                                                                                                                                  
--- PASS: TestAccDataSourceAWSS3BucketObjects_all (63.88s)                                                                                                                                                                                                                                                                                                          
--- PASS: TestAccDataSourceAWSS3BucketObjects_fetchOwner (64.07s)                                                                                                                                                                                                                                                                                                   
--- PASS: TestAccDataSourceAWSS3BucketObjects_prefixes (64.25s)                                                                                                                                                                                                                                                                                                     
--- PASS: TestAccDataSourceAWSS3BucketObjects_maxKeys (64.51s)                                                                                                                                                                                                                                                                                                      
--- PASS: TestAccDataSourceAWSS3BucketObject_basic (64.52s)                                                                                                                                                                                                                                                                                                         
--- PASS: TestAccDataSourceAWSS3BucketObject_ObjectLockLegalHoldOn (66.58s)                                                                                                                                                                                                                                                                                         
--- PASS: TestAccDataSourceAWSS3BucketObject_allParams (67.14s)                                                                                                                                                                                                                                                                                                     
--- PASS: TestAccDataSourceAWSS3BucketObject_ObjectLockLegalHoldOff (67.20s)                                                                                                                                                                                                                                                                                        
--- PASS: TestAccDataSourceAWSS3BucketObject_kmsEncrypted (88.80s)                                                                                                                                                                                                                                                                                                  
PASS

Resource aws_s3_bucket_object only
I confirmed all aws_s3_bucket tests were also passing. I am excluding to not make this any longer 😄

--- PASS: TestAccAWSS3BucketObject_withContentCharacteristics (42.46s)
--- PASS: TestAccAWSS3BucketObject_content (42.63s)
--- PASS: TestAccAWSS3BucketObject_contentBase64 (42.92s)
--- PASS: TestAccAWSS3BucketObject_etagEncryption (43.23s)
--- PASS: TestAccAWSS3BucketObject_sse (43.31s)
--- PASS: TestAccAWSS3BucketObject_source (43.68s)
--- PASS: TestAccAWSS3BucketObject_empty (40.56s)
--- PASS: TestAccAWSS3BucketObject_updatesWithVersioning (68.06s)
--- PASS: TestAccAWSS3BucketObject_kms (68.37s)
--- PASS: TestAccAWSS3BucketObject_updateSameFile (69.84s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithOn (70.31s)
--- PASS: TestAccAWSS3BucketObject_updates (70.72s)
--- PASS: TestAccAWSS3BucketObject_metadata (89.07s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockRetentionStartWithNone (94.71s)
--- PASS: TestAccAWSS3BucketObject_acl (95.03s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithNone (96.01s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockRetentionStartWithSet (116.73s)
--- PASS: TestAccAWSS3BucketObject_tags (116.76s)
--- PASS: TestAccAWSS3BucketObject_tagsLeadingSlash (116.92s)
--- PASS: TestAccAWSS3BucketObject_storageClass (136.38s)

Sweeper

> go test ./aws -v -sweep=us-west-2 -sweep-run=aws_s3_bucket_object -timeout 10h                                                                                                                                                     [3267] 
2019/09/18 15:36:29 [DEBUG] Running Sweepers for region (us-west-2):
2019/09/18 15:36:29 [INFO] Building AWS auth structure
2019/09/18 15:36:29 [INFO] Setting AWS metadata API timeout to 100ms
2019/09/18 15:36:30 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2019/09/18 15:36:30 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
2019/09/18 15:36:30 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2019/09/18 15:36:30 [DEBUG] Trying to get account information via sts:GetCallerIdentity
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: 187416307283-awsmacietrail-dataevent
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: aws-athena-query-results-187416307283-us-east-1
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: aws-glue-scripts-187416307283-us-west-2
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: aws-glue-temporary-187416307283-us-west-2
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: aws-logs-187416307283-us-east-2
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: aws-logs-187416307283-us-west-2
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: cf-templates-oiwusxkzrgc2-us-east-2
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: cf-templates-oiwusxkzrgc2-us-west-2
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: codepipeline-us-east-1-951157624913
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: codepipeline-us-west-2-667973318359
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: dve1hlgecfc7zqfp
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: elasticbeanstalk-us-east-1-187416307283
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: elasticbeanstalk-us-east-2-187416307283
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: elasticbeanstalk-us-west-2-187416307283
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: fileset-testing20190905121318114700000001
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: hashicorp-rosemary
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: hc-vmware
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: hc-vmware-eu-central-1
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: jbardin-test-bucket-123456
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: jbardin-tfstate-bar
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: jbardin-tfstate-baz
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: klaemmity-tfstate
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: kops-tfacc-f76ff092f401e16c
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: matkins-codepipeline-test
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: matthew-test11
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: mybucket-backup.6479347118024743098
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: ptyng-test-install
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: terraform-remote-s3-test-5c6f5cb7-0
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: terraform-smoke-builds
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-alex-preview
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-alex-preview2
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-athena-db-7nhx0-4768609397815105539
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-deploy-alex
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-globalaccelerator-accelerator-20190524111053595100000001
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-globalaccelerator-accelerator-20190908084652883200000001
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-io-dev
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-k8s-acc-test
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-macie-test-bucket-921738264640544542
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-objects-test-bucket-959887671659694398
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-pipeline-thumb-5069129394962874975
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-redshift-logging-5085425770880693968
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-redshift-logging-5537609857990134597
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-redshift-logging-9209707503695989768
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket: tf-spot-datafeed-3152131621798213916
2019/09/18 15:36:31 [INFO] Skipping S3 Bucket (tf-test-bucket-1015001844523492068) in different region: eu-west-1
2019/09/18 15:36:32 [INFO] Skipping S3 Bucket (tf-test-bucket-2462321507307938293) in different region: eu-west-1
2019/09/18 15:36:32 [INFO] Skipping S3 Bucket (tf-test-bucket-7400895003238920446) in different region: eu-west-1
2019/09/18 15:36:32 [INFO] Skipping S3 Bucket (tf-test-bucket-8842679965349041616) in different region: eu-west-1
2019/09/18 15:36:33 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-1797137958034649312) in different region: eu-west-1
2019/09/18 15:36:33 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-1894199430597971175) in different region: eu-west-1
2019/09/18 15:36:34 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-2585329347318042546) in different region: eu-west-1
2019/09/18 15:36:34 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-2681041731892586355) in different region: eu-west-1
2019/09/18 15:36:34 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-3483181150841905629) in different region: eu-west-1
2019/09/18 15:36:35 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-5069795270948682873) in different region: eu-west-1
2019/09/18 15:36:35 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-5131017677431567128) in different region: eu-west-1
2019/09/18 15:36:35 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-6763539745773117567) in different region: eu-west-1
2019/09/18 15:36:36 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-6907905928609493092) in different region: eu-west-1
2019/09/18 15:36:36 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-7230987159112743782) in different region: eu-west-1
2019/09/18 15:36:36 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-7604800632528659999) in different region: eu-west-1
2019/09/18 15:36:37 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-8204856786873725638) in different region: eu-west-1
2019/09/18 15:36:37 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-830397826573218978) in different region: eu-west-1
2019/09/18 15:36:38 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-848967846571865386) in different region: eu-west-1
2019/09/18 15:36:38 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-8657480719510764401) in different region: eu-west-1
2019/09/18 15:36:38 [INFO] Skipping S3 Bucket (tf-test-bucket-destination-957551716302418994) in different region: eu-west-1
2019/09/18 15:36:38 [INFO] Skipping S3 Bucket: tf-transcoder-524808050876013644
2019/09/18 15:36:38 [INFO] Skipping S3 Bucket: tf-transcoding-pipe-5846680540474152421
2019/09/18 15:36:38 [INFO] Skipping S3 Bucket: wafaclg9ik3
2019/09/18 15:36:38 [INFO] Skipping S3 Bucket: wafaclgo6v0
2019/09/18 15:36:38 [INFO] Skipping S3 Bucket: wafaclmtuaw
2019/09/18 15:36:38 [INFO] Skipping S3 Bucket: wafaclovldg
2019/09/18 15:36:38 [INFO] Skipping S3 Bucket: wafaclu8cag
2019/09/18 15:36:38 Sweeper Tests ran:
        - aws_s3_bucket_object
ok      github.com/terraform-providers/terraform-provider-aws/aws       9.451s

aws/data_source_aws_s3_bucket_object_test.go
func TestAccDataSourceAWSS3BucketObject_ObjectLockLegalHoldOn(t *testing.T) {
rInt := acctest.RandInt()
retainUntilDate := time.Now().UTC().AddDate(0, 0, 10).Format(time.RFC3339)
resourceOnlyConf, conf := testAccAWSDataSourceS3ObjectConfig_objectLockLegalHoldOn(rInt, retainUntilDate)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the future it really is hard to gain context around what these variables represent give the function signature. We should try to use more descriptive names here. We should also look at ways for structuring the resource and data source configurations a bit to make tests like this easier. Then again the resource test should cover the testAccCheckAWSS3BucketObjectExists("aws_s3_bucket_object.object", &rObj), case.

No action items here. Just mentioning for future reference.

aws/resource_aws_s3_bucket.go

if err != nil {
return fmt.Errorf("Error S3 Bucket force_destroy error deleting: %s", err)
return fmt.Errorf("error S3 Bucket force_destroy: %s", err)
}

This comment was marked as outdated.

Sign in to view

@nywilken nywilken added this to the v2.29.0 milestone Sep 18, 2019
@nywilken nywilken merged commit 1b8826e into hashicorp:master Sep 18, 2019
nywilken added a commit that referenced this pull request Sep 18, 2019
@nywilken
update CHANGELOG for #9942
862fb09
@ewbankkit ewbankkit deleted the issue-7158-MkII branch September 19, 2019 00:30
@ghost
Copy link

ghost commented Sep 20, 2019

This has been released in version 2.29.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ansoni ansoni mentioned this pull request Sep 21, 2019
Closed
bflad added a commit that referenced this pull request Oct 4, 2019
@bflad
resource/aws_s3_bucket: Prevent infinite deletion recursion with forc…
6803a54 
…e_destroy argument and object keys with empty "directory" prefixes

Reference: #9942

While aws_s3_bucket_object resources cannot create object keys with extra slashes (empty "directory" prefixes, e.g. `//extraleadingslash.txt`), other AWS services and applications using the S3 Bucket can. This problem seems related to the refactoring in #9942, which switched from using the `DeleteObjects` API call (`Key` parameters in request body) to the `DeleteObject` (`Key` parameter in request URI).

We may want to reconsider changing `deleteAllS3ObjectVersions()` back to using `DeleteObjects` for efficiency, catching the returned `Errors` from the response to handle any `PutObjectLegalHold` changes necessary, but I think that is out of scope of this fix.

The previous failing behavior from the new accceptance test was that it would always just run until the testing timeout (never actually able to delete the S3 Bucket).

Output from acceptance testing (`TestAccAWSCloudTrail/Trail/basic` originally displayed this issue before the new acceptance testing):

```
--- PASS: TestAccAWSCloudTrail/Trail/basic (61.14s)

--- PASS: TestAccAWSS3Bucket_acceleration (60.47s)
--- PASS: TestAccAWSS3Bucket_basic (30.85s)
--- PASS: TestAccAWSS3Bucket_Bucket_EmptyString (31.39s)
--- PASS: TestAccAWSS3Bucket_Cors_Delete (28.55s)
--- PASS: TestAccAWSS3Bucket_Cors_EmptyOrigin (32.60s)
--- PASS: TestAccAWSS3Bucket_Cors_Update (51.53s)
--- PASS: TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled (49.39s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed (31.37s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical (57.21s)
--- PASS: TestAccAWSS3Bucket_forceDestroy (25.84s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithEmptyPrefixes (26.06s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled (30.31s)
--- PASS: TestAccAWSS3Bucket_generatedName (28.75s)
--- PASS: TestAccAWSS3Bucket_LifecycleBasic (70.74s)
--- PASS: TestAccAWSS3Bucket_LifecycleExpireMarkerOnly (52.23s)
--- PASS: TestAccAWSS3Bucket_Logging (45.53s)
--- PASS: TestAccAWSS3Bucket_namePrefix (36.83s)
--- PASS: TestAccAWSS3Bucket_objectLock (50.36s)
--- PASS: TestAccAWSS3Bucket_Policy (68.41s)
--- PASS: TestAccAWSS3Bucket_region (29.66s)
--- PASS: TestAccAWSS3Bucket_Replication (209.69s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (136.24s)
--- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (33.68s)
--- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (205.78s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutPrefix (63.72s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (66.43s)
--- PASS: TestAccAWSS3Bucket_RequestPayer (52.27s)
--- PASS: TestAccAWSS3Bucket_shouldFailNotFound (13.76s)
--- PASS: TestAccAWSS3Bucket_tagsWithNoSystemTags (91.66s)
--- PASS: TestAccAWSS3Bucket_tagsWithSystemTags (126.69s)
--- PASS: TestAccAWSS3Bucket_UpdateAcl (49.76s)
--- PASS: TestAccAWSS3Bucket_Versioning (71.84s)
--- PASS: TestAccAWSS3Bucket_Website_Simple (71.35s)
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (71.47s)
--- PASS: TestAccAWSS3Bucket_WebsiteRoutingRules (51.40s)
```
@bflad bflad mentioned this pull request Oct 4, 2019
Merged
@ghost
Copy link

ghost commented Nov 1, 2019

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Nov 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@bflad bflad bflad left review comments

@nywilken nywilken nywilken approved these changes

Assignees

@bflad bflad

Labels
documentation Introduces or discusses updates to documentation. enhancement Requests to existing resources that expand the functionality or scope. service/s3 Issues and PRs that pertain to the s3 service. size/XXL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Milestone
v2.29.0
3 participants
@ewbankkit @bflad @nywilken