Enhancement: Step Functions for Express Workflows

[rockycore]

This adds support for AWS Step Functions for Express Workflows, that are ideal for high-volume, event-processing workloads such as IoT data ingestion, streaming data processing and transformation, and mobile application backends. They can run for up to five minutes. Execution history is optionally available in Amazon CloudWatch Logs. Express Workflows employ an at-least-once model, where there is a possibility that an execution might be run more than once. This makes them ideal for orchestrating idempotent actions such as transforming input data and storing via PUT in Amazon DynamoDB. Express Workflow executions are billed by the number of executions, the duration of execution, and the memory consumed.

Changes proposed in this pull request:
resource_aws_sfn_state_machine.go: Add type and logging_configuration arguments
resource_aws_sfn_state_machine_test.go: Extend acceptance tests
sfn_state_machine.html.markdown: Extend documentation

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Relates OR Closes 11348 11374

Release note for CHANGELOG:

IMPROVEMENTS:
Add `type` and `logging_configuration` arguments: aws_sfn_state_machine

Output from acceptance testing:

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSSfnStateMachine'

==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSSfnStateMachine -timeout 120m
=== RUN   TestAccAWSSfnStateMachine_createUpdate
=== PAUSE TestAccAWSSfnStateMachine_createUpdate
=== RUN   TestAccAWSSfnStateMachine_express_createUpdate
=== PAUSE TestAccAWSSfnStateMachine_express_createUpdate
=== RUN   TestAccAWSSfnStateMachine_standard_createUpdate
=== PAUSE TestAccAWSSfnStateMachine_standard_createUpdate
=== RUN   TestAccAWSSfnStateMachine_Tags
=== PAUSE TestAccAWSSfnStateMachine_Tags
=== RUN   TestAccAWSSfnStateMachine_express_LoggingConfiguration
=== PAUSE TestAccAWSSfnStateMachine_express_LoggingConfiguration
=== CONT  TestAccAWSSfnStateMachine_createUpdate
=== CONT  TestAccAWSSfnStateMachine_express_LoggingConfiguration
=== CONT  TestAccAWSSfnStateMachine_Tags
=== CONT  TestAccAWSSfnStateMachine_standard_createUpdate
=== CONT  TestAccAWSSfnStateMachine_express_createUpdate
--- PASS: TestAccAWSSfnStateMachine_standard_createUpdate (82.32s)
--- PASS: TestAccAWSSfnStateMachine_express_createUpdate (96.30s)
--- PASS: TestAccAWSSfnStateMachine_createUpdate (99.12s)
--- PASS: TestAccAWSSfnStateMachine_Tags (109.09s)
--- PASS: TestAccAWSSfnStateMachine_express_LoggingConfiguration (110.80s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       110.860s

...

[grahamjenson]

Suggested change

	if d.HasChange("logging_configuration") && d.Get("type").(string) == sfn.StateMachineTypeExpress {
	if d.HasChange("logging_configuration") {

[grahamjenson]

Suggested change

~> *NOTE:* Logging is only accepted for EXPRESS Workflows. See the [AWS Step Functions Developer Guide](https://docs.aws.amazon.com/step-functions/latest/dg/welcome.html) for more information about enabling Step Function logging.

[grahamjenson]

Suggested change

* `logging_configuration` - (Optional) Defines what execution history events are logged and where they are logged. The logging_configuration parameter is only valid when type is set to EXPRESS. By default, the level is set to OFF. For more information see [Logging Express Workflows](https://docs.aws.amazon.com/step-functions/latest/dg/cw-logs.html) and [Log Levels](https://docs.aws.amazon.com/step-functions/latest/dg/cloudwatch-log-level.html) in the AWS Step Functions User Guide.

* `logging_configuration` - (Optional) Defines what execution history events are logged and where they are logged. By default, the level is set to OFF. For more information see [Logging Express Workflows](https://docs.aws.amazon.com/step-functions/latest/dg/cw-logs.html) and [Log Levels](https://docs.aws.amazon.com/step-functions/latest/dg/cloudwatch-log-level.html) in the AWS Step Functions User Guide.

[grahamjenson]

@rockycore I added a few suggestions for removing the constraint for only logging with express workflows.

[joanayma]

Hi. Is there any estimation for this to be merged? I see it's conflicts with docs, which I could even try to fix. I'm interested on logging configuration. Thanks.

[dkujawski]

I was testing this today. From AWS documentation: https://docs.aws.amazon.com/step-functions/latest/dg/concepts-standard-vs-express.html
"The Type you choose cannot be changed after your state machine has been created."

When running a plan/apply on an existing state machine this implementation advertises that the existing resource will be modified. This is incorrect. It needs to destroy and rebuild a new state machine with the desired type if it needs to be changed.

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create
  ~ update in-place

Terraform will perform the following actions:

  ~ module.base.module.backups.aws_sfn_state_machine.sfn_sm
      type:       "STANDARD" => "EXPRESS"

I am new to the code base. Happy to investigate the fix.

EDIT: Just realized that I was running terraform v11.11 for this test. Will setup a new test with the latest. Unsure if that would change the results.

[manherfo]

@grahamjenson , could you fix the conflicts? I'm very interested in implementing this feature

[grahamjenson]

@manherfo @dkujawski @joanayma I am developing this feature on #12249 where I have fixed the conflicts

[ewbankkit]

Replaced by #12249.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!