Enabling Step Function Logging To CloudWatch

[grahamjenson]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

I would like logging from Step Functions to CloudWtch logs enabled as described here https://aws.amazon.com/about-aws/whats-new/2020/02/aws-step-functions-supports-cloudwatch-logs-standard-workflows/

This should include the log_level, include_execution_data, cloud_watch_log_group options

References

https://aws.amazon.com/about-aws/whats-new/2020/02/aws-step-functions-supports-cloudwatch-logs-standard-workflows/

[ewbankkit]

Requires AWS SDK v1.29.10:

• Update module aws/aws-sdk-go to v1.29.12 #12128

[grahamjenson]

Cheers @ewbankkit for the quick reply!

[grahamjenson]

Now that #12128 has been merged any word on this feature?

[grahamjenson]

This looks to be implemented by #11570

[ewbankkit]

@grahamjenson Yes, #11570 seems to address this; The only change required there is in the documentation noting that logging_configuration is only valid for EXPRESS workflows - That can now be removed.

[grahamjenson]

@ewbankkit I have submitted #12249 which is a branch off the previous PR so with the requested changes.

[mmritesh]

By when can we expect this feature to be out??

[michelleyang111]

Hi @DrFaust92, and @ewbankkit, what value should i pass to type and logging_configuration to step function in terraform? I can't find the information online.

[ramakrishna2324]

is there any step in terraform to add logging_configuration to stepfunction

[kevindixon]

Can anyone provide an update on this issue? Any chance of an ETA?

[men6288]

Plus 1 on this one, would be great to have

[ramakrishna2324]

I found a method in cloud formation. using that method to enable log configuration.

[trevjonez]

I ended up using a provisioner "local-exec" and a few template files to workaround this issue.

aws stepfunctions update-state-machine \
  --state-machine-arn '${state-machine-arn}' \
  --logging-configuration='${logging-configuration}' \
  --no-cli-pager

render this json and pass it as the logging-configuration in the bash template.

{
  "level": "ALL",
  "includeExecutionData": true,
  "destinations": [
    {
      "cloudWatchLogsLogGroup": {
        "logGroupArn": "${log-group}:*"
      }
    }
  ]
}

[lu-liu-rft]

I found a method in cloud formation. using that method to enable log configuration.

Hi can you please share the approach by using Cloudwatch? Many thanks.

[lu-liu-rft]

Update request + 1

From the Terraform doc here: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sfn_state_machine
Seems like this issue hasn't been resolved or the doc hasn't been updated.

[lu-liu-rft]

Also can we enable X-ray for stepfunction in Terraform?

[lu-liu-rft]

I ended up using a provisioner "local-exec" and a few template files to workaround this issue.

aws stepfunctions update-state-machine \
  --state-machine-arn '${state-machine-arn}' \
  --logging-configuration='${logging-configuration}' \
  --no-cli-pager

render this json and pass it as the logging-configuration in the bash template.

{
  "level": "ALL",
  "includeExecutionData": true,
  "destinations": [
    {
      "cloudWatchLogsLogGroup": {
        "logGroupArn": "${log-group}:*"
      }
    }
  ]
}

Hi are you using 'null_resource'? Just wondering how you specified triggers parameter? I keep getting error A reference to a resource type must be followed by at least one attributeaccess, specifying the resource name.

[toddlers]

@lu-liu-rft here is the working example with x-ray

resource "aws_sfn_state_machine" "sfn_state_machine" {
  name     = "mystatemachine"
  role_arn = "arn:aws:iam::1234567890:role/service-role/StepFunctions-MyStateMachine-role-a6146d54"

  definition = <<EOF
{
  "Comment": "A Hello World example of the Amazon States Language using an AWS Lambda Function",
  "StartAt": "HelloWorld",
  "States": {
    "HelloWorld": {
      "Type": "Pass",
      "End": true
    }
  }
}
EOF
}

resource "aws_cloudwatch_log_group" "yada" {
  name = "/aws/vendedlogs/states/myloggroup"
}

resource "null_resource" "enable_step_function_logging" {
      triggers = {
    state_machine_arn  = aws_sfn_state_machine.sfn_state_machine.arn
    logs_params=<<PARAMS
    {
        "level":"ALL",
        "includeExecutionData":true,
        "destinations":[
            {
                "cloudWatchLogsLogGroup":{
                    "logGroupArn":"${aws_cloudwatch_log_group.yada.arn}:*"
                    }
                }
            ]
            }
    PARAMS
    }
  provisioner "local-exec" {
    command = <<EOT
set -euo pipefail

aws stepfunctions update-state-machine --state-machine-arn ${self.triggers.state_machine_arn}  --tracing-configuration enabled=true --logging-configuration='${self.triggers.logs_params}'

EOT
    # interpreter = ["bash"]
  }
}

[ewbankkit]

This has been implemented via #12249 and released in v3.27.0 of the Terraform AWS Provider.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!