stepfunctions: resultwriter on jsonata distributed maps fails deployments

[nywhere]

Describe the bug

The following is working with DistributedMaps and JsonPath language:

map = sfn.DistributedMap(
    result_writer=sfn.ResultWriter(
        bucket=s3_bucket, 
        prefix=prefix
    )
)

With JSONata synth is ok, but the deployment fails with:

Resource handler returned message: "Invalid State Machine Definition: 'SCHEMA_VALIDATION_FAILED: These fields are required: [Arguments] at /States/Stages/ItemProcessor/States/Stage/ResultWriter'

Meanwhile step function UI designer does add "Arguments":

"ResultWriter": {
    "Resource": "arn:aws:states:::s3:putObject",
    "Arguments": {
        "Bucket": "mybucket",
        "Prefix": "tmp/logs"
    }
}

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Version

No response

Expected Behavior

ResultWriter should still work with JSONata language

Current Behavior

ResultWriter only works with JsonPath language.

Reproduction Steps

• Create state machine
• Add a distributed map with JSONata language
• Add a ResultWriter
• Deploy

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.178.1 (build ae342cb)

Framework Version

No response

Node.js Version

v18.16.0

OS

OSX 15.3

Language

Python

Language Version

No response

Other information

No response

[pahud]

After analyzing the code and issue, here's what I found:

1. The issue appears to be in the CDK's implementation of the ResultWriter for DistributedMap states when using JSONata language. The ResultWriter class in aws-stepfunctions/lib/states/distributed-map/result-writer.ts doesn't properly handle the required "Arguments" field in the ASL (Amazon States Language) JSON when using JSONata.
2. According to the AWS Step Functions documentation and the error message, when using ResultWriter with JSONata, the ASL should include an "Arguments" field like this:

"ResultWriter": {
    "Resource": "arn:aws:states:::s3:putObject",
    "Arguments": {
        "Bucket": "mybucket",
        "Prefix": "tmp/logs"
    }
}

3. The current CDK implementation appears to be missing this required field structure when using JSONata, while it works correctly with JsonPath.

We need to ensure that when JSONata is enabled, the ResultWriter generates the correct field name.

From the documentation, when using JSONata: [1]

The Arguments and Output fields only support JSONata [2]

The InputPath, Parameters, ResultSelector, ResultPath, OutputPath fields only support JSONPath

Specifically, it states:

The Arguments and Output fields only support JSONata, so it is invalid to use them with workflows that use JSONPath. Conversely, InputPath, Parameters, ResultSelector, ResultPath, OutputPath, and other JSONPath fields are only supported in JSONPath, so it is invalid to use path-based fields when using JSONata as your top level workflow or state query language.

This means that when you opt-in to using JSONata in your Step Functions workflow: [3]

• You must use Arguments instead of Parameters.
• You cannot mix JSONPath fields with JSONata fields in the same workflow
• The syntax for JSONata expressions uses {% %} delimiters

Confirmed this is a bug CDK should fix while workarounds are available.

Short-term workaround: Use JsonPath instead of JSONata for now when using ResultWriter with DistributedMap.
Alternative workaround: Define the state machine using raw ASL JSON if you must use JSONata with ResultWriter.

[pahud]

I'm escalating the team for inputs.

[GavinZZ]

Thanks Pahud for the detailed information and thanks @nywhere for reporting this issue. After deep dive, I think this is a valid issue and we should raise the priority to p1 given that if users use JSONata as query language as the top-level language, there is no workaround for users to use JSONPath for Result Writers or Item Readers.

[GavinZZ]

Closing this issue as a PR to fix this is merged. Pending next release.

[github-actions]

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.