(batch): Batch Compute on Fargate with CfnComputeEnvironment requires instance_role and instance_types

[nagmesh]

AWS Batch on Fargate does not allow instanceRole and instanceType in the API. However, CDK requires this parameter to be present.

Reproduction Steps

from aws_cdk import aws_iam as iam
from aws_cdk import aws_ecs as ecs
from aws_cdk import aws_batch as batch

class FaultBatchStack(core.Stack):

    def __init__(
        self,
        scope: core.Construct,
        construct_id: str,
        **kwargs
    ) -> None:
        super().__init__(scope, construct_id, **kwargs)
        batch_service_role = iam.Role(
            self,
            "BatchServiceRole",
            assumed_by=iam.ServicePrincipal('batch.amazonaws.com'),
        )
        batch_service_role.add_managed_policy(iam.ManagedPolicy.from_aws_managed_policy_name("service-role/AWSBatchServiceRole"))

        batch_ecs_role = iam.Role(
            self,
            "BatchFargateRole",
            assumed_by=iam.ServicePrincipal('ecs.amazonaws.com'),
        )

        batch_cfn_compute = batch.CfnComputeEnvironment(
            self,
            "RdsBatchCompute",
            type='MANAGED',
            service_role = batch_service_role.role_arn,
            compute_resources= batch.CfnComputeEnvironment.ComputeResourcesProperty(
                type = 'FARGATE',
                maxv_cpus=50,
                minv_cpus=0,
                # instance_role = batch_ecs_role.role_arn,
                # instance_types = ['m5.large'],
                subnets = [vpc.private_subnets[0].subnet_id, vpc.private_subnets[1].subnet_id, vpc.private_subnets[2].subnet_id],
            security_group_ids=[rds_sec_group_id]) 
            
        )

The above will fail on synth/diff
However, if you uncomment instance_Type and instance_role it will deploy but fail at CFN level.

What did you expect to happen?

Allow Fargate task to get created without need for instance_type or instance_role

What actually happened?

CDK stack deployed but failed in CloudFormation console with
An error occurred (ClientException) when calling the CreateComputeEnvironment operation: Error executing request, Exception : instanceRole is not applicable for Fargate.,

Environment

• CDK CLI Version : 1.77.0 (build a941c53)
• Framework Version:
• Node.js Version: 14.9.0
• OS : mac 10.15.7
• Language (Version): Python 3.8.6

Other

---

This is 🐛 Bug Report

[dsudduth]

Confirming this bug. Fields that are not required when selecting Fargate are failing the deployment. To mitigate the issue, we had to fallback to using CfnResource as a "Escape Hatch".

See https://docs.aws.amazon.com/cdk/latest/guide/cfn_layer.html

        compute_resources = core.CfnResource(
            self,
            'RdsBatchCompute',
            type='AWS::Batch::ComputeEnvironment',
            properties={
                'Type': 'Managed',
                'ServiceRole': batch_service_role.role_name,
                'ComputeResources': {
                    'Type': 'FARGATE',
                    'MaxvCpus': 50,
                    'SecurityGroupIds': ['sg-0b80c5a2090102d9f'],
                    'Subnets': [
                        vpc.private_subnets[0].subnet_id
                    ]
                }
            }
        )

[iliapolo]

@nagmesh Thanks for reporting this.

The underlying problem is that the CloudFormation schema says its required at the same time it says it shouldn't be supplied when using fargate.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-batch-computeenvironment-computeresources.html#cfn-batch-computeenvironment-computeresources-instancerole

We will address it.

In the meantime, you can remove those two properties:

batch_cfn_compute.add_property_deletion_override('ComputeResources.InstanceRole')
batch_cfn_compute.add_property_deletion_override('ComputeResources.InstanceTypes')

cc @dsudduth

[dsudduth]

@nagmesh Thanks for reporting this.

The underlying problem is that the CloudFormation schema says its required at the same time it says it shouldn't be supplied when using fargate.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-batch-computeenvironment-computeresources.html#cfn-batch-computeenvironment-computeresources-instancerole

We will address it.

In the meantime, you can remove those two properties:

batch_cfn_compute.add_property_deletion_override('ComputeResources.InstanceRole')
batch_cfn_compute.add_property_deletion_override('ComputeResources.InstanceTypes')

cc @dsudduth

Really appreciate the help. Thanks for the workaround!

[abdullahodibat]

Hello,
i am having the same issue. and the addPropertyDeletionOverride is not working also.

  val computeEnvironment = CfnComputeEnvironment.Builder.create(this, "ComputeEnvironment")
    .computeEnvironmentName(s"${context.settings.environment.value}-batch-v2")
    .`type`("MANAGED")
    .serviceRole(serviceRole.getRoleName)
    .computeResources(
      ComputeResourcesProperty.builder()
        .maxvCpus(256)
        .`type`("FARGATE_SPOT")
        .securityGroupIds(cfl(securityGroup.getSecurityGroupId))
        .subnets(cfl(Deployment.VPC.Subnets.Private(context.environment): _*))
        .tags(cfm("Name" -> s"${context.settings.environment.value}-batch"))
        .build()
    )
    .state("ENABLED")
    .build()

  computeEnvironment.addPropertyDeletionOverride("ComputeResources.InstanceRole")
  computeEnvironment.addPropertyDeletionOverride("ComputeResources.InstanceTypes")

and still i get:

[error] (run-main-0) java.lang.NullPointerException: instanceRole is required
[error] java.lang.NullPointerException: instanceRole is required

[iliapolo]

@abdullahodibat You still need to use .instanceRole and .instanceTypes at construction, just use dummy values and remove them afterwards with addPropertyDeletionOverride

[iliapolo]

Should be resolved once we merge #12204.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.