Can't Override BuildEnvironment.type to "ARM_CONTAINER"

[ryanh-ai]

There is no way to set the BuildEnvironment type setting so that you can set environment type to ARM_CONTAINER and use an ARM based build container, even when you set the build image to the ARM container.

Reproduction Steps

from aws_cdk import core
from aws_cdk import aws_codebuild as codebuild
from aws_cdk import aws_s3 as s3
from aws_cdk import aws_efs as efs_
from aws_cdk import aws_ec2 as ec2


class PipelineStack(core.Stack):
    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        vpc = ec2.Vpc.from_lookup(self, "vpc", vpc_name="xxxxxxx-VPC")
        privateSubnets = vpc.private_subnets
        securityGroup = ec2.SecurityGroup.from_security_group_id(
            self, "sg", security_group_id='sg-xxxxxxxx'
        )

        efs = efs_.CfnFileSystem(
            self,
            "codebuildFS",
            encrypted=False,
            performance_mode="generalPurpose",
            throughput_mode="bursting",
        )

        efs_dns = efs.ref + "efs.us-east-2.amazonaws.com"

        mounts = []
        for subnet in privateSubnets:
            mounts.append(efs_.CfnMountTarget(
                self,
                "Mount-" + subnet.subnet_id,
                file_system_id=efs.ref,
                security_groups=[securityGroup.security_group_id],
                subnet_id=subnet.subnet_id,
            ))

        # The code that defines your stack goes here
        github_source = codebuild.Source.git_hub(
            owner="canada4663",
            repo=id,
            webhook=True,  # optional, default: true if `webhookFilteres` were provided, false otherwise
            webhook_filters=[
                codebuild.FilterGroup.in_event_of(
                    codebuild.EventAction.PUSH
                ).and_branch_is("master")
            ],
        )

        artifact_bucket = s3.Bucket(self, id + "-bucket")

        build_env = codebuild.BuildEnvironment(
            privileged=True,
            build_image=codebuild.LinuxBuildImage.AMAZON_LINUX_2_ARM,
        )
        build_env.type='ARM_CONTAINER' ##can't override here

        env_vars = {
            "EFS_DIR": codebuild.BuildEnvironmentVariable(value="/efs"),
            "EFS_DNS": codebuild.BuildEnvironmentVariable(value=efs_dns)
        }

        build = codebuild.Project(
            self,
            id + "-project",
            project_name=id,
            source=github_source,
            environment=build_env,
            environment_variables=env_vars,
            cache=codebuild.Cache.bucket(artifact_bucket),
            vpc=vpc,
            security_groups=[securityGroup],
        )

Error Log

SINGLE_BUILD_CONTAINER_DEAD: Build container found dead before completing the build. Build container died because it was out of memory, or the Docker image is not supported

Upon inspecting the console, it is using the LINUX_CONTAINER instance sizes for ComputeType: BUILD_GENERAL1_LARGE - 15GB and 8vCPU, instead of 16GB and 8vCPU

When inspecting the underlying code you can see that type is hardcoded to "LINUX_CONTAINER" and there is no way to override it.

Environment

• CLI Version : 1.19
• Framework Version: 1.19
• OS : Ubuntu Linux on Host Machine
• Language : Python

Other

---

This is 🐛 Bug Report

[ryanh-ai]

UPDATE: temporary workaround confirmed by adding below "escape hatch" provisions after codebuild.Project(...) is instantiated:

cfn_build = build.node.default_child
cfn_build.add_override("Environment.Type", "ARM_CONTAINER")

This is now appropriately provisioning an ARM_CONTAINER and build is executing appropriately.

[skinny85]

Thanks for reporting @canada4663 ! I'm glad you were able to unblock yourself.

Apparently, we missed the fact that you need to override the type of the image for ARM-based images like AMAZON_LINUX_2_ARM. This definitely requires a fix on our side.

[ryanh-ai]

awesome and thanks.

[samuelkarp]

For Arm builds, the type needs to be ARM_CONTAINER and the only supported computeType is BUILD_GENERAL1_LARGE.

If anyone is looking for a workaround with a codebuild.PipelineProject in Typescript, this is what worked for me:

const armTestProject = new codebuild.PipelineProject(this, 'linux-arm64', {
  buildSpec: myBuildSpec,
  environment: {
    buildImage: codebuild.LinuxBuildImage.AMAZON_LINUX_2_ARM,
    computeType: codebuild.ComputeType.LARGE
  }
})

const cfnArmTestProject = armTestProject.node.defaultChild as codebuild.CfnProject
cfnArmTestProject.addOverride('Properties.Environment.Type','ARM_CONTAINER')

[shellscape]

@skinny85 the associated PR didn't fix the underlying problem here. The workaround in #5517 (comment) is still required for using LinuxBuildImage.fromAsset since LinuxArmBuildImage has no fromAsset method available. Still a giant gaping hole.

[skinny85]

@shellscape #5541 was specifically about fixing codebuild.LinuxBuildImage.AMAZON_LINUX_2_ARM, so it's not surprising that it didn't do anything about the lack of fromAsset() in LinuxArmBuildImage 🙂.

If you need a fromAsset() for the ARM CodeBuild images, I think it would be really easy to add it - I think it's just a matter of copying what's happening in LinuxBuildImage into the LinuxArmBuildImage class, somewhere here.

If you open a Pull Request with that simple change, I'm sure you can quickly get it in the next CDK release.