Lambda Layers - New Version Every Run

[angrychimp]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.1.8
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v4.21.0

Affected Resource(s)

• aws_lambda_layer_version

Terraform Configuration Files

terraform {
	backend "s3" {
		bucket = "test-bucket"
		key    = "tfstate"
		region = "us-west-2"
	}
}

provider "aws" {
	alias  = "us-west-2"
	region = "us-west-2"
}

data "aws_s3_object" "test-package" {
    provider = aws.us-west-2
    bucket = "test-bucket"
    key    = "package.zip"
}

resource "aws_lambda_layer_version" "test-layer" {
    provider            = aws.us-west-2
    layer_name          = "test-layer"
    description         = "${substr(data.aws_s3_object.test-package.etag, 0, 8)}"
    compatible_runtimes = ["python3.9"]
    skip_destroy        = true
    s3_bucket           = data.aws_s3_object.test-package.bucket
    s3_key              = data.aws_s3_object.test-package.key
    source_code_hash    = "${base64sha256(data.aws_s3_object.test-package.etag)}"
}

Debug Output

Debug output can be provided upon request, but since I have no idea how to scrub all the various tokens and other sensitive info, I don't want to just broadcast it to the world.

Panic Output

None

Expected Behavior

After first apply, second apply should generate no changes.

Actual Behavior

Each apply creates a new version, reporting that the source_code_hash has changed, even though it has not.

Steps to Reproduce

1. Create a simple layer package, upload to S3
2. terraform apply
3. Repeat step 2 (make no other changes)

Important Factoids

We have external processes that update the ZIP package on S3 asynchronously. We rely on the S3 object etag to let us know when that package has been updated. My assumption is that AWS decompresses the archive file, and compiles a SHA256 checksum of the resulting image/fileset. This will never match a checksum for the compressed archive file.

Either way, I can't seem to find a way to make the checksums match. I'm hoping there's a way to store the source_code_hash provided to the aws_lambda_layer_version with the project state, and compare that each time (rather than relying on the AWS provided checksum).

As a workaround, I've been using the S3 object ETag hash to populate the layer description, which accomplishes the desired functionality. It would be ideal though to be able to rely on the source_code_hash.

References

• Lambda Layers- New Version Every Run #7185

[jornfranke]

Remove the source_code_hash - then it is only published once it has been changed.

Note: If you remove source_code_hash then I expect that it publishes a new version. However, the second apply (and subsequent ones) will not unless your loading process overwrites the existing s3 object (does not matter if it has changed or not). Generally I recommend to get rid of the external asynchronous process for putting the package on S3 and all do it through Terraform. It can introduce all sorts of problems (e.g. updates where there are no updates, unclear if security patches have been deployed or not etc.)

[angrychimp]

@jornfranke If you simple remove the source_code_hash it will no longer attempt to update the layer. There is nothing to tell Terraform that the state has changed, unless you start using a local ZIP file as the source.

I'm aware that it would be beneficial to have the process generating the layer source code as part of the main Terraform collection here, but that's just not possible with this particular application.

The underlying method in use here does not require a source hash for comparison - source_code_hash only exists for Terraform and I'm assuming it's there to monitor for state changes. With the current functionality, it does not accomplish its goal. So we have a workaround, but I would argue the team either fix how source_code_hash works or remove it completely. Currently it's just misleading.

[jornfranke]

@jornfranke If you simple remove the source_code_hash it will no longer attempt to update the layer. There is nothing to tell Terraform that the state has changed, unless you start using a local ZIP file as the source.

I cannot confirm this behaviour you describe. With which version did you test?

[angrychimp]

I'm testing with version 1.2.4

Your comment was:

If you remove source_code_hash then I expect that it publishes a new version.

If this is the very first apply then sure - source_code_hash doesn't matter. Neither do any of the other parameters. Maybe I don't understand what you were trying to say?

My whole point with this issue is that source_code_hash currently serves zero purpose. If your layer source is a locally created archive, Terraform can use state for that resource to determine if a change needs to be applied to the layer. But if you use S3 for your layer source, Terraform has no way of knowing whether or not the object has actually changed, so it will always think there's a change to the layer, and it will create a new version upon each apply. There's no functional purpose for source_code_hash.

Clearly I have a workaround. I'm using it in production. It's fine. But I'd like to see source_code_hash work so that I don't have to put a partial hash into my layer description.

[jornfranke]

Have you tested this? If you put the the object on S3 via s3_bucket_object resource it all works as it should be - there is no need for source_code_hash and also Terraform does not apply it if it does not change, but only if it changes.
This I have also tested with the latest Terraform provider. I assume the Terraform version itself does not matter for this case.

If you put the object outside Terraform - which in any case is a very bad idea (not reproducible pipelines, unclear what is deployed etc.) - you could try the s3_bucket_object datasource. However, I would advise against changing deployments outside the control of the pipeline.

[angrychimp]

I feel like we're drifting off topic here. My whole point is that source_code_hash currently serves no purpose. I'm asking if that can either be fixed, or removed.

[stonefield]

1. I do not think this is S3 specific. It only concern the handling of source_code_hash. I have the same problem with a local file, picked up from data.archive_file.
2. I've looked at the source code and done a simple comparison between aws_lambda_layer_version and aws_lambda_function. It looks like ForceNewshould not be a part of the schema definition. It should be an easy fix.

I assume the line should be removed

terraform-provider-aws/internal/service/lambda/layer_version.go

Line 123 in 1076f59

ForceNew: true,

It has been there since initial implementation of aws_lambda_layer_version

Please prioritize.

[filol]

@justinretzolk Can you assign someone of your team to review this little PR #32535 Thanks !

[github-actions]

This functionality has been released in v5.13.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.