Automatically update AWS resources with AWS IP Ranges

This project creates Lambda function that automatically create or update AWS resource with AWS service's IP ranges from the ip-ranges.json file.
You can configure which service and region to get range. You can also configure to which resources you want to create or update with those ranges.
Use cases include allowing CloudFront requests, API Gateway requests, Route53 health checker and EC2 IP range (which includes AWS Lambda and CloudWatch Synthetics).
The resources are created or updated in the region where the CloudFormation stack is created.

NOTE ABOUT CloudFront:
There is already a managed VPC Prefix List for CloudFront.
So, doesn't make sense to use this code to create Prefix List for CloudFront. Please, use the managed one.
It does make sense to use this code to handle WAF IPSet for CloudFront.
https://aws.amazon.com/about-aws/whats-new/2022/02/amazon-cloudfront-managed-prefix-list/

NOTE ABOUT Route 53 health checks:
There is already a managed VPC Prefix List for Route 53 health checks.
So, doesn't make sense to use this code to create Prefix List for Route 53 health checks. Please, use the managed ones.
It does make sense to use this code to handle WAF IPSet for Route 53 health checks.
https://aws.amazon.com/about-aws/whats-new/2023/09/amazon-route-53-managed-prefix-lists-health-checks/

NOTE
This is an upgraded version of the repository below. This repo add support for VPC Prefix List and allow you to have resources for more than one service.
https://github.com/aws-samples/aws-waf-ipset-auto-update-aws-ip-ranges

Overview

The CloudFormation template cloudformation/template.yml creates a stack with the following resources:

1. AWS Lambda function with customizable config file called services.json. The function's code is in lambda/update_aws_ip_ranges.py and is written in Python compatible with version 3.12.
2. Lambda function's execution role.
3. SNS subscription and Lambda invocation permissions for the arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged SNS topic.

                          +-----------------+         +---------------------+
                          | Lambda          |         |                     |
                          | Execution Role  |    +--->+AWS WAF IPv4/IPv6 Set|
                          +--------+--------+    |    |                     |
                                   |             |    +---------------------+
                                   |             |
+--------------------+    +--------+--------+    |
|SNS Topic           +--->+ Lambda function +----+
|AmazonIpSpaceChanged|    +--------+--------+    |
+--------------------+             |             |    +-------------------+
                                   |             |    |                   |
                                   v             +--->+AWS VPC Prefix List|
                          +--------+--------+         |                   |
                          | CloudWatch Logs |         +-------------------+
                          +-----------------+

Supported resources

It supports to create or update the following resource:

• WAF IPSet (only WAFv2, WAF Classic is not supported)
• VPC Prefix List

NOTE
If you miss some AWS resource that should be supported, feel free to open an issue or contribute with a pull request.
Keep in mind that you can reference a prefix list in the following AWS resources:

• VPC security groups
• Subnet route tables
• Transit gateway route tables
• AWS Network Firewall rule groups

It will create or update AWS resource, but it will NEVER remove it after it is created.
As soon it is created it can only be updated by this solution.
If you need to remove any resource created by solution you need to do it manually.

Considerations

• Lambda code MUST have a config file called services.json in the root path. See below more details about its format.
• WAF IPSet will just be updated if there are entries to remove or to add.
• VPC Prefix List will just be updated if there are entries to remove or to add.
• When VPC Prefix List is created, the max entries configuration will be the length of current IP ranges for that service plus 10.
• When VPC Prefix List is updated, if current max entries configuration is lower than the length of current IP ranges for that service, it will change the max entries to the length of current IP ranges. If it fails to update, due to size restriction where Prefix List is used, it will NOT update the IP ranges.
• If it fails to create or update resource for any service, the code will not stop, it will continue to handle the other resource and services.
• It only creates resource for service and IP version if there is at least one IP range. Otherwise, it will not create.
• Resources are named as aws-ip-ranges-<SERVICE_NAME>-<IP_VERSION>.
  Where:
  • <SERVICE_NAME> is the service name inside ip-ranges.json file. Converted to lower case and replaced _ with -.
  • <IP_VERSION> is ipv4 or ipv6.

Examples:

• aws-ip-ranges-api-gateway-ipv4
• aws-ip-ranges-route53-healthchecks-ipv4
• aws-ip-ranges-route53-healthchecks-ipv6

NOTE ABOUT REGIONS DEPLOY
There is no reason to deploy this solution twice inside the same region.
If you have a reason for doing it, please open an issue and let's talk about it.

Lambda configuration

To configure which service lambda should handle IP ranges or which region, you need to change the file services.json.

To see the list of possible service names inside ip-ranges.json file, run the command below:

curl -s 'https://ip-ranges.amazonaws.com/ip-ranges.json' | jq -r '.prefixes[] | .service' | sort -u

To see the list of possible region names inside ip-ranges.json file, run the command below:

curl -s 'https://ip-ranges.amazonaws.com/ip-ranges.json' | jq -r '.prefixes[] | .region' | sort -u

See below the file commented.

{
    "Services": [
        {
            # Service name. MUST match the service name inside ip-ranges.json file.
            # Case is sensitive.
            "Name": "API_GATEWAY",
            
            # Region name. It is an array, so you can specify more than one region. MUST match the region name inside ip-ranges.json file.
            # Case is sensitive.
            #
            # Please not that there is one region called GLOBAL inside ip-ranges.json file.
            # If you want to get IP ranges from all region keep the array empty.
            #
            # If you specify more than one region, or keep it empty, it will aggregate the IP ranges from those regions inside the resource at the region where Lambda function is running.
            # It will NOT create the resources on each region specified.
            "Regions": ["sa-east-1"],
            
            "PrefixList": {
                # Indicate if VPC Prefix List should be create for IP ranges from this service. It will be created in the same region where Lambda function is running.
                "Enable": true,
                # Indicate if VPC Prefix List IP ranges should be summarized or not for this specific service.
                "Summarize": true
            },
            
            "WafIPSet": {
                # Indicate if WAF IPSet should be create for IP ranges from this service. It will be created in the same region where Lambda function is running.
                "Enable": true,
                # Indicate if WAF IPSet IP ranges should be summarized or not for this specific service.
                "Summarize": true,
                # WAF IPSet scope to create or update resources. Possible values are ONLY "CLOUDFRONT" and "REGIONAL".
                # Case is sensitive.
                #
                # Note that "CLOUDFRONT" can ONLY be used in North Virginia (us-east-1) region. So, you MUST deploy it on North Virginia (us-east-1) region.
                "Scopes": ["CLOUDFRONT", "REGIONAL"]
            }
        }
    ]
}

Example:

{
    "Services": [
        {
            "Name": "API_GATEWAY",
            "Regions": ["sa-east-1"],
            "PrefixList": {
                "Enable": true,
                "Summarize": true
            },
            "WafIPSet": {
                "Enable": true,
                "Summarize": true,
                "Scopes": ["REGIONAL"]
            }
        },
        {
            "Name": "CLOUDFRONT_ORIGIN_FACING",
            "Regions": [],
            "PrefixList": {
                "Enable": false,
                "Summarize": false
            },
            "WafIPSet": {
                "Enable": true,
                "Summarize": false,
                "Scopes": ["REGIONAL"]
            }
        },
        {
            "Name": "EC2_INSTANCE_CONNECT",
            "Regions": ["sa-east-1"],
            "PrefixList": {
                "Enable": true,
                "Summarize": false
            },
            "WafIPSet": {
                "Enable": true,
                "Summarize": false,
                "Scopes": ["REGIONAL"]
            }
        },
        {
            "Name": "ROUTE53_HEALTHCHECKS",
            "Regions": [],
            "PrefixList": {
                "Enable": false,
                "Summarize": false
            },
            "WafIPSet": {
                "Enable": true,
                "Summarize": false,
                "Scopes": ["REGIONAL"]
            }
        }
    ]
}

Setup

These are the overall steps to deploy:

Setup using CloudFormation

1. Validate CloudFormation template file.
2. Create the CloudFormation stack.
3. Package the Lambda code into a .zip file.
4. Update Lambda function with the packaged code.

Setup using Terraform

1. Initialize Terraform state
2. Validate Terraform template.
3. Apply Terraform template.

After setup

1. Trigger a test Lambda invocation.
2. Reference resources
3. Clean-up

Setup using CloudFormation

To simplify setup and deployment, assign the values to the following variables. Replace the values according to your deployment options.

export AWS_REGION="sa-east-1"
export CFN_STACK_NAME="update-aws-ip-ranges"

IMPORTANT: Please, use AWS CLI v2

1. Validate CloudFormation template

Ensure the CloudFormation template is valid before use it.

aws cloudformation validate-template --template-body file://cloudformation/template.yml

2. Create CloudFormation stack

At this point it will create Lambda function with a dummy code.
You will update it later.

aws cloudformation create-stack --stack-name "${CFN_STACK_NAME}" \
  --capabilities CAPABILITY_IAM \
  --template-body file://cloudformation/template.yml && {
    ### Wait for stack to be created
    aws cloudformation wait stack-create-complete --stack-name "${CFN_STACK_NAME}"
}

If the stack creation fails, troubleshoot by reviewing the stack events. The typical failure reasons are insufficient IAM permissions.

3. Create the packaged code

Before you package it, please change lambda/services.json file according to your requirement. For more information read the section Lambda configuration above.

zip --junk-paths update_aws_ip_ranges.zip lambda/update_aws_ip_ranges.py lambda/services.json

4. Update lambda package code

FUNCTION_NAME=$(aws cloudformation describe-stack-resources --stack-name "${CFN_STACK_NAME}" --query "StackResources[?LogicalResourceId=='LambdaUpdateIPRanges'].PhysicalResourceId" --output text)
aws lambda update-function-code --function-name "${FUNCTION_NAME}" --zip-file fileb://update_aws_ip_ranges.zip --publish

NOTE: Every time you change Lambda function configuration file services.json you need to execute steps 3 and 4 again.

Setup using Terraform

Terraform template uses the following providers:

• aws
• archive

IMPORTANT: Please, use Terraform version 1.3.7 or higher

1. Initialize Terraform state

cd terraform/
terraform init

2. Validate Terraform template

Ensure Terraform template is valid before use it.

terraform validate

3. Apply Terraform template

Before you apply, please change lambda/services.json file according to your requirement. For more information read the section Lambda configuration above.

terraform apply

NOTE: Every time you change Lambda function configuration file services.json you need to execute this step again.

After setup

1a. Trigger a test Lambda invocation with the AWS CLI

After the stack is created, AWS resources are not created or updated until a new SNS message is received. To test the function and create or update AWS resources with the current IP ranges for the first time, do a test invocation with the AWS CLI command below:

CloudFormation

aws lambda invoke \
  --function-name "${FUNCTION_NAME}" \
  --cli-binary-format 'raw-in-base64-out' \
  --payload file://lambda/test_event.json lambda_return.json

Terraform

FUNCTION_NAME=$(terraform output | grep 'lambda_name' | cut -d ' ' -f 3 | tr -d '"')
aws lambda invoke \
  --function-name "${FUNCTION_NAME}" \
  --cli-binary-format 'raw-in-base64-out' \
  --payload file://lambda/test_event.json lambda_return.json

After successful invocation, you should receive the response below with no errors.

{
    "StatusCode": 200,
    "ExecutedVersion": "$LATEST"
}

The content of the lambda_return.json will list all AWS resources created or updated by the Lambda function with IP ranges from configured services.

1b. Trigger a test Lambda invocation with the AWS Console

Alternatively, you can invoke the test event in the AWS Lambda console with sample event below. This event uses a test-hash md5 string that the function parses as a test event.

{
  "Records": [
    {
      "EventVersion": "1.0",
      "EventSubscriptionArn": "arn:aws:sns:EXAMPLE",
      "EventSource": "aws:sns",
      "Sns": {
        "SignatureVersion": "1",
        "Timestamp": "1970-01-01T00:00:00.000Z",
        "Signature": "EXAMPLE",
        "SigningCertUrl": "EXAMPLE",
        "MessageId": "12345678-1234-1234-1234-123456789012",
        "Message": "{\"create-time\": \"yyyy-mm-ddThh:mm:ss+00:00\", \"synctoken\": \"0123456789\", \"md5\": \"test-hash\", \"url\": \"https://ip-ranges.amazonaws.com/ip-ranges.json\"}",
        "Type": "Notification",
        "UnsubscribeUrl": "EXAMPLE",
        "TopicArn": "arn:aws:sns:EXAMPLE",
        "Subject": "TestInvoke"
      }
    }
  ]
}

2. Reference resources

For WAF IPSet, see Using an IP set in a rule group or Web ACL.
For VPC Prefix List, see Reference prefix lists in your AWS resources.

3. Clean-up

Remove the temporary files, remove CloudFormation stack and destroy Terraform resources.

CloudFormation

rm update_aws_ip_ranges.zip
rm lambda_return.json
aws cloudformation delete-stack --stack-name "${CFN_STACK_NAME}"
unset AWS_REGION
unset CFN_STACK_NAME

Terraform

rm lambda_return.json
terraform destroy

ATTENTION
When you remove CloudFormation stack, or destroy Terraform resources, it will NOT remove WAF IPSet or VPC Prefix List created by this solution.
If you want to remove it, you need to do it manually.

Lambda function customization

After the stack is created, you can customize the Lambda function's execution log level by editing the function's environment variables.

• LOG_LEVEL: Optional. Set log level to increase or reduce verbosity. The default value is INFO. Possible values are:
  • CRITICAL
  • ERROR
  • WARNING
  • INFO
  • DEBUG

Troubleshooting

Wrong WAF IPSet Scope

An error occurred (WAFInvalidParameterException) when calling the ListIPSets operation: Error reason: The scope is not valid., field: SCOPE_VALUE, parameter: CLOUDFRONT

Scope name CLOUDFRONT is correct, but it MUST be running on North Virginia (us-east-1) region. If it runs outside North Virginia, you will see the error above.
Please make sure it is running on North Virginia (us-east-1) region.

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.