Using parser with envelopes.ApiGatewayEnvelope

[walmsles]

Using parser() to parse Event for API Gateway Lambda.
APIGatewayProxyEventModel has version as mandatory attribute
APIGatewayEventRequestContext has authorizer as mandatory attribute

These are not present in ALL Lambda events from API Gateway via Lambda Proxy integration.

What were you trying to accomplish?
Trying to parse payload as a Model according to documentation on parser

Expected Behavior

To correctly parse the API Gateway Event passed to Lambda without throwing an exception

Current Behavior

Validation exception is thrown for missing fields.

Possible Solution

class APIGatewayEventRequestContext(BaseModel):
    ...
    authorizer: Optional[APIGatewayEventAuthorizer]
    ...

class APIGatewayProxyEventModel(BaseModel):
    version: Optional[str]
    ...

Steps to Reproduce (for bugs)

import json
from aws_lambda_powertools.utilities.parser import BaseModel, envelopes
from aws_lambda_powertools.logging import Logger
from aws_lambda_powertools.utilities.parser.parser import parse

# Specify the API payload Model expected
class UserModel(BaseModel):
    username: str
    password1: str
    password2: str

logger = Logger()

def test(event, context):
    parsed_event = parse(event=event, model=UserModel, envelope=envelopes.ApiGatewayEnvelope)
    response = {
        "statusCode": 200,
        "body": json.dumps(event.dict())
    }

    return response

Environment

• Powertools version used: 1.17.1
• Packaging format (Layers, PyPi): PyPi
• AWS Lambda function runtime: 3.8
• Debugging logs

[ERROR] ValidationError: 2 validation errors for APIGatewayProxyEventModel
version
  field required (type=value_error.missing)
requestContext -> authorizer
  field required (type=value_error.missing)
Traceback (most recent call last):
  File "/var/task/handler.py", line 34, in test
    parsed_event = parse(event=event, model=UserModel, envelope=envelopes.ApiGatewayEnvelope)
  File "/var/task/aws_lambda_powertools/utilities/parser/parser.py", line 148, in parse
    return envelope().parse(data=event, model=model)
  File "/var/task/aws_lambda_powertools/utilities/parser/envelopes/apigw.py", line 30, in parse
    parsed_envelope = APIGatewayProxyEventModel.parse_obj(data)
  File "/var/task/pydantic/main.py", line 578, in parse_obj
    return cls(**obj)
  File "/var/task/pydantic/main.py", line 406, in __init__
    raise validation_error

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.

[heitorlessa]

Hey @walmsles, thanks for flagging it!

Could you give an example of an event that this isn't true so we can flag the official aws docs too?

Happy to accept a PR to make this optional, or work on this right after a Mypy fix I'm working

[walmsles]

Hey have submitted a PR for this (just now :-)).

My PR is consistent with sample events here

The event samples (above and below) are also consistent with output from: sam local generate-event apigateway aws-proxy

Event from API GW:

{
        "resource": "/test",
        "path": "/test",
        "httpMethod": "POST",
        "headers": {
            "Accept": "*/*",
            "Accept-Encoding": "gzip, deflate, br",
            "CloudFront-Forwarded-Proto": "https",
            "CloudFront-Is-Desktop-Viewer": "true",
            "CloudFront-Is-Mobile-Viewer": "false",
            "CloudFront-Is-SmartTV-Viewer": "false",
            "CloudFront-Is-Tablet-Viewer": "false",
            "CloudFront-Viewer-Country": "AU",
            "Content-Type": "application/json",
            "Host": "nlioqcygg3.execute-api.ap-southeast-2.amazonaws.com",
            "Postman-Token": "0d2f11a9-ba6d-49bc-9c1b-795b435592cc",
            "User-Agent": "PostmanRuntime/7.28.0",
            "Via": "1.1 c0abed86b33ff7ae86d6e529296ac06e.cloudfront.net (CloudFront)",
            "X-Amz-Cf-Id": "r3RE7FPFa1B0oq9LlMkCOMKbPUwU5fdD9Zvx6l1GFGExhpgKS8FCXQ==",
            "X-Amzn-Trace-Id": "Root=1-60f286cd-1f3632430b6bbd507a246206",
            "X-Forwarded-For": "124.170.115.3, 130.176.156.116",
            "X-Forwarded-Port": "443",
            "X-Forwarded-Proto": "https"
        },
        "multiValueHeaders": {
            "Accept": [
                "*/*"
            ],
            "Accept-Encoding": [
                "gzip, deflate, br"
            ],
            "CloudFront-Forwarded-Proto": [
                "https"
            ],
            "CloudFront-Is-Desktop-Viewer": [
                "true"
            ],
            "CloudFront-Is-Mobile-Viewer": [
                "false"
            ],
            "CloudFront-Is-SmartTV-Viewer": [
                "false"
            ],
            "CloudFront-Is-Tablet-Viewer": [
                "false"
            ],
            "CloudFront-Viewer-Country": [
                "AU"
            ],
            "Content-Type": [
                "application/json"
            ],
            "Host": [
                "nlioqcygg3.execute-api.ap-southeast-2.amazonaws.com"
            ],
            "Postman-Token": [
                "0d2f11a9-ba6d-49bc-9c1b-795b435592cc"
            ],
            "User-Agent": [
                "PostmanRuntime/7.28.0"
            ],
            "Via": [
                "1.1 c0abed86b33ff7ae86d6e529296ac06e.cloudfront.net (CloudFront)"
            ],
            "X-Amz-Cf-Id": [
                "r3RE7FPFa1B0oq9LlMkCOMKbPUwU5fdD9Zvx6l1GFGExhpgKS8FCXQ=="
            ],
            "X-Amzn-Trace-Id": [
                "Root=1-60f286cd-1f3632430b6bbd507a246206"
            ],
            "X-Forwarded-For": [
                "124.170.115.3, 130.176.156.116"
            ],
            "X-Forwarded-Port": [
                "443"
            ],
            "X-Forwarded-Proto": [
                "https"
            ]
        },
        "queryStringParameters": null,
        "multiValueQueryStringParameters": null,
        "pathParameters": null,
        "stageVariables": null,
        "requestContext": {
            "resourceId": "w845zr",
            "resourcePath": "/test",
            "httpMethod": "POST",
            "extendedRequestId": "CmoAGFq7SwMFQ4A=",
            "requestTime": "17/Jul/2021:07:29:17 +0000",
            "path": "/dev/test",
            "accountId": "674048636348",
            "protocol": "HTTP/1.1",
            "stage": "dev",
            "domainPrefix": "nlioqcygg3",
            "requestTimeEpoch": 1626506957232,
            "requestId": "e227c43b-0957-4354-9f78-b0874f771630",
            "identity": {
                "cognitoIdentityPoolId": null,
                "accountId": null,
                "cognitoIdentityId": null,
                "caller": null,
                "sourceIp": "124.170.115.3",
                "principalOrgId": null,
                "accessKey": null,
                "cognitoAuthenticationType": null,
                "cognitoAuthenticationProvider": null,
                "userArn": null,
                "userAgent": "PostmanRuntime/7.28.0",
                "user": null
            },
            "domainName": "nlioqcygg3.execute-api.ap-southeast-2.amazonaws.com",
            "apiId": "nlioqcygg3"
        },
        "body": "{\n    \"data\": \"test message\"\n}",
        "isBase64Encoded": false
    },
    "timestamp": "2021-07-17 07:29:17,259+0000",
    "service": "service_undefined",
    "xray_trace_id": "1-60f286cd-1f3632430b6bbd507a246206"
}

[heitorlessa]

Staged for next week's release - Friday the latest. Let me know if this is impacting production and I can bring it forward

[walmsles]

All good, not urgent at all. Thanks 👍

[heitorlessa]

hey @walmsles - This is now fixed in 1.18.0

More details in the release notes: https://github.com/awslabs/aws-lambda-powertools-python/releases/tag/v1.18.0