SAM ignoring --env-vars on local invoke

[gmariani]

Description:

This worked maybe a year or two ago but when I revisited an existing project it no longer loads the environmental variables.

I added some details but changed names of things to make it more generic. Folder structure looks like:

C:\Users\USERNAME\Development\PROJECT_NAME\lambda\LAMBDA_NAME
/.aws-sam
/node_modues
/src
- /handlers
- /lib
- app.js
buildspec.yml
CHANGELOG.md
env.json
package.json
package-lock.json
README.md
samconfig.toml
template.yaml

Steps to reproduce:

In my project I have a env.json file with tokens. I then go to test my build via:

sam build && sam local invoke --env-vars env.json

{
    "Parameters": {
        "USERNAME": "redacted",
        "PASS": "redacted",
        "DISCORD_WEBHOOK_ID": "redacted",
        "DISCORD_WEBHOOK_TOKEN": "redacted"
    }
}

Observed result:

None of the parameters in my env.json are populated in process.env.

{
  AWS_LAMBDA_FUNCTION_VERSION: '$LATEST',
  HOSTNAME: 'redacted',
  AWS_SAM_LOCAL: 'true',
  AWS_SESSION_TOKEN: '',
  AWS_LAMBDA_FUNCTION_TIMEOUT: '100',
  AWS_LAMBDA_LOG_GROUP_NAME: 'aws/lambda/ProjectToDiscord',
  LD_LIBRARY_PATH: '/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib',
  LAMBDA_TASK_ROOT: '/var/task',
  AWS_LAMBDA_RUNTIME_API: '127.0.0.1:9001',
  AWS_LAMBDA_LOG_STREAM_NAME: '$LATEST',
  AWS_EXECUTION_ENV: 'AWS_Lambda_nodejs18.x',
  AWS_LAMBDA_FUNCTION_NAME: 'ProjectToDiscord',
  PATH: '/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin',
  TABLE_NAME: 'ProjectCache',
  AWS_DEFAULT_REGION: 'us-east-1',
  PWD: '/var/task',
  AWS_SECRET_ACCESS_KEY: 'redacted',
  LANG: 'en_US.UTF-8',
  LAMBDA_RUNTIME_DIR: '/var/runtime',
  NODE_PATH: '/opt/nodejs/node18/node_modules:/opt/nodejs/node_modules:/var/runtime/node_modules:/var/runtime:/var/task',
  TZ: ':/etc/localtime',
  AWS_REGION: 'us-east-1',
  AWS_LAMBDA_FUNCTION_HANDLER: 'src/handlers/index.handler',
  AWS_ACCESS_KEY_ID: 'redacted',
  SHLVL: '0',
  HOME: '/root',
  AWS_ACCOUNT_ID: 'redacted',
  _HANDLER: 'src/handlers/index.handler',
  AWS_LAMBDA_FUNCTION_MEMORY_SIZE: '128',
  NODE_EXTRA_CA_CERTS: '/var/runtime/ca-cert.pem'
}

Expected result:

To have my parameters from env.json populated in process.env.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Windows 11 Pro 22H2
2. sam --version: SAM CLI, version 1.99.0
3. AWS region: us-east-1
4. node -v: v19.6.1

{
  "version": "1.99.0",
  "system": {
    "python": "3.8.8",
    "os": "Windows-10-10.0.22621-SP0"
  },
  "additional_dependencies": {
    "docker_engine": "24.0.6",
    "aws_cdk": "Not available",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

Add --debug flag to command you are running

C:\Users\USERNAME\Development\PROJECT_NAME\lambda\LAMBDA_NAME>sam build && sam local invoke --env-vars env.json --log-file error.log --debug

2023-10-25 01:07:56,764 | Config file location: C:\Users\USERNAME\Development\PROJECT_NAME\lambda\LAMBDA_NAME\samconfig.toml    
2023-10-25 01:07:56,766 | Loading configuration values from [default.['local', 'invoke'].parameters]
(env.command_name.section) in config file at 'C:\Users\USERNAME\Development\PROJECT_NAME\lambda\LAMBDA_NAME\samconfig.toml'...  
2023-10-25 01:07:56,767 | Configuration values successfully loaded.
2023-10-25 01:07:56,768 | Configuration values are: {}
2023-10-25 01:07:56,771 | Using SAM Template at
C:\Users\USERNAME\Development\PROJECT_NAME\lambda\LAMBDA_NAME\.aws-sam\build\template.yaml
2023-10-25 01:07:56,789 | Using config file: samconfig.toml, config environment: default
2023-10-25 01:07:56,790 | Expand command line arguments to:
2023-10-25 01:07:56,791 |
--template_file=C:\Users\USERNAME\Development\PROJECT_NAME\lambda\LAMBDA_NAME\.aws-sam\build\template.yaml --env_vars=env.json  
--log_file=error.log --no_event --layer_cache_basedir=C:\Users\USERNAME\AppData\Roaming\AWS SAM\layers-pkg
--container_host=localhost --container_host_interface=127.0.0.1
2023-10-25 01:07:56,792 | local invoke command is called
2023-10-25 01:07:56,795 | No Parameters detected in the template
2023-10-25 01:07:56,818 | There is no customer defined id or cdk path defined for resource ProjectCache, so we will use the  
resource logical id as the resource id
2023-10-25 01:07:56,819 | Sam customer defined id is more priority than other IDs. Customer defined id for resource
ProjectToDiscord is ProjectToDiscord
2023-10-25 01:07:56,821 | 0 stacks found in the template
2023-10-25 01:07:56,821 | No Parameters detected in the template
2023-10-25 01:07:56,836 | There is no customer defined id or cdk path defined for resource ProjectCache, so we will use the  
ProjectToDiscord is ProjectToDiscord
2023-10-25 01:07:56,838 | 2 resources found in the stack
2023-10-25 01:07:56,839 | Found Serverless function with name='ProjectToDiscord' and CodeUri='ProjectToDiscord'
2023-10-25 01:07:56,840 | --base-dir is not presented, adjusting uri ProjectToDiscord relative to
C:\Users\USERNAME\Development\PROJECT_NAME\lambda\LAMBDA_NAME\.aws-sam\build\template.yaml
2023-10-25 01:07:56,848 | Found one Lambda function with name 'ProjectToDiscord'
2023-10-25 01:07:56,849 | Invoking src/handlers/index.handler (nodejs18.x)
2023-10-25 01:07:56,849 | Environment variables data found in the CloudFormation parameter file format
2023-10-25 01:07:56,850 | Loading AWS credentials from session with profile 'None'
2023-10-25 01:07:56,857 | Resolving code path. Cwd=C:\Users\USERNAME\Development\PROJECT_NAME\lambda\LAMBDA_NAME\.aws-sam\build,CodeUri=C:\Users\USERNAME\Development\PROJECT_NAME\lambda\LAMBDA_NAME\.aws-sam\build\ProjectToDiscord
2023-10-25 01:07:56,858 | Resolved absolute path to code is
C:\Users\USERNAME\Development\PROJECT_NAME\lambda\LAMBDA_NAME\.aws-sam\build\ProjectToDiscord
2023-10-25 01:07:56,859 | Code C:\Users\USERNAME\Development\PROJECT_NAME\lambda\LAMBDA_NAME\.aws-sam\build\ProjectToDiscord isnot a zip/jar file
2023-10-25 01:07:58,287 | Local image is up-to-date
2023-10-25 01:07:58,295 | Using local image: public.ecr.aws/lambda/nodejs:18-rapid-x86_64.

2023-10-25 01:07:58,296 | Mounting
C:\Users\USERNAME\Development\PROJECT_NAME\lambda\LAMBDA_NAME\.aws-sam\build\ProjectToDiscord as /var/task:ro,delegated, insideruntime container
2023-10-25 01:07:59,201 | Starting a timer for 100 seconds for function 'ProjectToDiscord'
2023-10-25 01:08:36,989 | Cleaning all decompressed code dirs

Also here is my template.yaml

AWSTemplateFormatVersion: "2010-09-09"
Description: Notify status to Discord
Transform: "AWS::Serverless-2016-10-31"
Globals:
  Function:
    Timeout: 100

Resources:
  ProjectCache:
    Type: AWS::Serverless::SimpleTable
    Properties:
      TableName: ProjectCache
  ProjectToDiscord:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: ProjectToDiscord
      Handler: src/handlers/index.handler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: Notify status of Favorite items to Discord
      Policies:
        # https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-policy-template-list.html
        - AWSLambdaBasicExecutionRole
        - DynamoDBCrudPolicy:
            TableName: !Ref ProjectCache
      Environment:
        Variables:
          TABLE_NAME: !Ref ProjectCache
          # USERNAME:
          # PASS:
          # DISCORD_WEBHOOK_ID:
          # DISCORD_WEBHOOK_TOKEN:
      Events:
        ProjectSchedule:
          Type: Schedule
          Properties:
            Schedule: "rate(1 minute)"

[alitarfa]

Hello @gmariani I faced the same issue, and when I left the env variable with "" blank worked.

like this TABLE_NAME: ""

[gmariani]

So I changed template.yaml to:

...
      Environment:
        Variables:
          TABLE_NAME: !Ref ProjectCache
          USERNAME: ""
          PASS: ""
          DISCORD_WEBHOOK_ID: ""
          DISCORD_WEBHOOK_TOKEN: ""
...

And it worked. Maybe this should be documented somewhere as it's not obvious that the variables need to exist to be overwritten instead of added in at runtime.

[hnnasit]

Hi @gmariani, as @alitarfa has mentioned the environment variables need to exist in the template file to be overriden using the --env-vars parameter. This is mentioned in the doc. I do agree the SAM CLI help text and the doc should be updated to reflect this.

[gmariani]

man, i looked all over that documentation and didn't find/notice that page... sigh, well good to know. I looked here https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/using-sam-cli-local-invoke.html and https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-local-invoke.html and both just mention that you pass them along, not that it's overriding anything, hence my confusion. Thanks again everyone!

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.