feat(event-invocation-type): Add support for Event invocation type

[ejhayes]

Issue #, if available:
This is mentioned in several different issues:
#510
#207
#12

Description of changes:
This adds support for the event invocation type. #508 made it possible to call local lambda using AWS CLI, but only for RequestResponse invocation type. This adds on to the local lambda invocation functionality by allowing Event invocation type.

This allows multiple lambda functions to call other lambda functions without having to wait for them to return (i.e. RequestResponse invocation type).

Usage
Run local lambda service:

samdev local start-lambda --debug

You'll see that local lambda service is at 127.0.0.1:3001:

2018-11-05 15:48:24 Starting the Local Lambda Service. You can now invoke your Lambda 
Functions defined in your template through the endpoint.
2018-11-05 15:48:24 Localhost server is starting up. Multi-threading = True
 * Tip: There are .env files present. Do "pip install python-dotenv" to use them.
2018-11-05 15:48:24  * Running on http://127.0.0.1:3001/ (Press CTRL+C to quit)

Test invocation using CLI:

aws --endpoint-url http://127.0.0.1:3001 lambda invoke --function-name 'MyFunctionName' --payload '{}' --invocation-type Event function_output.txt

This can also be done within a lambda function. Note that the endpoint referenced cannot be 127.0.0.1 since the function code will be running within a container. Docker for Mac has a special hostname for the underlying host machine named host.docker.internal. Additional documentation for that can be found here:

// since this is running in a container, must reference the host machine
// on Mac this is host.docker.internal.
// https://docs.docker.com/docker-for-mac/networking/#i-want-to-connect-from-a-container-to-a-service-on-the-host
var ep = new aws.Endpoint('http://host.docker.internal:3001');

// configure lambda to use local endpoint
var lambda = new aws.Lambda({endpoint: ep});

// invoke using Event InvocationType
lambda.invoke({
	FunctionName: 'MyFunctionName',
	InvocationType: 'Event'
}, (err,data) => {
	if (err) {
		console.log('Error: ', err);
		console.log(err);
	}

	console.log('Function Result: ', JSON.stringify(data));		
	
	callback(null, {statusCode: 200, body: 'Done.'});
})

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[binarymist]

Does this mean we'll no longer get:
invocation-type: Event is not supported. RequestResponse is only supported.
when doing:

  const lambdaParams = {
    InvocationType: 'Event',
    FunctionName: 'myLambdaFunc',
    Payload: {bla}
  };

  const response = lambda.invoke(lambdaParams);

?

[jfuss]

@ejhayes Thanks for adding this! It looks really solid. I have a couple minor comments to address but overall great job! I even pulled and played with it locally.

Could you please rebase on top of develop as well? There were lots of changes introduced to support Layers recently.

[jfuss]

This requestId will not match the requestId in the container. This will be very confusing when reading the output. Can we not add the request id?

Ideally, if we could set the id from SAM CLI in the docker container, it would be much better but last time I checked, their was a way to do this.

[ejhayes]

So I just checked awslambda.js in the docker container and it looks like awsRequestId is set from invokeId. invokeId is set to uuid(). If we could just allow an environment variable to overwrite this value then we could pass it from SAM CLI. I don't know where the code for awslambda.js is coming from otherwise I'd make this change myself:

var fs = require('fs')
var crypto = require('crypto')

var HANDLER = process.argv[2] || process.env.AWS_LAMBDA_FUNCTION_HANDLER || process.env._HANDLER || 'index.handler'
var EVENT_BODY = process.argv[3] || process.env.AWS_LAMBDA_EVENT_BODY ||
  (process.env.DOCKER_LAMBDA_USE_STDIN && fs.readFileSync('/dev/stdin', 'utf8')) || '{}'

var FN_NAME = process.env.AWS_LAMBDA_FUNCTION_NAME || 'test'
var VERSION = process.env.AWS_LAMBDA_FUNCTION_VERSION || '$LATEST'
var MEM_SIZE = process.env.AWS_LAMBDA_FUNCTION_MEMORY_SIZE || '1536'
var TIMEOUT = process.env.AWS_LAMBDA_FUNCTION_TIMEOUT || '300'
var REGION = process.env.AWS_REGION || process.env.AWS_DEFAULT_REGION || 'us-east-1'
var ACCOUNT_ID = process.env.AWS_ACCOUNT_ID || randomAccountId()
var ACCESS_KEY_ID = process.env.AWS_ACCESS_KEY_ID || 'SOME_ACCESS_KEY_ID'
var SECRET_ACCESS_KEY = process.env.AWS_SECRET_ACCESS_KEY || 'SOME_SECRET_ACCESS_KEY'
var SESSION_TOKEN = process.env.AWS_SESSION_TOKEN
var INVOKED_ARN = process.env.AWS_LAMBDA_FUNCTION_INVOKED_ARN || arn(REGION, ACCOUNT_ID, FN_NAME)
var INVOKE_ID = process.env.AWS_LAMBDA_INVOKE_ID || uuid() // TODO: add this line

function consoleLog(str) {
  process.stderr.write(formatConsole(str))
}

function systemLog(str) {
  process.stderr.write(formatSystem(str) + '\n')
}

function systemErr(str) {
  process.stderr.write(formatErr(str) + '\n')
}

function handleResult(resultStr, cb) {
  if (!process.stdout.write('\n' + resultStr + '\n')) {
    process.stdout.once('drain', cb)
  } else {
    process.nextTick(cb)
  }
}

// Don't think this can be done in the Docker image
process.umask(2)

process.env.AWS_LAMBDA_FUNCTION_NAME = FN_NAME
process.env.AWS_LAMBDA_FUNCTION_VERSION = VERSION
process.env.AWS_LAMBDA_FUNCTION_MEMORY_SIZE = MEM_SIZE
process.env.AWS_LAMBDA_LOG_GROUP_NAME = '/aws/lambda/' + FN_NAME
process.env.AWS_LAMBDA_LOG_STREAM_NAME = new Date().toISOString().slice(0, 10).replace(/-/g, '/') +
  '/[' + VERSION + ']' + crypto.randomBytes(16).toString('hex')
process.env.AWS_REGION = REGION
process.env.AWS_DEFAULT_REGION = REGION
process.env._HANDLER = HANDLER

var OPTIONS = {
  initInvokeId: uuid(),
  invokeId: INVOKE_ID, // TODO: update this
  handler: HANDLER,
  suppressInit: true,
  credentials: {
    key: ACCESS_KEY_ID,
    secret: SECRET_ACCESS_KEY,
    session: SESSION_TOKEN,
  },
  eventBody: EVENT_BODY,
  contextObjects: {
    // clientContext: '{}',
    // cognitoIdentityId: undefined,
    // cognitoPoolId: undefined,
  },
  invokedFunctionArn: INVOKED_ARN,
}

[jfuss]

These can be in args as well.

[jfuss]

Is this really needed if we are sending this data back in the next line? My opinion is to remove this.

[jfuss]

Same as the above comment on LOG.debug

[jfuss]

We should add explicit integ test for Invoke with --invocation-type Event.

[jfuss]

We should be mocking out Thread here too and asserting the mock is called correctly.

[jfuss]

@binarymist Yes. This PR is to add support for invoking a local function through sam local start-lambda with an invocation-type of Event.

[ejhayes]

@jfuss thanks for the input. AFK right now but will try to get this stuff addressed later this week.

[jfuss]

@ejhayes Just checking back. Have you gotten a chance to address feedback? Would love to get this into an upcoming release if you have some time to wrap it up.

[sriram-mv]

@ejhayes Hey! this is a cool feature to add, if are you able to address feedback, we can look at merging this in!

[ejhayes]

@thesriram @jfuss somehow this got missed. Let me merge in the latest changes and address the feedback from earlier. I agree, it would definitely be good to get this into an upcoming release!

[klichukb]

Any progress?

[xylesoft]

Hi, has there been any progress with implementing the Event InvocationType in sam? Are there any workarounds for the time being?

[andy318]

Hello,
We need this badly too.
Is there an ETA for the addition of this capability?
Regards...

[hburrows]

Another vote for getting support for Event invocation-type. I really need this. I'm trying to use RequestResponse as a work-around and it's painful

[warrenca]

Hey! Great work! I'd love this feature to be out soon.

[rockaBe]

How about this feature?
@ejhayes do you still own this or should someone take over? Unfortunately my python skills are not the best, but I would like see this feature available.

[ajpetersons]

Would very much appreciate this feature, maybe someone can take over?

[edjames]

Hello. Does anyone know if this PR is being considered for merge?
I really appreciate all the had work that goes into this but any guidance on possible merge ETA would be extremely useful.

Thanks in advance.

[jfuss]

This is still a good feature to add, however this looks like the PR is stalled. I am going to close this but if someone wants to re-pick this up, feel free to publish a new PR.

[bennebbenneb]

@jfuss What needs to be updated before the merge can be made? It looks like @ejhayes addressed the feedback with the latest commit.

4d7950a

[svenemtell]

It's a shame that this great feature has stalled. @jfuss Isn't the feedback addressed as @bennebbenneb suggests?

[moehawamdeh]

This is a much needed feature, any update?

[royassis]

Waiting as well

[kadetXx]

@ejhayes @jfuss It's been about 5 years since this PR was initially opened and after getting an error trying to use the Event invocation type in aws sam locally, I stumbled upon this PR and I'm very surprised that even after five years, this is still an unresolved issue.

Please do something, thanks!

[nicodalfonso]

Here's another vote to support Event invocation locally.

[leonmyr97]

Another vote. Would be awesome if this can be implemented

[AlexTalis]

Here's a workaround I use to call a local lambda asynchronously. Call it over HTTP, wait for a moment for the request to go through, then abort the request and move on. This is good enough for my local testing because it allows the invoked function to continue running, while the calling function can exit and let API gateway request complete normally without timing out. This is Typescript/Node.js, but should be implementable in other languages.

if (process.env.AWS_SAM_LOCAL === 'true') {
  const url = `http://host.docker.internal:3011/2015-03-31/functions/${FunctionName}/invocations`;
  const controller = new AbortController();
  fetch(url, {
    method: 'POST',
    signal: controller.signal,
    body: Payload,
    headers: {
      'Content-Type': 'application/json',
    },
  }).catch(() => {});
  await new Promise<void>(resolve =>
    setTimeout(() => {
      controller.abort();
      resolve();
    }, 1500),
  );
} else {
  await new Lambda().invoke({ FunctionName, InvocationType: 'Event', Payload }).promise();
}

[Taiters]

Does anyone know if this is actively being worked on (e.g. In another PR / issue)?
I'm interested in this too, so if it's not already being looked at, I'd be happy to try putting a PR together