[lambda] specify multiple folders as a single lambda asset

[efimk-lu]

Hello,
Great framework, keep up the good work.

Right now in order to define a lambda function you need to set where the code resides, e.g. code: lambda.Code.asset('lambda'). Sometimes you would like to define multiple folders, for example the actual code and node_modules, although packing can be done as a preprocess step before actually running cdk deploy, it would be cleaner to do it as part of the lambda configuration.

[rix0rrr]

Wouldn't you pick the single directory that contains both your code and the node_modules directory, like this?

.
|-- my_code.ts
`-- node_modules
    `-- some_library

[efimk-lu]

I've been thinking on the following structure

/
... package.json
... Various conf file
... src
...... my_code.js
... test
...... test_my_code.js
... node_modules
... cdk
...... deploy.js

[eladb]

I think that's definitely an interesting idea. One would also need to map the sources to their location in the destination tree (for example, you would probably want src/my_code.js in the root of the destination tree and node_modules next to it, right?

[efimk-lu]

@eladb right

[justinmchase]

I have a mono-repo and so I can't just include the root of the project. I use tsc to compile code from src/ -> dist/, from ts to js. My node_modules exists at the root of the project and I don't want to include all dependencies even.

Something like this would be great:

    const example = new Function(this, 'Example'), {
      runtime: Runtime.NodeJS810,
      handler: 'index.handler',
      code: Code.files([
        'dist/funcs/example/*',
        'node_modules/*'
        '!node_modules/aws-sdk'
    });

Similar to the way gulp would let you include src files.

Right now I'm looking at adding a package.json into my func folder and running npm i --production but its a bit of a pain, I don't have any mechanism for copying non-code files currently.

[eladb]

I can see the value in this, and I think we can support something like that, but I was wondering if this is actually the best approach for bundling node.js dependencies for Lambda. Excluding certain modules from the root of your node_modules directory might really not be the right approach. I'd argue that your npm i --production approach to generate a clean node_modules tree that includes only the deps needed for your runtime code is a much more robust approach.

I wonder if perhaps there's something we can add to the CDK such that utilizing this approach will be straightforward (i.e. run npm install from within the toolkit?)

It's definitely a topic of interest for us, which we would love to address end-to-end (for node and all other languages as well). Copy: @sam-goodwin

[justinmchase]

@eladb I agree after digging into it more.

I wrote a little script to do this, to illustrate the issue:

#!/bin/bash
set -e

function prep-func {
  func=$1

  echo "$func:"
  cp src/funcs/$func/package.json dist/funcs/$func/package.json
  cp src/funcs/$func/package-lock.json dist/funcs/$func/package-lock.json
  pushd dist/funcs/$func      &> /dev/null
  npm i --production --silent
  popd                        &> /dev/null
}

echo "compiling..." 
npx tsc

echo "preparing functions..."
prep-func "shared"
prep-func "exportDelivery"

echo "deploying to amazon..."
npx cdk deploy --app dist/cloud/index.js

echo "done."

I think its fine to let people resolve this on their own on second thought, because I'm not sure how you could really solve this in a general way. These are steps for the pre-deploy build system of each individual project, and if I have a mono-repo I need to ensure that each of those sup-repos are building their dependencies at the correct time with the correct settings, not you.

[nija-at]

still relevant

[shahbour]

Any plan on this feature ?

[eladb]

Perhaps @jogold will be interested to add this capability to our new shiny staging system.

[gabrielpeery]

Hey has there been any movement on this since August? I could really use this feature in my mono-repo.

[zarnoevic]

+1

[bes]

Would love this feature, have a somewhat intricate folder structure and I really don't want to duplicate a lot of files...

[ronnathaniel]

+1

[github-actions]

This issue has received a significant amount of attention so we are automatically upgrading its priority. A member of the community will see the re-prioritization and provide an update on the issue.

[awspbade]

+1

[ahzia]

I have created a draft PR for this, it is not finalized but I appreciate any review or feedback for it.
PR link

[github-anis-snoussi]

+1

[coolbotic]

+1

[talhankoc]

+1

[madeline-k]

#4776 looks to be related to this

[mccauleyp]

+1

[elena-cav]

+1

[mfittko]

so any way around except for using docker for packaging a custom folder structure? +1

[kareldonk]

+1