Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: custom runtimes to define function #1602

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
6 changes: 6 additions & 0 deletions .changeset/long-berries-greet.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@aws-amplify/integration-tests': minor
'@aws-amplify/backend-function': minor
---
MarlonJD marked this conversation as resolved.
Show resolved Hide resolved

add custom provided function support to define function
32 changes: 32 additions & 0 deletions packages/backend-function/src/factory.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import { Policy, PolicyStatement } from 'aws-cdk-lib/aws-iam';
import fsp from 'fs/promises';
import path from 'node:path';
import { AmplifyUserError } from '@aws-amplify/platform-core';
import { NodejsFunction } from 'aws-cdk-lib/aws-lambda-nodejs';

const createStackAndSetContext = (): Stack => {
const app = new App();
Expand Down Expand Up @@ -750,4 +751,35 @@ void describe('AmplifyFunctionFactory', () => {
);
});
});

void describe('provided function runtime property', () => {
void it('sets valid runtime', () => {
const lambda = defineFunction((scope) => {
return new NodejsFunction(scope, 'nodejs-provided', {
entry:
'./packages/backend-function/src/test-assets/default-lambda/handler.ts',
runtime: Runtime.NODEJS_22_X,
});
}).getInstance(getInstanceProps);
const template = Template.fromStack(lambda.stack);

template.hasResourceProperties('AWS::Lambda::Function', {
Runtime: Runtime.NODEJS_22_X.name,
});
});

void it('provided function defaults to oldest runtime', () => {
const lambda = defineFunction((scope) => {
return new NodejsFunction(scope, 'nodejs-provided', {
entry:
'./packages/backend-function/src/test-assets/default-lambda/handler.ts',
});
}).getInstance(getInstanceProps);
const template = Template.fromStack(lambda.stack);

template.hasResourceProperties('AWS::Lambda::Function', {
Runtime: Runtime.NODEJS_16_X.name,
});
});
});
});
110 changes: 45 additions & 65 deletions packages/backend-function/src/factory.ts
Copy link

@arash2060 arash2060 Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up on my overall comment, I'm not sure how we're going to handle environment variables of type Record<string, string | BackendSecret>;. (see line 88 of this file) I see that you can add an environment variable via addEnvironment(key: string, value: string, options?: EnvironmentOptions): this; method of PythonFunction. However, the value cannot be of type BackendSecret

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MarlonJD @arash2060 I am so excited to see this functionality so close! I am new to Amplify and have been enjoying the ease of development...but got hung up royally today trying to figure out how to add a custom query to my graphql schema via a lambda with python 3_11 runtime. This functionality will be well-used and loved as I build out slew of microservices that use LangChain. I am wondering if you would consider deploying without supporting environment variables for now as a way to get this into our hands. From my understanding, that would require adding a permission for the lambda to access the secret store with boto3 and a warning about the additional cost and latency while the full functionality is being built out. Is this feasible? Curious to hear your thoughts.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MarlonJD @arash2060 I am so excited to see this functionality so close! I am new to Amplify and have been enjoying the ease of development...but got hung up royally today trying to figure out how to add a custom query to my graphql schema via a lambda with python 3_11 runtime. This functionality will be well-used and loved as I build out slew of microservices that use LangChain. I am wondering if you would consider deploying without supporting environment variables for now as a way to get this into our hands. From my understanding, that would require adding a permission for the lambda to access the secret store with boto3 and a warning about the additional cost and latency while the full functionality is being built out. Is this feasible? Curious to hear your thoughts.

Hey there! If you want to auto deploy with amplify we should use AWS Secret Manager, it's because secret('foo') using this, you can easily use .env file and manuel deploy without this I can give an example for this, if you have another solution or idea please tell me mayne we can build, I'm thinking publish my amplify fork and use this on amplify auto deploy because of limit issue and this python and golang functions, I'll share with you this examples here If you want to use custom functions on auto build in amplify console.

Original file line number Diff line number Diff line change
@@ -1,8 +1,4 @@
import {
FunctionOutput,
functionOutputKey,
} from '@aws-amplify/backend-output-schemas';
import { AttributionMetadataStorage } from '@aws-amplify/backend-output-storage';
import { FunctionOutput } from '@aws-amplify/backend-output-schemas';
import {
AmplifyUserError,
CallerDirectoryExtractor,
Expand All @@ -20,18 +16,18 @@ import {
GenerateContainerEntryProps,
LogLevel,
LogRetention,
ResourceAccessAcceptor,
ResourceAccessAcceptorFactory,
ResourceNameValidator,
ResourceProvider,
SsmEnvironmentEntry,
StackProvider,
} from '@aws-amplify/plugin-types';
import { Duration, Size, Stack, Tags } from 'aws-cdk-lib';
import { Rule } from 'aws-cdk-lib/aws-events';
import * as targets from 'aws-cdk-lib/aws-events-targets';
import { Policy } from 'aws-cdk-lib/aws-iam';
import {
CfnFunction,
IFunction,
ILayerVersion,
LayerVersion,
Runtime,
Expand All @@ -40,16 +36,19 @@ import { NodejsFunction, OutputFormat } from 'aws-cdk-lib/aws-lambda-nodejs';
import { Construct } from 'constructs';
import { readFileSync } from 'fs';
import { createRequire } from 'module';
import { fileURLToPath } from 'node:url';
import { EOL } from 'os';
import * as path from 'path';
import { FunctionEnvironmentTranslator } from './function_env_translator.js';
import { FunctionEnvironmentTypeGenerator } from './function_env_type_generator.js';
import { FunctionLayerArnParser } from './layer_parser.js';
import { convertLoggingOptionsToCDK } from './logging_options_parser.js';
import { convertFunctionSchedulesToRuleSchedules } from './schedule_parser.js';

const functionStackType = 'function-Lambda';
import {
ProvidedFunctionFactory,
ProvidedFunctionProps,
} from './provided_function_factory.js';
import { AmplifyFunctionBase } from './function_construct_base.js';
import { FunctionResourceAccessAcceptor } from './resource_access_acceptor.js';

export type AddEnvironmentFactory = {
addEnvironment: (key: string, value: string | BackendSecret) => void;
Expand All @@ -70,17 +69,38 @@ export type FunctionSchedule = TimeInterval | CronSchedule;
export type FunctionLogLevel = LogLevel;
export type FunctionLogRetention = LogRetention;

/**
* Entry point for defining a function in the Amplify ecosystem
*/
export const defineFunction = (
props: FunctionProps = {}
export function defineFunction(
props?: FunctionProps
): ConstructFactory<
ResourceProvider<FunctionResources> &
ResourceAccessAcceptorFactory &
AddEnvironmentFactory &
StackProvider
> => new FunctionFactory(props, new Error().stack);
>;
export function defineFunction(
provider: (scope: Construct) => IFunction,
providerProps?: ProvidedFunctionProps
): ConstructFactory<
ResourceProvider<FunctionResources> &
ResourceAccessAcceptorFactory &
StackProvider
>;
/**
* Entry point for defining a function in the Amplify ecosystem
*/
// This is the "implementation overload", it's not visible in public api.
// We have to use function notation instead of arrow notation.
// Arrow notation does not support overloads.
// eslint-disable-next-line no-restricted-syntax
export function defineFunction(
propsOrProvider: FunctionProps | ((scope: Construct) => IFunction) = {},
providerProps?: ProvidedFunctionProps
): unknown {
if (propsOrProvider && typeof propsOrProvider === 'function') {
return new ProvidedFunctionFactory(propsOrProvider, providerProps);
}
return new FunctionFactory(propsOrProvider, new Error().stack);
}

export type FunctionProps = {
/**
Expand Down Expand Up @@ -477,14 +497,10 @@ class FunctionGenerator implements ConstructContainerEntryGenerator {
}

class AmplifyFunction
extends Construct
implements
ResourceProvider<FunctionResources>,
ResourceAccessAcceptorFactory,
AddEnvironmentFactory
extends AmplifyFunctionBase
implements AddEnvironmentFactory
{
readonly resources: FunctionResources;
readonly stack: Stack;
private readonly functionEnvironmentTranslator: FunctionEnvironmentTranslator;
constructor(
scope: Construct,
Expand All @@ -493,9 +509,7 @@ class AmplifyFunction
backendSecretResolver: BackendSecretResolver,
outputStorageStrategy: BackendOutputStorageStrategy<FunctionOutput>
) {
super(scope, id);

this.stack = Stack.of(scope);
super(scope, id, outputStorageStrategy);

const runtime = nodeVersionMap[props.runtime];

Expand Down Expand Up @@ -616,52 +630,18 @@ class AmplifyFunction
},
};

this.storeOutput(outputStorageStrategy);

new AttributionMetadataStorage().storeAttributionMetadata(
Stack.of(this),
functionStackType,
fileURLToPath(new URL('../package.json', import.meta.url))
);
this.storeOutput();
}

addEnvironment = (key: string, value: string | BackendSecret) => {
this.functionEnvironmentTranslator.addEnvironmentEntry(key, value);
};

getResourceAccessAcceptor = () => ({
identifier: `${this.node.id}LambdaResourceAccessAcceptor`,
acceptResourceAccess: (
policy: Policy,
ssmEnvironmentEntries: SsmEnvironmentEntry[]
) => {
const role = this.resources.lambda.role;
if (!role) {
// This should never happen since we are using the Function L2 construct
throw new Error(
'No execution role found to attach lambda permissions to'
);
}
policy.attachToRole(role);
ssmEnvironmentEntries.forEach(({ name, path }) => {
this.functionEnvironmentTranslator.addSsmEnvironmentEntry(name, path);
});
},
});

/**
* Store storage outputs using provided strategy
*/
private storeOutput = (
outputStorageStrategy: BackendOutputStorageStrategy<FunctionOutput>
): void => {
outputStorageStrategy.appendToBackendOutputList(functionOutputKey, {
version: '1',
payload: {
definedFunctions: this.resources.lambda.functionName,
},
});
};
getResourceAccessAcceptor = (): ResourceAccessAcceptor =>
new FunctionResourceAccessAcceptor(
this,
this.functionEnvironmentTranslator
);
}

const isWholeNumberBetweenInclusive = (
Expand Down
58 changes: 58 additions & 0 deletions packages/backend-function/src/function_construct_base.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import { Construct } from 'constructs';
import {
BackendOutputStorageStrategy,
FunctionResources,
ResourceAccessAcceptor,
ResourceAccessAcceptorFactory,
ResourceProvider,
} from '@aws-amplify/plugin-types';
import { Stack } from 'aws-cdk-lib';
import {
FunctionOutput,
functionOutputKey,
} from '@aws-amplify/backend-output-schemas';
import { AttributionMetadataStorage } from '@aws-amplify/backend-output-storage';
import { fileURLToPath } from 'node:url';

const functionStackType = 'function-Lambda';

/**
* A base class for function constructs.
*/
export abstract class AmplifyFunctionBase
extends Construct
implements ResourceProvider<FunctionResources>, ResourceAccessAcceptorFactory
{
readonly stack: Stack;
abstract resources: FunctionResources;

abstract getResourceAccessAcceptor: () => ResourceAccessAcceptor;

/**
* Creates base function construct.
*/
protected constructor(
scope: Construct,
id: string,
private readonly outputStorageStrategy: BackendOutputStorageStrategy<FunctionOutput>
) {
super(scope, id);

this.stack = Stack.of(scope);

new AttributionMetadataStorage().storeAttributionMetadata(
Stack.of(this),
functionStackType,
fileURLToPath(new URL('../package.json', import.meta.url))
);
}

protected storeOutput = (): void => {
this.outputStorageStrategy.appendToBackendOutputList(functionOutputKey, {
version: '1',
payload: {
definedFunctions: this.resources.lambda.functionName,
},
});
};
}
2 changes: 2 additions & 0 deletions packages/backend-function/src/index.ts
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
export * from './factory.js';
import { ProvidedFunctionProps } from './provided_function_factory.js';
export { ProvidedFunctionProps };
Loading
Loading