Skip to content

Commit

Permalink
feat(sls-rspack): support user supplied scripts
Browse files Browse the repository at this point in the history
  • Loading branch information
codingnuclei committed Oct 28, 2024
1 parent 26f0149 commit 22eac3a
Show file tree
Hide file tree
Showing 17 changed files with 184 additions and 34 deletions.
54 changes: 46 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,14 @@ Example serverless projects are under the [/examples](/examples) directory.

For Developers - [DEVELOPER.MD](./docs/DEVELOPER.md)


## Features

- From zero to hero: configuration possibilities range from zero-config to fully customizable
- Supports `sls package`, `sls deploy`
- Build and runtime performance at its core


## Table of Contents

- [Install](#install)
Expand All @@ -30,6 +32,7 @@ For Developers - [DEVELOPER.MD](./docs/DEVELOPER.md)
- [External Dependencies](#external-dependencies)
- [Known Issues](#known-issues)


## Install

```sh
Expand All @@ -48,6 +51,7 @@ plugins:
- @kitchenshelf/serverless-rspack
```


## Plugin Options

By default, no plugin options is required, but you can override the reasonable defaults via the `custom.rspack` section in the `serverless.yml` file.
Expand All @@ -70,13 +74,14 @@ See [example folder](../../examples) for example plugin option configuration.
| `zipConcurrency` | The number of concurrent zip operations to run at once. eg. `8`. _NOTE_: This can be memory intensive and could produce slower builds. | `Infinity` |
| `keepOutputDirectory` | Keeps the `.rspack` output folder. Useful for debugging. | `false` |
| `stats` | Generate packaging information that can be used to analyze module dependencies and optimize compilation speed. | `false` |
| [`config`](#config-file) | rspack config options | `undefined` |
| `config.path` | Relative rspack config path | `undefined` |
| [`config.strategy`](#config-file-merge-strategies) | Strategy to use when a rspack config is provided | `override` |
| `esm` | Output format will be ESM (experimental) | `false` |
| `mode` | Used to set the build mode of Rspack to enable the default optimization strategy (https://www.rspack.dev/config/mode) | `production` |
| `tsConfig` | Relative path to your tsconfig | `undefined` |
| [`externals`](#external-dependencies) | Provides a way of excluding dependencies from the output bundles | `undefined` |
| [`config`](#config-file) | rspack config options. | `undefined` |
| `config.path` | Relative rspack config path. | `undefined` |
| [`config.strategy`](#config-file-merge-strategies) | Strategy to use when a rspack config is provided. | `override` |
| `esm` | Output format will be ESM (experimental). | `false` |
| `mode` | Used to set the build mode of Rspack to enable the default optimization strategy (https://www.rspack.dev/config/mode). | `production` |
| `tsConfig` | Relative path to your tsconfig. | `undefined` |
| [`externals`](#external-dependencies) | Provides a way of excluding dependencies from the output bundles. | `undefined` |
| [`scripts`](#scripts) | Array of scripts to execute after your code has been bundled by rspack. | `undefined` |

#### Read-only default Rspack Options

Expand All @@ -93,6 +98,8 @@ The following `rspack` options are automatically set and **cannot** be overwritt
| -------- | --------------------------------------------------------------------------------------------- | ----------- |
| `rspack` | Set this property on a function definition to force the handler to be processed by the plugin | `undefined` |



## Supported Runtimes

This plugin will automatically process any function that has a runtime that starts with `node` i.e `nodejs20.x`
Expand All @@ -106,6 +113,7 @@ If you wish to force a function to be process set `rspack: true` on a function d
⚠️ **Note: this will only work if your custom runtime and function are written in JavaScript/Typescript.
Make sure you know what you are doing when this option is set to `true`**


## Advanced Configuration

### Config file
Expand Down Expand Up @@ -156,8 +164,38 @@ custom:
- "^@smithy\/.*$"
- '^isin-validator$'
```
### Scripts

Run custom shell commands after your code has been bundled by rspack.

**Order**: `bundle` -> `scripts` -> `package`

Scripts are executed from the root of the service directory. This is useful for modifying the output of the build before it is packaged.

#### Usage

```yml
custom:
rspack:
externals: ['^@aws-sdk/.*$', '^sharp$'],
scripts:
- 'echo "First script"'
- 'cd $KS_BUILD_OUTPUT_FOLDER/App1 && npx npm init -y && npm pkg delete main && npm install --force --os=linux --cpu=x64 --include=optional sharp @img/sharp-linux-x64'
- 'echo "Last script"'
```
⚠️ **Note: Scripts run sequentially and will fail the build if any errors occur in any of the scripts.**

#### Environment Variables

The following environment variables are available to your scripts:

- `process.env`: All system environment variables.
- `KS_SERVICE_DIR`: The absolute path to the service directory (e.g. `/Users/user/code/my-service`).
- `KS_BUILD_OUTPUT_FOLDER`: The name of the build output folder (e.g. `.rspack`).
- `KS_PACKAGE_OUTPUT_FOLDER`: The name of the package output folder (e.g. `.serverless`).


# Known Issues
## Known Issues

- Invoke Local does not work with ESM enabled when using serverless V3: [ISSUE-11308](https://github.com/serverless/serverless/issues/11308#issuecomment-1719297694)

Expand Down
5 changes: 3 additions & 2 deletions examples/complete/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,15 @@
"package": "sls package --verbose"
},
"dependencies": {
"isin-validator": "^1.1.1"
"isin-validator": "^1.1.1",
"sharp": "^0.33.5"
},
"devDependencies": {
"@kitchenshelf/serverless-rspack": "../../dist/libs/serverless-rspack",
"@rspack/core": "1.0.5",
"@types/node": "^18.7.21",
"serverless": "^3.22.0",
"serverless-offline": "^10.2.1",
"@kitchenshelf/serverless-rspack": "../../dist/libs/serverless-rspack",
"ts-node": "^10.9.1",
"typescript": "^4.8.3"
}
Expand Down
6 changes: 6 additions & 0 deletions examples/complete/serverless.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,12 @@ custom:
- "^@aws-sdk\/.*$"
- "^@smithy\/.*$"
- '^isin-validator$'
- '^sharp$'
scripts:
- 'echo "First script"'
- 'cd $KS_BUILD_OUTPUT_FOLDER/app3 && npx npm init -y && npm install --force --os=linux --cpu=x64 --include=optional sharp @img/sharp-linux-x64'
- 'cp ./src/my-image.jpeg $KS_BUILD_OUTPUT_FOLDER/app3'
- 'echo "Last script"'

provider:
name: aws
Expand Down
10 changes: 10 additions & 0 deletions examples/complete/src/App3.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,25 @@
import { lib1Export } from '@kitchenshelf/path-import-libs/lib1/lib1';
import validateIsin from 'isin-validator';
import { readFileSync } from 'node:fs';
import path from 'node:path';
import sharp from 'sharp';

export async function handler(event: string) {
const isInvalid = validateIsin(event);
const imagePath = path.join(__dirname, '../my-image.jpeg');
const imageBuffer = readFileSync(imagePath);

const { info } = await sharp(imageBuffer)
.raw()
.toBuffer({ resolveWithObject: true });

return {
statusCode: lib1Export(!!isInvalid),
body: JSON.stringify({
handler: 'App3',
message: isInvalid ? 'ISIN is invalid!' : 'ISIN is fine!',
input: event,
info,
}),
};
}
Binary file added examples/complete/src/my-image.jpeg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
17 changes: 8 additions & 9 deletions libs/serverless-rspack/src/lib/bundle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,7 @@ export async function bundle(

if (this.providedRspackConfig && this.pluginOptions.config?.strategy) {
this.log.verbose(
'Config merge strategy:',
this.pluginOptions.config.strategy
`[Bundle] Config merge strategy: ${this.pluginOptions.config.strategy}`
);
if (this.pluginOptions.config.strategy === 'combine') {
const baseConfig = defaultConfig(
Expand Down Expand Up @@ -56,8 +55,8 @@ export async function bundle(
this.log
);
}
this.log.verbose('Bundling with config: ', config);
const startPack = Date.now();
this.log.verbose(`[Bundle] Bundling with config: ${JSON.stringify(config)}`);
const startBundle = Date.now();

return new Promise((resolve) => {
rspack(config, (x, y) => {
Expand All @@ -69,13 +68,13 @@ export async function bundle(
JSON.stringify(c)
);
} catch (error) {
this.log?.error('Failed to write stats file: ', error);
this.log?.error(`[Bundle] Failed to write stats file: ${error}`);
}
}
this.log.verbose(
`[Performance] Bundle service ${this.serverless.service.service} [${
Date.now() - startPack
} ms]`
`[Performance] Bundle total execution time for service ${
this.serverless.service.service
} [${Date.now() - startBundle} ms]`
);
resolve('Success!'); // Yay! Everything went well!
});
Expand Down Expand Up @@ -172,7 +171,7 @@ function mergeArrayUniqueStrategy(logger: RspackServerlessPlugin['log']) {
);
if (matchedPlugin) {
logger.warning(
`You have provided your own ${matchedPlugin.name}. This will override the default one provided by @kitchenshelf/serverless-rspack.`
`[Bundle] You have provided your own ${matchedPlugin.name}. This will override the default one provided by @kitchenshelf/serverless-rspack.`
);
} else {
plugins.push(basePlugin);
Expand Down
2 changes: 1 addition & 1 deletion libs/serverless-rspack/src/lib/hooks/initialize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,5 +35,5 @@ export async function Initialize(this: RspackServerlessPlugin) {
`No functions detected in service - you can remove this plugin from your service`
);
}
this.log.verbose('Function Entries:', this.functionEntries);
this.log.verbose('[sls-rspack] Function Entries:', this.functionEntries);
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import type { RspackServerlessPlugin } from '../../serverless-rspack.js';
import { isInvokeOptions } from '../../types.js';

export async function BeforeInvokeLocalInvoke(this: RspackServerlessPlugin) {
this.log.verbose('before:invoke:local:invoke');
this.log.verbose('[sls-rspack] before:invoke:local:invoke');

if (!isInvokeOptions(this.options)) {
throw new this.serverless.classes.Error(
Expand All @@ -22,6 +22,8 @@ export async function BeforeInvokeLocalInvoke(this: RspackServerlessPlugin) {
[invokeFunc]: this.functionEntries[invokeFunc],
});

await this.scripts();

this.serverless.config.servicePath =
this.serviceDirPath + '/' + this.buildOutputFolder + '/' + invokeFunc;
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import type { RspackServerlessPlugin } from '../../serverless-rspack.js';
export async function AfterPackageCreateDeploymentArtifacts(
this: RspackServerlessPlugin
) {
this.log.verbose('after:package:createDeploymentArtifacts');
this.log.verbose('[sls-rspack] after:package:createDeploymentArtifacts');
await this.cleanup();
this.log.verbose(
`[Performance] Hook createDeploymentArtifacts ${
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ import type { RspackServerlessPlugin } from '../../serverless-rspack.js';
export async function BeforePackageCreateDeploymentArtifacts(
this: RspackServerlessPlugin
) {
this.log.verbose('before:package:createDeploymentArtifacts');
this.log.verbose('[sls-rspack] before:package:createDeploymentArtifacts');
this.timings.set('before:package:createDeploymentArtifacts', Date.now());
await this.bundle(this.functionEntries);
await this.scripts();
await this.pack();
}
10 changes: 6 additions & 4 deletions libs/serverless-rspack/src/lib/pack.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,12 @@ export async function pack(this: RspackServerlessPlugin) {

const artifactPath = path.join(
this.serviceDirPath,
this.packageOutputPath,
this.packageOutputFolder,
zipName
);
try {
this.log.verbose(`[Pack] Compressing ${name} to ${artifactPath}`);
const startZip = Date.now();

await zipDirectory(
this.serviceDirPath + '/' + this.buildOutputFolder + '/' + name,
artifactPath
Expand All @@ -34,15 +34,17 @@ export async function pack(this: RspackServerlessPlugin) {
} - ${humanSize(size)} [${Date.now() - startZip} ms]`
);
} catch (error) {
this.log.error(error);
throw new this.serverless.classes.Error(
`[Pack] Failed to zip ${name}: ${error}`
);
}
loadedFunc.package = {
artifact: artifactPath,
};
};

this.log.verbose(
`[Performance] Pack service ${this.serverless.service.service} with concurrency: [${this.pluginOptions.zipConcurrency}] `
`[Pack] Pack service ${this.serverless.service.service} with concurrency: [${this.pluginOptions.zipConcurrency}] `
);

await pMap(
Expand Down
40 changes: 40 additions & 0 deletions libs/serverless-rspack/src/lib/scripts.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import type { RspackServerlessPlugin } from './serverless-rspack.js';
import { execSync } from 'node:child_process';

export async function scripts(this: RspackServerlessPlugin) {
if (this.pluginOptions.scripts && this.pluginOptions.scripts.length > 0) {
this.log.verbose(
`[Scripts] Running ${this.pluginOptions.scripts.length} scripts`
);
const startScripts = Date.now();
for (const [index, script] of this.pluginOptions.scripts.entries()) {
const startZip = Date.now();
try {
execSync(script, {
cwd: this.serviceDirPath,
stdio: this.options.verbose ? 'inherit' : 'ignore',
env: {
...process.env,
KS_SERVICE_DIR: this.serviceDirPath,
KS_BUILD_OUTPUT_FOLDER: this.buildOutputFolder,
KS_PACKAGE_OUTPUT_FOLDER: this.packageOutputFolder,
},
});
this.log.verbose(
`[Performance] Script ${index + 1} of ${
this.pluginOptions.scripts.length
} [${Date.now() - startZip} ms]`
);
} catch (error) {
throw new this.serverless.classes.Error(
`Failed to execute script: ${script}`
);
}
}
this.log.verbose(
`[Performance] Scripts total execution time for service ${
this.serverless.service.service
} [${Date.now() - startScripts} ms]`
);
}
}
14 changes: 9 additions & 5 deletions libs/serverless-rspack/src/lib/serverless-rspack.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import { BeforeInvokeLocalInvoke } from './hooks/invoke-local/before-invoke.js';
import { AfterPackageCreateDeploymentArtifacts } from './hooks/package/after-create-deployment-artifacts.js';
import { BeforePackageCreateDeploymentArtifacts } from './hooks/package/before-create-deployment-artifacts.js';
import { pack } from './pack.js';
import { scripts } from './scripts.js';
import {
PluginFunctionEntries,
PluginOptions,
Expand All @@ -24,7 +25,7 @@ export class RspackServerlessPlugin implements ServerlessPlugin {
serviceDirPath: string;
buildOutputFolder: string;
buildOutputFolderPath: string;
packageOutputPath: string;
packageOutputFolder: string;

log: ServerlessPlugin.Logging['log'];
serverless: Serverless;
Expand All @@ -39,6 +40,7 @@ export class RspackServerlessPlugin implements ServerlessPlugin {

protected bundle = bundle.bind(this);
protected pack = pack.bind(this);
protected scripts = scripts.bind(this);

constructor(
serverless: Serverless,
Expand All @@ -57,7 +59,7 @@ export class RspackServerlessPlugin implements ServerlessPlugin {
this.options = options;
this.log = logging.log;
this.serviceDirPath = this.serverless.config.serviceDir;
this.packageOutputPath = SERVERLESS_FOLDER;
this.packageOutputFolder = SERVERLESS_FOLDER;
this.buildOutputFolder = WORK_FOLDER;
this.buildOutputFolderPath = path.join(
this.serviceDirPath,
Expand All @@ -81,7 +83,9 @@ export class RspackServerlessPlugin implements ServerlessPlugin {
}

protected buildFunctionEntries(functions: string[]) {
this.log.verbose('Building function entries for: ', functions);
this.log.verbose(
`[sls-rspack] Building function entries for: ${functions}`
);
let entries: PluginFunctionEntries = {};

functions.forEach((functionName) => {
Expand Down Expand Up @@ -127,7 +131,7 @@ export class RspackServerlessPlugin implements ServerlessPlugin {
) {
const handler = serverlessFunction.handler;
this.log.verbose(
`Processing function ${name} with provided handler ${handler}`
`[sls-rspack] Processing function ${name} with provided handler ${handler}`
);

const handlerFile = this.getHandlerFile(handler);
Expand All @@ -149,7 +153,7 @@ export class RspackServerlessPlugin implements ServerlessPlugin {
const ext = path.extname(file);

this.log.verbose(
`Determined: filePath: [${safeFilePath}] - fileName: [${fileName}] - ext: [${ext}]`
`[sls-rspack] Determined: filePath: [${safeFilePath}] - fileName: [${fileName}] - ext: [${ext}]`
);
const outputExtension = this.isESM() ? 'mjs' : 'js';

Expand Down
Loading

0 comments on commit 22eac3a

Please sign in to comment.