Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pooling and request limiting to prevent memory leaks #110

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
6c12e78
Add pooling and request limiting to prevent memory leaks
seanmorris Sep 14, 2023
e968403
Adding Pool class.
seanmorris Sep 14, 2023
b515f13
Reverting change.
seanmorris Sep 14, 2023
5c7b52d
Removing console.log
seanmorris Sep 14, 2023
a33b0c5
Max request options.
seanmorris Sep 14, 2023
f8326cd
Tweaks.
seanmorris Sep 14, 2023
cb383fe
Tweaks.
seanmorris Sep 14, 2023
ac4ea1b
Initialize all instances
seanmorris Sep 14, 2023
0b9044f
Tweak.
seanmorris Sep 14, 2023
fc0f995
Tweak.
seanmorris Sep 14, 2023
e18b1fd
Consolidating FINALLY block.
seanmorris Sep 15, 2023
0ffaad7
Cleaner pooling algorithm.
seanmorris Sep 15, 2023
3d26067
Comments
seanmorris Sep 15, 2023
909d08a
Comments.
seanmorris Sep 15, 2023
1c91f34
Comments.
seanmorris Sep 15, 2023
a86c4c9
Catch spawn failures
seanmorris Sep 15, 2023
4b634bb
Making private methods privater.
seanmorris Sep 15, 2023
3f7b796
Tweaks.
seanmorris Sep 15, 2023
db3c5d6
Tweaks.
seanmorris Sep 15, 2023
68f6290
Correcting CLI switches, killing instances when done.
seanmorris Sep 16, 2023
9167e77
Detect error in test.
seanmorris Sep 16, 2023
70c3a22
Correcting multi-await
seanmorris Sep 19, 2023
aebc3d5
Correcting multi-await
seanmorris Sep 19, 2023
96a2e82
Scaling back default maxRequests
seanmorris Sep 27, 2023
0cb98b7
Pull request tweaks.
seanmorris Oct 3, 2023
a73874f
Doc Comments.
seanmorris Oct 5, 2023
326b410
Doc Comments.
seanmorris Oct 5, 2023
966a9df
Testing tweaks.
seanmorris Oct 5, 2023
488ad3c
Testing tweaks.
seanmorris Oct 5, 2023
a7708af
Testing tweaks.
seanmorris Oct 5, 2023
e01d6d4
All tests pass.
seanmorris Oct 5, 2023
eca5995
Passing linter
seanmorris Oct 5, 2023
3d8a1c2
Passing linter
seanmorris Oct 5, 2023
271c935
Passing linter
seanmorris Oct 5, 2023
a7581dc
Temporarily skip typecheck
seanmorris Oct 5, 2023
8d968d7
Revert temp change.
seanmorris Oct 5, 2023
04a3e5e
Send 500 error to browser instead of re-queueing request.
seanmorris Oct 6, 2023
65c28e2
Correcting default fatal handler
seanmorris Oct 6, 2023
a1b809e
Abstracting node pools
seanmorris Oct 6, 2023
102adbc
Tweaks.
seanmorris Oct 6, 2023
954d33f
Tweaks
seanmorris Oct 6, 2023
6bc105d
Tweaks
seanmorris Oct 6, 2023
b3c6a08
Tweaks
seanmorris Oct 6, 2023
be0c0cb
Merge branch 'trunk' into sm-preventing-memory-leaks
seanmorris Nov 22, 2023
fabcafa
Bumping versions.
seanmorris Dec 4, 2023
56484b5
Non-working copy operation
seanmorris Dec 4, 2023
c59bea8
Revering extra change.
seanmorris Dec 6, 2023
c133a1e
PR comments.
seanmorris Dec 7, 2023
a657040
PR comments.
seanmorris Dec 8, 2023
75bd3eb
Separating debug flags into its own PR
seanmorris Dec 8, 2023
c3b2269
Incrementing verion numbers
seanmorris Dec 21, 2023
1f9f696
Formatting.
seanmorris Dec 21, 2023
f352bb8
Lint & Tests.
seanmorris Dec 21, 2023
0e6c8ba
Merge conflict.
seanmorris Dec 27, 2023
0ad0b9d
PR comment tweaks.
seanmorris Dec 27, 2023
5d18b81
Restoring codemirror dependency
seanmorris Dec 27, 2023
b09b13b
Revering extra change
seanmorris Dec 27, 2023
0bd1222
PR comment tweaks.
seanmorris Dec 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 86 additions & 29 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 6 additions & 6 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,13 @@
"@codemirror/state": "6.2.0",
"@codemirror/theme-one-dark": "6.1.1",
"@codemirror/view": "6.9.3",
"@php-wasm/node": "0.1.56",
"@php-wasm/progress": "0.1.56",
"@php-wasm/universal": "0.1.56",
"@php-wasm/web": "0.1.57",
"@php-wasm/node": "0.5.2",
"@php-wasm/progress": "0.5.2",
"@php-wasm/universal": "0.5.2",
"@php-wasm/web": "0.5.2",
"@uiw/react-codemirror": "^4.21.20",
"@wp-playground/blueprints": "0.1.56",
"@wp-playground/client": "0.2.0",
"@wp-playground/blueprints": "0.5.2",
"@wp-playground/client": "0.5.2",
"classnames": "^2.3.2",
"comlink": "^4.4.1",
"compressible": "2.0.18",
Expand Down
52 changes: 26 additions & 26 deletions packages/interactive-code-block/public/blueprint.json
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
{
"landingPage": "/wp-admin/post.php?post=4&action=edit",
"steps": [
{
"step": "login",
"username": "admin",
"password": "password"
},
{
"step": "installTheme",
"themeZipFile": {
"resource": "wordpress.org/themes",
"slug": "adventurer"
}
},
{
"step": "installPlugin",
"pluginZipFile": {
"resource": "wordpress.org/plugins",
"slug": "interactive-code-block"
}
},
{
"step": "runPHP",
"code": "<?php require '/wordpress/wp-load.php'; wp_insert_post(['post_title' => 'Interactive code block demo!','post_content' => '<!-- wp:wordpress-playground/interactive-code {\"code\":\"PD9waHAKCmVjaG8gIlRoaXMgY29kZSBzbmlwcGV0IGlzIGludGVyYWN0aXZlISI7Cg==\",\"cachedOutput\":\"VGhpcyBjb2RlIHNuaXBwZXQgaXMgaW50ZXJhY3RpdmUh\"} --><div class=\"wp-block-wordpress-playground-interactive-code\">PD9waHAKCmVjaG8gIlRoaXMgY29kZSBzbmlwcGV0IGlzIGludGVyYWN0aXZlISI7Cg==</div><!-- /wp:wordpress-playground/interactive-code -->', 'post_status' => 'publish', 'post_type' => 'post',]);"
}
]
"landingPage": "/wp-admin/post.php?post=4&action=edit",
"steps": [
{
"step": "login",
"username": "admin",
"password": "password"
},
{
"step": "installTheme",
"themeZipFile": {
"resource": "wordpress.org/themes",
"slug": "adventurer"
}
},
{
"step": "installPlugin",
"pluginZipFile": {
"resource": "wordpress.org/plugins",
"slug": "interactive-code-block"
}
},
{
"step": "runPHP",
"code": "<?php require '/wordpress/wp-load.php'; wp_insert_post(['post_title' => 'Interactive code block demo!','post_content' => '<!-- wp:wordpress-playground/interactive-code {\"code\":\"PD9waHAKCmVjaG8gIlRoaXMgY29kZSBzbmlwcGV0IGlzIGludGVyYWN0aXZlISI7Cg==\",\"cachedOutput\":\"VGhpcyBjb2RlIHNuaXBwZXQgaXMgaW50ZXJhY3RpdmUh\"} --><div class=\"wp-block-wordpress-playground-interactive-code\">PD9waHAKCmVjaG8gIlRoaXMgY29kZSBzbmlwcGV0IGlzIGludGVyYWN0aXZlISI7Cg==</div><!-- /wp:wordpress-playground/interactive-code -->', 'post_status' => 'publish', 'post_type' => 'post',]);"
}
]
}
31 changes: 15 additions & 16 deletions packages/wp-now/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,24 +6,23 @@ It uses automatic mode detection to provide a fast setup process, regardless of

![Demo GIF of wp-now](https://github.com/WordPress/wordpress-playground/assets/36432/474ecb85-d789-4db9-b820-a19c25da79ad)


## Table of Contents

- [Requirements](#requirements)
- [Usage](#usage)
- [Automatic Modes](#automatic-modes)
- [Arguments](#arguments)
- [Technical Details](#technical-details)
- [Using Blueprints](#using-blueprints)
- [Defining Custom URLs](#defining-custom-urls)
- [Defining Debugging Constants](#defining-debugging-constants)
- [Known Issues](#known-issues)
- [Comparisons](#comparisons)
- [Laravel Valet](#laravel-valet)
- [wp-env](#wp-env)
- [Contributing](#contributing)
- [Testing](#testing)
- [Publishing](#publishing)
- [Requirements](#requirements)
- [Usage](#usage)
- [Automatic Modes](#automatic-modes)
- [Arguments](#arguments)
- [Technical Details](#technical-details)
- [Using Blueprints](#using-blueprints)
- [Defining Custom URLs](#defining-custom-urls)
- [Defining Debugging Constants](#defining-debugging-constants)
- [Known Issues](#known-issues)
- [Comparisons](#comparisons)
- [Laravel Valet](#laravel-valet)
- [wp-env](#wp-env)
- [Contributing](#contributing)
- [Testing](#testing)
- [Publishing](#publishing)

## Requirements

Expand Down
8 changes: 8 additions & 0 deletions packages/wp-now/src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ export interface CliOptions {
port?: number;
blueprint?: string;
reset?: boolean;
maxRequests?: number;
maxJobs?: number;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it's late in the process but now as this is basically ready I'm questioning how clear these names are. we're not limiting the total number of requests, and what is a job?

what if we either renamed them or created them as sub-options like poolOptions?

something like maxRequestsPerWorkerLifespan is verbose but does far more to communicate the role of the config, and maxPhpWorkers (or even the already existing numberOfPhpInstances) skips the term for one that's already in scope when creating one of these config values.

}

export const enum WPNowMode {
Expand All @@ -43,6 +45,8 @@ export interface WPNowOptions {
wpContentPath?: string;
wordPressVersion?: string;
numberOfPhpInstances?: number;
maxRequests?: number;
maxJobs?: number;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the interplay here between maxJobs and numberOfPhpInstances?
we didn't recreate the same config value did we?

could we use numberOfPhpInstances to set maxJobs or are these somehow indicating different things: number of workers in a pool vs. number of pools? would it make sense to only use one and then to leave the existing config for WPNowOptions?

blueprintObject?: Blueprint;
reset?: boolean;
}
Expand All @@ -54,6 +58,8 @@ export const DEFAULT_OPTIONS: WPNowOptions = {
projectPath: process.cwd(),
mode: WPNowMode.AUTO,
numberOfPhpInstances: 1,
maxRequests: 128,
maxJobs: 1,
reset: false,
};

Expand Down Expand Up @@ -111,6 +117,8 @@ export default async function getWpNowConfig(
phpVersion: args.php as SupportedPHPVersion,
projectPath: args.path as string,
wordPressVersion: args.wp as string,
maxRequests: args.maxRequests as number,
maxJobs: args.maxJobs as number,
port,
reset: args.reset as boolean,
};
Expand Down
44 changes: 44 additions & 0 deletions packages/wp-now/src/node-pool.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
import { LatestSupportedPHPVersion, PHPResponse } from '@php-wasm/universal';
import { NodePHP } from '@php-wasm/node';
import { Pool } from './pool';

export type NodePoolOptions = {
spawn?: () => Promise<NodePHP>;
reap?: (instance: NodePHP) => void;
fatal?: (instance: NodePHP, error: any) => any;
maxRequests?: number;
maxJobs?: number;
};

const defaultSpawn = async () => await NodePHP.load(LatestSupportedPHPVersion);

const defaultFatal = (instance: NodePHP, error: any) =>
new PHPResponse(
500,
{},
new TextEncoder().encode(
`500 Internal Server Error:\n\n${String(
error && error.stack ? error.stack : error
)}`
)
);

const defaultReap = (instance: NodePHP) => {
try {
instance.exit();
} catch {
// Do nothing.
}
};

export class NodePool extends Pool {
constructor({
maxRequests = 128,
maxJobs = 1,
spawn = defaultSpawn,
fatal = defaultFatal,
reap = defaultReap,
} = {}) {
super({ maxRequests, maxJobs, spawn, fatal, reap });
}
}
Loading
Loading