Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blueprints: Explore switching to the PHP implementation #1051

Closed
wants to merge 11 commits into from

Conversation

adamziel
Copy link
Collaborator

@adamziel adamziel commented Feb 23, 2024

What is this PR doing?

Explores using the blueprints.phar PHP library in Playground.

For more context see:

For now, the PHP library works nicely in local CLI but throws the following error in Playground:

CleanShot 2024-02-23 at 15 24 21@2x

Work explored in this PR

Arbitrary http:// and https:// network calls

The Blueprint library isn't currently able to make any network calls.

There are two problems:

  1. allow_url_fopen is set to 0. This is easy to change.
  2. The web version of Playground doesn't support the http:// and https:// stream wrappers.

There are two possible solutions:

  1. Implement a Fetch HTTP transport, similarly to what we do in WordPress. This is easy and mostly reliable, but doesn't generalize to other PHP code which means we'll need a transport every time we want to support another request-issuing library.
  2. Add support for the http:// and https:// stream wrappers. This is challenging! All the network traffic originating in the browser must eventually be funneled through fetch(). However, HTTPS communication requires exchanging data packets through a raw socket. Perhaps Playground could run a MITM attack on the WebAssembly PHP instance and rewrite those decrypted raw bytes as fetch() input?

Update I wend for HTTPS termination and rewriting and it worked! It's still a super rough MVP, but it works! It should also remove the need for a custom Fetch transport for WordPress and also open the door to supporting libcurl.

proc_open improvemens

PHP subprocesses

#1031 is merged into this PR

cwd and env variables in proc_open

The Blueprints library leans on running PHP subprocesses in a specific working directory and passing data through env variables, e.g. like DOCROOT=/wordpress.

Exit code propagation

The trunk version of Playground doesn't correctly communicate the proc_open exit code to the PHP code. The following update to phpwasm-emscripten-library.js solves the problem:

- HEAPU8[exitCodePtr] =  PHPWASM.child_proc_by_pid[pid].exitCode;
- wakeUp(0);
+ HEAPU32[exitCodePtr >> 2] =  PHPWASM.child_proc_by_pid[pid].exitCode;
+ wakeUp(pid);

Also, proc_get_status() now returns the correct exitcode and stopped values.

Fixing a race condition in the startup sequence

If the PHP web worker thread takes too long to start up, e.g. because it synchronously generates SSL keys, the remote.html will correctly resolve the client.isReady() promise, but the client object on the topmost HTML page will not. That results in a blank page in this demo, and can result in an always-empty empty progress bar on https://playground.wordpress.net/.

Streaming ZIPs

Stream-unzipping gutenberg.zip in the browser yields the following error:

Error: PHP.run() failed with exit code 255 and the following output: PHP Fatal error:  Out of memory (allocated 16777216 bytes) (tried to allocate 131072 bytes) in phar:///wordpress/blueprints.phar/src/WordPress/Zip/ZipStreamReader.php on line 207

Other rough notes

  • Worker thread startup race condition when ssl keys are generated
  • Need to HTTPS certificates generate when WebSocket starts
  • Box.phar "check-requirements": false, breaks HTTP requests in Playground :o
  • Out of memory error when fed real 40MB WordPress.zip – is it related to chunking/buffering/blocking calls to onData()?

@adamziel adamziel changed the title Blueprints: Explore switching to the PHP library Blueprints: Explore switching to the PHP implementation Feb 23, 2024
@adamziel adamziel requested a review from a team as a code owner February 24, 2024 01:16
@adamziel
Copy link
Collaborator Author

adamziel commented Feb 26, 2024

This Blueprint works in the latest version of this branch AND in native PHP! Internally, it relies on concurrent file downloads and WP-CLI! #1053 was the crucial bit to make the network transfer work.

$blueprint = BlueprintBuilder::create()
	->withFile( 'wordpress.txt', (new UrlResource())->setUrl('https://downloads.wordpress.org/plugin/hello-dolly.zip') )
	->withSiteOptions( [
		'blogname' => 'My Playground Blog',
	] )
	->withWpConfigConstants( [
		'WP_DEBUG'         => true,
		'WP_DEBUG_LOG'     => true,
		'WP_DEBUG_DISPLAY' => true,
		'WP_CACHE'         => true,
	] )
	->withPlugins( [
		'https://downloads.wordpress.org/plugin/hello-dolly.zip',
		'https://downloads.wordpress.org/plugin/classic-editor.zip',
		'https://downloads.wordpress.org/plugin/gutenberg.17.7.0.zip',
	] )
	->withTheme( 'https://downloads.wordpress.org/theme/pendant.zip' )
	->withContent( 'https://raw.githubusercontent.com/WordPress/theme-test-data/master/themeunittestdata.wordpress.xml' )
	->andRunSQL( <<<'SQL'
		CREATE TABLE tmp_table ( id INT );
		INSERT INTO tmp_table VALUES (1);
		INSERT INTO tmp_table VALUES (2);
		SQL
	)
	->withFile( 'wordpress.txt', 'Data' )
	->toBlueprint()

CleanShot 2024-02-26 at 17 11 20@2x

@adamziel
Copy link
Collaborator Author

All the necessary Playground enhancements are duct-taped together in this PR, the next steps are to split them out into separate PRs, merge into core, and rebase this PR until it only ships the showcase page.

adamziel added a commit that referenced this pull request Feb 27, 2024
Introduces a naive shell command parser to provide equally good support
for the following two types of `proc_open()` calls:

```php
proc_open([ "wp-cli.phar", "plugin", "install", "gutenberg" ]);
proc_open("wp-cli.phar plugin install gutenberg" ]);
```

The command parsing semantics are extremely naive at this point and only
cover splitting the command into an array of arguments as follows:

```ts
splitShellCommand(`wp option set      blogname "My \"fancy\" blog "'name'`);
> ["wp", "option", "set", "blogname", `my "fancy" blog name`]
```

There is no support for inline ENV variables, pipes, or redirects. For
those, we might need to build an actual shell binary OR turn to
something like [bun
shell](#1062).

## Testing instructions

This PR ships unit tests so just confirm the CI checks pass.

## Related resources

* #1031
* #1062
* #1051
@adamziel
Copy link
Collaborator Author

All the PHP.wasm updates required for the PHP Blueprints to run are now merged 🎉 The demo page is explored in PR #1070. The only remaining part of this PR that did not get merged into Playground core is a generic HTTPS -> network bridge that doesn't require implementing special transport classes in PHP. Let's split it out into a new PR before closing this one.

adamziel added a commit that referenced this pull request Feb 28, 2024
## What is this PR doing?

Supersedes #1051

Adds a PHP Blueprints demo page where the use of
[blueprints.phar](WordPress/blueprints#28) PHP
library in Playground may be further explored. The showcase is
intentionally not added to
http://localhost:5400/website-server/demos/index.html as PHP Blueprints
may become a part of Playground core soon enough.

For more context see:

* #1025
* https://github.com/WordPress/blueprints

## How does it work?

* The built Blueprints library is included with this PR via the
`blueprints.phar` file
* A number of PHP.wasm improvements have been merged to support it:
   * #1064
   * #1065
   * #1069
* This PR ships a `fetch` subprocess handler to enable streaming network
data in the Blueprints library – it uses [a special network transport
called
`fetch`](https://github.com/WordPress/blueprints/blob/efa8deef56095bd5bcb94868787e29f7b54350f3/src/WordPress/DataSource/PlaygroundFetchSource.php)
that requests network data via `proc_open()` when running in Playground.
Why subprocesses? They provide everything a custom network handler
needs: pipes, asynchronous execution, exit codes, internal PHP plumbing.

## Follow-up work

* Support a real-time progress bar

## Testing instructions

Go to http://localhost:5400/website-server/demos/php-blueprints.html and
confirm it looks like on the screenshot below:

![CleanShot 2024-02-28 at 15 46
14@2x](https://github.com/WordPress/wordpress-playground/assets/205419/47a91d99-07f3-40a5-a046-b58f8cda952e)
@adamziel
Copy link
Collaborator Author

adamziel commented Mar 1, 2024

For posterity, the HTTPS proxy explored in this PR requires the following change in the PHP library UrlSource class:

		$response = $this->client->request( 'GET', $url, [
			'on_progress'   => function ( int $dlNow, int $dlSize, array $info ) use ( $url ): void {
				$this->events->dispatch( new ProgressEvent(
					$url,
					$dlNow,
					$dlSize
				) );
			},
+			//        We use a fake SSL server in there to MITM the HTTPS requests
+			//        and funnel them through fetch() – and fetch() handles HTTPS
+			//  	  security for us.
+			'verify_host'   => false,
+			'verify_peer'   => false,
+			'crypto_method' => \STREAM_CRYPTO_METHOD_TLSv1_0_CLIENT,
		] );

@adamziel
Copy link
Collaborator Author

adamziel commented Mar 1, 2024

Also, response buffering in UrlSource seems to cause OOM errors

		$onChunk = function ( $chunk ) use ( $url, $response, $stream ) {
			// Handle response caching
			static $bufferedChunks = [];
			$bufferedChunks[] = $chunk;
			if ( feof( $stream ) ) {
				$this->cache->set( $url, implode( '', $bufferedChunks ) );
				$bufferedChunks = [];
			}
		};

adamziel added a commit to WordPress/blueprints-library that referenced this pull request Mar 1, 2024
This PR makes the Blueprints PHP library work in Playground:

https://playground.wordpress.net/demos/php-blueprints.html

The main change is the addition of `PlaygroundFetchSource` to download
data using browser's `fetch()` (Playground support added in
WordPress/wordpress-playground#1070). The rest
is cosmetics.

Related Playground PR
WordPress/wordpress-playground#1051
@adamziel adamziel force-pushed the use-php-blueprints branch 2 times, most recently from 9ddf54b to 2037020 Compare March 7, 2024 11:46
@adamziel
Copy link
Collaborator Author

adamziel commented Mar 7, 2024

All parts of this PR except #1093 are now merged into trunk. Let me close it then, and let's explore the native HTTPS support in #1093.

@adamziel adamziel closed this Mar 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant