Cache Playground assets to enable offline support #1535

bgrgicak · 2024-06-21T12:29:35Z

Motivation for the change, related issues

The promise of Playground is WordPress anywhere.
Playground can run in the browser without a server, but it still requires the internet on every load.

This PR adds caching of Playground assets and ensures they are available when offline.

Implementation details

The Remote Service Worker will start caching when initialized and clean previous versions if they exist.

The WorkerCache class uses browser cache to preload data when the service worker is loaded.
On each unscoped fetch request, it will attempt to load data from the cache by using the cachedFetch method.

Only URLs that aren't PHP scripts (end with .php) and valid domains will be cached. (see isValidHostname for details)

Testing Instructions (or ideally a Blueprint)

Follow the Test offline support instructions
If you previously loaded the site unregister the service worker and clear cache storage
After loading Playground only once in your browser turn off the server
In Dev tools > Application > Service Worker set it to Offline (this ensures remote assets are loaded)
Refresh the page
Confirm that Playground loaded
Browse the site, go to WP-admin, make changes...
It should all work as usual

Note: In Chrome the offline checkbox sometimes stops working. When this happens Playground will load without remote assets and admin styles (including the admin bar) will look broken. If it happens you can confirm it by running window.navigator.onLine. To resolve it uncheck and check the Offline checkbox and reload.

bgrgicak · 2024-06-24T12:39:24Z

Playground can now load from the cache when there is no server.

There is still an issue on the first load where some static assets aren't pulled from cache.
Also, assets are cached only on the second load.

Other TODOs:

Store cache with a version prefix
Flush cache when version changes

packages/php-wasm/web-service-worker/src/initialize-service-worker.ts

packages/playground/remote/src/lib/worker-caching.ts

packages/php-wasm/web/src/lib/register-service-worker.ts

packages/playground/remote/src/lib/worker-caching.ts

adamziel · 2024-07-23T00:18:20Z

On the first sight this looks really good. I'll give this another read during the day – it looks like we're almost there! Thank you @bgrgicak ! ❤️

adamziel · 2024-07-23T11:18:19Z

I loaded Playground
I checked the "Offline" checkbox in the devtools
I opened the options modal and selected the "browser storage"
After the page refreshed, CSS assets wouldn't load
I refreshed the Playground view (not the entire page), and then the CSS assets loaded

I suspect the issue is because:

On the the initial load, we expect to load the CSS assets until we backfill the ZIP file
The server isn't there when we're offline so all these HTTP requests fail
window.navigator.onLine here is true because we actually are online.

This seems fine. I don't think we need to fix anything here, let's just document it somewhere where we'll look for the root cause in the future. Perhaps in the offline testing instructions in the README and here?

adamziel · 2024-07-23T11:24:37Z

packages/playground/remote/service-worker.ts

+ *
+ * These assets are listed in the `/assets-required-for-offline-mode.json` file
+ * and contain JavaScript, CSS, and other assets required to load the site without
+ * making any network requests.


Let's make it clear we skipped the await on purpose.

adamziel · 2024-07-23T11:27:13Z

packages/playground/remote/service-worker.ts

+ *
+ * If your build version doesn't change while developing locally check `buildVersionPlugin` for more details on how it's generated.
+ */
+cache.cleanup();


The .cleanup() name make it seem like we're evicting the assets we just requested, how about calling it .removeOutdatedFiles() and calling if before .cacheOfflineModeAssets()?

Also, let's make it clear we've skipped the await on purpose

adamziel · 2024-07-23T11:32:44Z

packages/playground/remote/src/lib/offline-mode-cache.ts

+		return await cache.then((c) => c.match(key, { ignoreSearch: true }));
+	};
+
+	shouldCacheUrl = (url: URL) => {


This is one eager function. It will cache everything we request on the playground.wordpress.net domain, including /demos/index.html and /client/index.js. I'm worry about the unintended consequences. Would there be any downside to limiting this to the offline mode assets, PHP builds, and WordPress builds?

Why not cache these files? They are small and will load faster after refresh if they are cached.
We would also need to maintain a list similar to the one in listAssetsRequiredForOfflineMode.

adamziel · 2024-07-23T11:33:15Z

This works really well with the "Offline" checkbox in devtools, but it breaks down as soon as I stop the local PHP server:

TypeError: Failed to update a ServiceWorker for scope ('http://localhost:9999/') with script ('http://localhost:9999/sw.js'): An unknown error occurred when fetching the script.

I think that's because of the feedback I provided earlier about rethrowing the error. Hm. I wonder what would be a useful thing to do there – we do want to throw if we have the internet and just can't fetch the service worker, but we do not want to throw if we're falling back the offline mode when playground.wordpress.net is down. That is a tough one.

Perhaps it's fine to not re-throw the error when we already have a previous version of the service worker in place. What do you think @brandonpayton @bgrgicak ?

adamziel · 2024-07-23T11:35:43Z

This works really well in my testing @bgrgicak, yay! 🎉 I was able to create a bunch of new Playgrounds, use the browser storage, preserve my site across multiple page loads and keep updating it as I went. Also, it loads so fast now that everything is cached offline. ❤️

I left a few comments, but I don't see any major blockers here. Feel free to merge and address them in a follow-up PR.

adamziel · 2024-07-23T13:19:07Z

Actually the missing CSS bothered me and I explored a refactor in #1643 – navigator.onLine doesn't seem to be useful, what we really want to know is whether we have the assets in cache.

adamziel · 2024-07-23T22:21:06Z

I found one more issue and proposed a fix in #1643

bgrgicak · 2024-07-24T04:46:40Z

I think that's because of the feedback I provided earlier about rethrowing the error. Hm. I wonder what would be a useful thing to do there – we do want to throw if we have the internet and just can't fetch the service worker, but we do not want to throw if we're falling back the offline mode when playground.wordpress.net is down. That is a tough one.

That's interesting. I didn't have many issues with it while testing, but I experienced that navigator.isOnline would start ignoring the checkbox after a few reloads. Simply rechecking it worked.
Let's leave the code as is and just log the error. If we notice issues in "real" offline after merging it we can get back to it.

Actually the missing CSS bothered me and I explored a refactor in #1643 – navigator.onLine doesn't seem to be useful, what we really want to know is whether we have the assets in cache.

Similarly, this just worked for me, but not relaying on the browser with isOnline seems like a more robust approach.
Thank you!

bgrgicak · 2024-07-24T04:47:39Z

This seems fine. I don't think we need to fix anything here, let's just document it somewhere where we'll look for the root cause in the future. Perhaps in the offline testing instructions in the README and here?

With #1643 we don't need the comment anymore, but I still added a comment for testing offline.

@bgrgicak

A set of bugfixes and enhancements for [PR 1535 – Cache Playground assets to enable offline support](#1535): ### Offline mode after a single page load **Without this PR:** When the service worker is first registered, it does not yet handle the network requests triggered by the page that registered it. Files fetched on the initial Playground load, like `wordpress-static.zip`, are not cached. This means the offline mode won't work yet. You need to do a full page reload, wait for another round of `fetch()`-es, and only then you get cached assets and a functional offline mode. **With this PR:** Service Worker calls `self.clients.claim()` to start controlling the Playground page when the service worker is initially registered. Static assets are cached and offline mode works after the very first page load. ### Backfill static assets before the first paint * If `wordpress-static.zip` is cached, it will be extracted during Playground boot. * If it's not cached, we'll boot and render Playground first, and only then start downloading and extracting that file. ### Other smaller changes * Simplified OfflineModeCache, used class methods (`function myFn(){}` instead of instance methods (`myFn = () => {}`) cc @bgrgicak --------- Co-authored-by: Bero <berislav.grgicak@gmail.com>

bgrgicak · 2024-07-24T04:52:33Z

I addressed the feedback and merged #1643. We should be ready to merge now. Thanks for your help @adamziel @brandonpayton!

bgrgicak added 10 commits June 20, 2024 13:37

ZIP WP assets

6d9e165

Add afterWordPressInstall hook to boot

75bc2bb

Download assets after WordPress is installed

6c2b0ad

Merge branch 'trunk' into add/wp-static-asset-zip

57daebb

Use getWordPressVersionFromPhp

42be4fc

Add offline testing instructions

be0aa0e

Add fetch caching

19da646

Fix manifests

1170ab5

Remove caching from remote

9df228d

PHP-wasm caching TODO clean up code

fd8dc13

bgrgicak self-assigned this Jun 21, 2024

bgrgicak added 7 commits June 24, 2024 11:52

Remove bootWordPress hook

8beee0c

Use wordPressSiteUrl

6cf5fef

Stream static asset response instead of awaiting

1f2347e

Use request handler to get document root

f1e2784

Merge branch 'add/wp-static-asset-zip' into add/fetch-caching

dbb43aa

Preload worker

1ec9d62

Prevent worker update error when offline

df506d1

bgrgicak added [Type] Enhancement New feature or request [Aspect] Networking [Aspect] Service Worker [Aspect] Website labels Jun 24, 2024

bgrgicak added 5 commits June 25, 2024 10:17

Add warning if assets weren't downloaded

3c98912

Merge branch 'trunk' into add/wp-static-asset-zip

e703eff

Async fetch

2e3910c

Update error messages

d2ca9c2

Use wpVersionToStaticAssetsDirectory

4a1f8d0

adamziel reviewed Jun 25, 2024

View reviewed changes

packages/php-wasm/web-service-worker/src/initialize-service-worker.ts Outdated Show resolved Hide resolved

Use getLoadedWordPressVersion

efb4b57

brandonpayton reviewed Jul 21, 2024

View reviewed changes

packages/playground/remote/src/lib/worker-caching.ts Outdated Show resolved Hide resolved

bgrgicak added 3 commits July 22, 2024 07:43

Cache the current hostname

b888b97

Merge branch 'trunk' into add/fetch-caching

a6462e0

Rename method to shouldCacheUrl

c8796d6

adamziel reviewed Jul 23, 2024

View reviewed changes

packages/php-wasm/web/src/lib/register-service-worker.ts Outdated Show resolved Hide resolved

adamziel reviewed Jul 23, 2024

View reviewed changes

packages/playground/remote/src/lib/worker-caching.ts Outdated Show resolved Hide resolved

bgrgicak mentioned this pull request Jul 23, 2024

Trigger asset caching from boot #1623

Closed

bgrgicak added 2 commits July 23, 2024 08:13

Merge branch 'trunk' into add/fetch-caching

f0d00e8

Rename OfflineModeCache class and update warning

8003609

bgrgicak requested a review from adamziel July 23, 2024 06:32

Fix sw update warning

60c8c60

adamziel reviewed Jul 23, 2024

View reviewed changes

Swallow the registration.update() error, add a few comments

c4533d9

adamziel mentioned this pull request Jul 23, 2024

Support offline mode after the first Playground page load #1643

Merged

Address feedback

77a813f

adamziel and others added 2 commits July 24, 2024 06:49

Merge branch 'trunk' into add/fetch-caching

fa8c800

bgrgicak merged commit 9f6ed38 into trunk Jul 24, 2024
5 checks passed

bgrgicak deleted the add/fetch-caching branch July 24, 2024 05:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cache Playground assets to enable offline support #1535

Cache Playground assets to enable offline support #1535

bgrgicak commented Jun 21, 2024 •

edited

Loading

bgrgicak commented Jun 24, 2024

adamziel commented Jul 23, 2024

adamziel commented Jul 23, 2024 •

edited

Loading

adamziel Jul 23, 2024 •

edited

Loading

adamziel Jul 23, 2024

adamziel Jul 23, 2024

adamziel Jul 23, 2024 •

edited

Loading

bgrgicak Jul 24, 2024

adamziel commented Jul 23, 2024 •

edited

Loading

adamziel commented Jul 23, 2024

adamziel commented Jul 23, 2024

adamziel commented Jul 23, 2024

bgrgicak commented Jul 24, 2024

bgrgicak commented Jul 24, 2024

bgrgicak commented Jul 24, 2024

Cache Playground assets to enable offline support #1535

Cache Playground assets to enable offline support #1535

Conversation

bgrgicak commented Jun 21, 2024 • edited Loading

Motivation for the change, related issues

Implementation details

Testing Instructions (or ideally a Blueprint)

bgrgicak commented Jun 24, 2024

adamziel commented Jul 23, 2024

adamziel commented Jul 23, 2024 • edited Loading

adamziel Jul 23, 2024 • edited Loading

Choose a reason for hiding this comment

adamziel Jul 23, 2024

Choose a reason for hiding this comment

adamziel Jul 23, 2024

Choose a reason for hiding this comment

adamziel Jul 23, 2024 • edited Loading

Choose a reason for hiding this comment

bgrgicak Jul 24, 2024

Choose a reason for hiding this comment

adamziel commented Jul 23, 2024 • edited Loading

adamziel commented Jul 23, 2024

adamziel commented Jul 23, 2024

adamziel commented Jul 23, 2024

bgrgicak commented Jul 24, 2024

bgrgicak commented Jul 24, 2024

bgrgicak commented Jul 24, 2024

bgrgicak commented Jun 21, 2024 •

edited

Loading

adamziel commented Jul 23, 2024 •

edited

Loading

adamziel Jul 23, 2024 •

edited

Loading

adamziel Jul 23, 2024 •

edited

Loading

adamziel commented Jul 23, 2024 •

edited

Loading