Routes / Endpoints referenced in <meta> tags do not get crawled

[LorisSigrist]

Describe the bug

The Problem

During prerender, paths and endpoints that are linked to on a page get crawled. But the ones in <meta> tags do not. This means that just linking to an endpoint with <meta name="og:image" content="/og/post-1.jpg" > is not enough to cause it to be included in the prerender.

This makes stuff like programmatic social-images way more complex, since we cannot simply prerender a dynamic endpoint into image files.

The challenge with solving this

It's not possible to simply treat all content attributes on meta tags as links, since they often aren't, and there is no way to differentiate between urls and strings just from the content attribute. Whether or not the content attribute contains an url depends on the name attribute.

This means that SvelteKit would need to maintain a "whitelist" of which name attributes on meta tags should cause the content attributes the be crawled. However, since the list of meta tags rarely changes, the maintenance burden would probably be minimal.

I've prototyped an implementation of this, and would be happy to submit a PR if people agree with the described behaviour.

Reproduction

https://github.com/LorisSigrist/sk-social-image-prerender-reproduction

Logs

Error: The following routes were marked as prerenderable, but were not prerendered because they were not found while crawling your app:
  - /og/[image].jpg

See https://kit.svelte.dev/docs/page-options#prerender-troubleshooting for info on how to solve this
    at prerender (file:///Users/sigrist/Desktop/sk-social-image-prerender-reproduction/node_modules/@sveltejs/kit/src/core/postbuild/prerender.js:470:9)
    at async process.<anonymous> (file:///Users/sigrist/Desktop/sk-social-image-prerender-reproduction/node_modules/@sveltejs/kit/src/utils/fork.js:25:17)

System Info

System:
    OS: macOS 13.3.1
    CPU: (8) arm64 Apple M1
    Memory: 127.17 MB / 8.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.1.0 - /opt/homebrew/bin/node
    Yarn: 1.22.19 - /opt/homebrew/bin/yarn
    npm: 9.6.4 - /opt/homebrew/bin/npm
    Watchman: 2023.05.08.00 - /opt/homebrew/bin/watchman
  Browsers:
    Chrome: 113.0.5672.92
    Chrome Canary: 115.0.5762.0
    Firefox: 112.0.2
    Safari: 16.4
  npmPackages:
    @sveltejs/adapter-static: ^2.0.2 => 2.0.2 
    @sveltejs/kit: ^1.5.0 => 1.16.3 
    svelte: ^3.54.0 => 3.59.1 
    vite: ^4.2.0 => 4.3.5

Severity

serious, but I can work around it

Additional Information

No response

[Conduitry]

Duplicate of #5228.