Mermaid rendering on RTD is brittle

[astrojuanlu]

Recently I've been spotting some intermittent failures caused by the Mermaid rendering:

WARNING: mermaid code 'flowchart TD\n  A[Start] --> B{Do you prefer developing your projects in notebooks?}\n  B -->|Yes| C[Use a Databricks workspace to develop a Kedro project]\n  B -->|No| D{Are you a beginner with Kedro?}\n  D -->|Yes| E[Use an IDE, dbx and Databricks Repos to develop a Kedro project]\n  D -->|No| F{Do you have advanced project requirements<br>e.g. CI/CD, scheduling, production-ready, complex pipelines, etc.?}\n  F -->|Yes| G{Is rapid development needed for your project needs?}\n  F -->|No| H[Use an IDE, dbx and Databricks Repos to develop a Kedro project]\n  G -->|Yes| I[Use an IDE, dbx and Databricks Repos to develop a Kedro project]\n  G -->|No| J[Use a Databricks job to deploy a Kedro project]': Mermaid exited with error:
[stderr]
b'\nTimeoutError: Timed out after 30000 ms while waiting for the WS endpoint URL to appear in stdout!\n    at ChromeLauncher.launch (file:///home/docs/.asdf/installs/nodejs/19.0.1/lib/node_modules/@mermaid-js/mermaid-cli/node_modules/puppeteer-core/lib/esm/puppeteer/node/ProductLauncher.js:119:23)\n    at async run (file:///home/docs/.asdf/installs/nodejs/19.0.1/lib/node_modules/@mermaid-js/mermaid-cli/src/index.js:404:19)\n    at async cli (file:///home/docs/.asdf/installs/nodejs/19.0.1/lib/node_modules/@mermaid-js/mermaid-cli/src/index.js:184:3)\n\n'
[stdout]
b''

https://readthedocs.org/projects/kedro/builds/22757197/

[astrojuanlu]

Looks like the pupetteer version pinned by mermaid-cli is quite old mermaid-js/mermaid-cli#627

[astrojuanlu]

Upstream issue puppeteer/puppeteer#10556

Ways we can try to fix this:

• Upgrade puppeteer
• Use Firefox engine
• Set timeout to 0

We'd need a package.json for this mermaid-js/mermaid-cli#627 (comment)

[astrojuanlu]

Or commit a pre-rendered PNG to the source tree and remind ourselves to re-render it when the source changes (which doesn't happen that often??)

[stichbury]

I like that option TBH have never been a big fan of using Mermaid to build dynamic graphics when a PNG will do.

[stichbury]

I want to make some changes to the docs that use the mermaid graphic anyway so will factor in a change to the graphics at the same time.

[astrojuanlu]

sphinxcontrib-mermaid doesn't seem to support static pre-rendering mgaitan/sphinxcontrib-mermaid#134

[stichbury]

I've been looking at tools for this like https://discourse.joplinapp.org/t/ability-to-save-export-mermaid-graphs-as-image/23491

[astrojuanlu]

Yeah we can actually take the output from build/html and paste it in our images directory. There is a number of workarounds, but it looks like we'd lose the convenience of having the Mermaid code there.

[stichbury]

We can put it in a comment though, right?

[astrojuanlu]

That's where my mind was right now...

Essentially we can call mermaid-cli ourselves once with these parameters, store the PNG under version control, and have the Mermaid code as a comment next to the diagram itself.

kedro/docs/source/conf.py

Line 554 in 543fa8b

mermaid_params = ["-p", here / "puppeteer-config.json", "-s", "2"]

I'll send a PR with this.