Markdown Block: Add full support for markdown-it features

[danjjohnson]

Primary issue: #11881

With reference to the markdown-it demo code:
https://markdown-it.github.io/

The following features are not supported in the Markdown block:

• Typographic replacements
• Syntax highlighting
• Autoconverted links
• Subscript / Superscript
• Inserted text
• Marked text
• Footnotes Markdown block: add support for footnotes. #30149
• Definition lists
• Abbreviations
• Custom containers

Additional context

3558150-zen

[jeherve]

Noting that some / most of those extra features are achieved via extra options and / or plugins, and thus would require extra dependencies. That's not to say we shouldn't implement them, though.

[DBJDBJ]

Wrong. Even I could deduce WordPress for some reason has no highlight.js included. It is not just enough to use some other markdown block/plugin. I tried and still, there was no syntax coloring.

So I did some very short googling and then I added it, and voila, I have the coloring now. https://dbj.org/cpp-zen-of-the-sad-path

Just then, I discovered this blog and post: https://adrien.poupa.fr/highlight-js-on-wordpress and left two comments there.

[robsonsobral]

Would a PR be welcome, @jeherve ?

[jeherve]

@robsonsobral Yep, I'd be happy to review a PR about this, thank you!

[robsonsobral]

I'm trying, @jeherve , but having a hard time to make the dev environment to work:

Error: RuntimeException: Failed to get url 'https://api.wordpress.org/core/version-check/1.7/?locale=en_US': cURL error 28: Resolving timed out after 10000 milliseconds.

I'm reading a lot online, without luck.

[jeherve]

@robsonsobral At what point do you run into this issue? When installing WordPress when the Docker instance is being set up?
Do you experience any issues when trying to access the URL directly?

[robsonsobral]

The container just stops after this error. It goes up, tries to check the new version of WP, fails and goes down.

[jeherve]

What happens when you try to load that URL in your browser? Can you access it?
Could you also tell me more about your set up (OS, versions, ...)?

[robsonsobral]

I've no idea what happened! I tried again and it worked! Maybe, my Docker installation is haunted!

Thank you, @jeherve ! I'm sorry.

[robsonsobral]

Jesus, it's hard!

└─> npm install
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: Jetpack@10.9.0-a.0
npm ERR! Found: react@17.0.2
npm ERR! node_modules/react
npm ERR!   optional react@"17.0.2" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer react@"^16.8" from @automattic/components@1.0.0-alpha.3
npm ERR! node_modules/@automattic/components
npm ERR!   @automattic/components@"1.0.0-alpha.3" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

Let's try harder:

└─> npm install --force
npm WARN using --force Recommended protections disabled.
npm ERR! code EUNSUPPORTEDPROTOCOL
npm ERR! Unsupported URL Type "workspace:": workspace:* || ^0.1

Should I give up?

[jeherve]

We don't use npm in this repository, so this won't work. We use pnpm instead. I would recommend checking the development guide, that should give you an idea of the tools you need:
https://github.com/Automattic/jetpack/blob/master/docs/development-environment.md#install-development-tools

To make all this easier to remember, we have a wrapper, jetpack, that you will be able to use to build the project and much more, without having to remember specific tools or commands. You can read more about it here:
https://github.com/Automattic/jetpack/blob/master/docs/development-environment.md#jetpack-cli

You can also check this script to figure out if you have all the tools you need:
https://github.com/Automattic/jetpack/blob/master/docs/development-environment.md#check-if-your-environment-is-ready-for-jetpack-development

[robsonsobral]

All tools installed and checked. However, now, I can't install jetpack...

Warning: require_once(/usr/local/src/jetpack-monorepo/projects/plugins/jetpack/jetpack_vendor/automattic/jetpack-assets/actions.php): Failed to open stream: No such file or directory in /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/jetpack-autoloader/class-version-loader.php on line 89

Jesus...

[jeherve]

Failed to open stream: No such file or directory in /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/jetpack-autoloader/class-version-loader.php on line 89

Do you see the vendor directories in your Jetpack plugin installation? Did you build the plugin so those dependencies could be installed? You can do so using the CLI tool I mentioned above, for example.

Let us know how that goes.

[robsonsobral]

Yeah, I've the directory projects\plugins\jetpack\vendor.

I'm sorry, @jeherve . I didn't imagine it would take so long. I'm not a WP dev.

[caraya]

Will solving this issue signify parity in terms of features supported between the classic and block implementations of Markdown in Jetpack?

I've been struggling with trying to get the Markdown block to work for my use cases and find it alarming that, even when the issue is resolved then a lot of the Markdown I write will not work with the Markdown block as implemented because the block doesn't implement the same Markdown features than the PHP version does

[jeherve]

I've the directory projects\plugins\jetpack\vendor.

Good. Do you have projects/plugins/jetpack/jetpack_vendor too? Do you see any errors when running jetpack build plugins/jetpack in general?

I'm sorry, @jeherve . I didn't imagine it would take so long. I'm not a WP dev.

I'm sorry too! Since this GitHub repository can be used to build many different things (WordPress plugins, composer packages, JavaScript packages), the original development environment is more involved than if we were just working with a single thing.
We've tried to simplify the setup process with the guides and the scripts like tools/check-development-environment.sh, but as you're experiencing yourself, it seems it doesn't work for everyone just yet! If you're still willing to give this a try, I'd be really curious to understand what's going wrong on your install, so we can fix it for you and for every future contributor!

Will solving this issue signify parity in terms of features supported between the classic and block implementations of Markdown in Jetpack?

Adding support for more markdown-it features (what we aim to do with this issue) would indeed bridge that gap, and fix a lot of the inconsistencies logged in #11881. That said, I can't guarantee that it will bring full parity, as ultimately the 2 features are implemented differently.

We can continue work towards that parity though, even we're done adding markdown-it features to the block. If there are specific differences that you know about, and that are not listed in #11881, do not hesitate to comment there so we know about it.

Thank you!

[robsonsobral]

@jeherve , I'm back!

Here we go:

Hell yeah! It is time to build!
Go ahead and sit back. Relax. This will take a few minutes.
  ✔ Install pnpm dependencies [complete]
  ✖ Build plugins/jetpack [complete]
    → Command failed with exit code 1: composer run --timeout=0 build-development

Let's remove PHP 8.1 and install 8.0. Now, I'm getting:

Error: Cannot find any-observable implementation nor global.Observable. You must install polyfill or call require("any-observable/register") with your preferred implementation, e.g. require("any-observable/register")('rxjs') on application load prior to any require("any-observable").
    at loadImplementation (/mnt/e/_repositories/jetpack/node_modules/.pnpm/any-observable@0.3.0/node_modules/any-observable/register.js:29:9)
    at /mnt/e/_repositories/jetpack/node_modules/.pnpm/any-observable@0.3.0/node_modules/any-observable/loader.js:30:18
    at Object.<anonymous> (/mnt/e/_repositories/jetpack/node_modules/.pnpm/any-observable@0.3.0/node_modules/any-observable/index.js:2:39)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/mnt/e/_repositories/jetpack/node_modules/.pnpm/@samverschueren+stream-to-observable@0.3.1_rxjs@6.6.7/node_modules/@samverschueren/stream-to-observable/index.js:2:20)
Something unexpected happened. See error above.

So, let's run pnpm install again:

Progress: resolved 2435, reused 2414, downloaded 0, added 0, done
. prepare$ husky install
│ husky - Git hooks installed
└─ Done in 99ms

And... Running jetpack build plugins/jetpack gives us:

Error: Cannot find any-observable implementation nor global.Observable. You must install polyfill or call require("any-observable/register") with your preferred implementation, e.g. require("any-observable/register")('rxjs') on application load prior to any require("any-observable").
    at loadImplementation (/mnt/e/_repositories/jetpack/node_modules/.pnpm/any-observable@0.3.0/node_modules/any-observable/register.js:29:9)
    at /mnt/e/_repositories/jetpack/node_modules/.pnpm/any-observable@0.3.0/node_modules/any-observable/loader.js:30:18
    at Object.<anonymous> (/mnt/e/_repositories/jetpack/node_modules/.pnpm/any-observable@0.3.0/node_modules/any-observable/index.js:2:39)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/mnt/e/_repositories/jetpack/node_modules/.pnpm/@samverschueren+stream-to-observable@0.3.1_rxjs@6.6.7/node_modules/@samverschueren/stream-to-observable/index.js:2:20)
Something unexpected happened. See error above.

The development checker reports:

Jetpack development environment check
=====================================

Monorepo path: /mnt/e/_repositories/jetpack

Shell tools
===========

* Usable version of bash                   ... ok (version 5.0.17(1)-release)
* Standard tools are available             ... yes
* If sed accepts -E                        ... yes
* Usable version of jq                     ... ok (version 1.6)

PHP tools
=========

* Usable version of PHP                    ... ok (version 8.0.19)
* Usable version of Composer               ... ok (version 2.2.12)
* Required extensions are installed        ... yes

JavaScript tools
================

* Usable version of Node.js                ... ok (version 16.14.2)
* Usable version of pnpm                   ... ok (version 6.32.6)
* [optional] nvm is available              ... no
  See https://github.com/Automattic/jetpack/blob/master/docs/development-environment.md#nodejs

Tools for contributing
======================

* Usable version of git                    ... ok (version 2.35.2)
* If this is a git checkout                ... yes
* If the repo is checked out using ssh     ... yes

Installation
============

* Command jetpack is available             ... yes
* If JS modules are installed              ... yes
* If PHP modules are installed             ... yes

Docker
======

* [optional] Docker is available           ... yes
* [optional] Docker-compose is available   ... yes (version unknown)
* [optional] Docker is running             ... yes

However, NVM is installed!

---

I really need this feature for a client and I thought it would be a matter of install deps and write some tests, but... I can't even run the development environment... 😞 I tried on Windows, on Ubuntu... Should I light a candle?

[robsonsobral]

Ok! Nothing a rm -rf ./node_modules can't solve!

How should I proceed, @jeherve ?

$ jetpack build
? What type of project are you working on today? (Use arrow keys)
❯ github-actions
  js-packages
  packages
  plugins

(I'm afraid of breaking something again)

[coder-karen]

Hi @robsonsobral ! The next step would be 'plugins', then 'jetpack' (I don't believe you should need to build dependencies which is the last prompt, but then again it shouldn't break anything if you do). You can also type in jetpack build plugins/jetpack directly, instead.

[robsonsobral]

Sooooo, I'm back to this point. I log into wp-admin, go to plugins, but I can't activate Jetpack.

I tried:

• Akismet;
• Jetpack Debug Tools;
• Core Control...

They activate just fine. No problem.

However, I can't activate Jetpack Version 10.9-a.0:

Plugin could not be activated because it triggered a fatal error.

Warning: require_once(/usr/local/src/jetpack-monorepo/projects/plugins/jetpack/jetpack_vendor/automattic/jetpack-assets/actions.php): Failed to open stream: No such file or directory in /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/jetpack-autoloader/class-version-loader.php on line 89

I'm working on master branch, just after the commit 79e490c from 2022-03-30. Am I on the right place, @coder-karen ?

Thank you so much for your attention and help.

[coder-karen]

You should still be able to build the Jetpack plugin and access it from the master branch, though if you're going to make any file changes you'd need to create a new feature branch. That said it should still ideally be working there on master.

Out of curiosity, do you get any errors when running pnpm install && pnpm cli-setup ? If a node version incompatibility is mentioned when running that, how does that compare with node -v? I know you shared the results of the development checker above but what I'm wondering is if there is another version installed elsewhere.

If the correct version of node shows when running node -v, possibly try running npm -g install pnpm.

Something else to try - nvm install && nvm use then run nvm list to confirm the versions. It may be worth trying node version 16.13.2 - if that shows up when using nvm list you could choose it with nvm use 16.13.2, otherwise nvm install 16.13.2 first.

It may also be worth re-running the command to update Jetpack dependencies: jetpack install -r && jetpack install --all. Do let us know how you get on here.

[robsonsobral]

pnpm install && pnpm cli-setup gave me an error:

Error: Command failed with exit code 2: composer update
    at makeError (/mnt/e/_repositories/jetpack/node_modules/.pnpm/execa@5.0.0/node_modules/execa/lib/error.js:59:11)
    at handlePromise (/mnt/e/_repositories/jetpack/node_modules/.pnpm/execa@5.0.0/node_modules/execa/index.js:114:26)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  shortMessage: 'Command failed with exit code 2: composer update',
  command: 'composer update',
  exitCode: 2,
  signal: undefined,
  signalDescription: undefined,
  stdout: undefined,
  stderr: undefined,
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false,
  context: [Object: null prototype] {}
}

So, what I did, nvm use --delete-prefix v16.13.2, npm -g uninstall pnpm and npm -g install pnpm@^6.23.6.

Now, pnpm install && pnpm cli-setup runs without errors! However, I still can't activate Jetpack. So, let's try jetpack install -r && jetpack install --all. And I get back to:

Error: Command failed with exit code 2: composer update
    at makeError (/mnt/e/_repositories/jetpack/node_modules/.pnpm/execa@5.0.0/node_modules/execa/lib/error.js:59:11)
    at handlePromise (/mnt/e/_repositories/jetpack/node_modules/.pnpm/execa@5.0.0/node_modules/execa/index.js:114:26)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  shortMessage: 'Command failed with exit code 2: composer update',
  command: 'composer update',
  exitCode: 2,
  signal: undefined,
  signalDescription: undefined,
  stdout: undefined,
  stderr: undefined,
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false,
  context: [Object: null prototype] {}
}

Then, again rm -rf node_modules/, pnpm install && pnpm cli-setup and jetpack build plugins/jetpack... No luck!

Let's restart Docker and...

There has been a critical error on this website.

Dammed... Let's reset the Docker image!

No luck! Back to not be able to activate Jetpack:

Warning: require_once(/usr/local/src/jetpack-monorepo/projects/plugins/jetpack/jetpack_vendor/automattic/jetpack-assets/actions.php): Failed to open stream: No such file or directory in /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/jetpack-autoloader/class-version-loader.php on line 89

Fatal error: Uncaught Error: Failed opening required '/usr/local/src/jetpack-monorepo/projects/plugins/jetpack/jetpack_vendor/automattic/jetpack-assets/actions.php' (include_path='.:/usr/share/php')
in /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/jetpack-autoloader/class-version-loader.php on line 89

Call stack:

   Automattic\J\A\jpf11009ded9fc4592b6a05b61ce272b3c_jetpackⓥ10_9_a_0\Version_Loader::load_filemap()
   /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/jetpack-autoloader/class-autoloader-handler.php:124
   Automattic\J\A\jpf11009ded9fc4592b6a05b61ce272b3c_jetpackⓥ10_9_a_0\Autoloader_Handler::activate_autoloader()
   /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/jetpack-autoloader/class-autoloader.php:80
   Automattic\J\A\jpf11009ded9fc4592b6a05b61ce272b3c_jetpackⓥ10_9_a_0\Autoloader::init()
   /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/autoload_packages.php:13
   require_once()
   /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/jetpack.php:141
   include_once()
   wp-admin/includes/plugin.php:2313
   plugin_sandbox_scrape()
   wp-admin/plugins.php:192

Query Monitor

Fatal error: Uncaught Error: Failed opening required '/usr/local/src/jetpack-monorepo/projects/plugins/jetpack/jetpack_vendor/automattic/jetpack-assets/actions.php' (include_path='.:/usr/share/php') in /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/jetpack-autoloader/class-version-loader.php:89 Stack trace: #0 /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/jetpack-autoloader/class-autoloader-handler.php(124): Automattic\Jetpack\Autoloader\jpf11009ded9fc4592b6a05b61ce272b3c_jetpackⓥ10_9_a_0\Version_Loader->load_filemap() #1 /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/jetpack-autoloader/class-autoloader.php(80): Automattic\Jetpack\Autoloader\jpf11009ded9fc4592b6a05b61ce272b3c_jetpackⓥ10_9_a_0\Autoloader_Handler->activate_autoloader() #2 /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/autoload_packages.php(13): Automattic\Jetpack\Autoloader\jpf11009ded9fc4592b6a05b61ce272b3c_jetpackⓥ10_9_a_0\Autoloader::init() #3 /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/jetpack.php(141): require_once('...') #4 /var/www/html/wp-admin/includes/plugin.php(2313): include_once('...') #5 /var/www/html/wp-admin/plugins.php(192): plugin_sandbox_scrape() #6 {main} thrown in /usr/local/src/jetpack-monorepo/projects/plugins/jetpack/vendor/jetpack-autoloader/class-version-loader.php on line 89

( I didn't know there's more info on the message. They are hidden beyond the scroll!)

Thank you for your patience!

[jeherve]

Interestingly, it always seems to come back to issues with composer on your setup, but I'm not sure why. What happens when you run composer install on the root of the monorepo, or cd projects/plugins/jetpack && composer install?

[robsonsobral]

Test 1:

$ composer install
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Package operations: 0 installs, 0 updates, 0 removals
Generating autoload files
2 packages you are using are looking for funding.
Use the `composer fund` command to find out more!

No luck!

Test 2:

$ cd projects/plugins/jetpack && composer install
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Nothing to install, update or remove
Generating autoload files
Generating jetpack autoloader (/mnt/e/_repositories/jetpack/projects/plugins/jetpack/vendor/jetpack-autoloader/)
  Generated: ../autoload_packages.php
  Generated: class-autoloader-handler.php
  Generated: class-autoloader-locator.php
  Generated: class-autoloader.php
  Generated: class-container.php
  Generated: class-hook-manager.php
  Generated: class-latest-autoloader-guard.php
  Generated: class-manifest-reader.php
  Generated: class-path-processor.php
  Generated: class-php-autoloader.php
  Generated: class-plugin-locator.php
  Generated: class-plugins-handler.php
  Generated: class-shutdown-handler.php
  Generated: class-version-loader.php
  Generated: class-version-selector.php
Generating jetpack autoloader manifests (/mnt/e/_repositories/jetpack/projects/plugins/jetpack/vendor/composer)
  Generated: jetpack_autoload_classmap.php
  Generated: jetpack_autoload_psr4.php
  Generated: jetpack_autoload_filemap.php
You are generating an unoptimized autoloader. If this is a production build, consider using the -o option.
36 packages you are using are looking for funding.
Use the `composer fund` command to find out more!

No luck!

Another suggestion, @jeherve ?

(Thank you so much you all)

[jeherve]

Alright. Does the same fatal error remain now, or is it different?

Do you see the composer dependencies when navigating to the plugin folder inside the Docker container?

$ jetpack docker sh

$ cd wp-content/plugins/jetpack && ls jetpack_vendor/automattic/jetpack-assets/

[robsonsobral]

It's always the same error, @jeherve.

cannot access 'jetpack_vendor/automattic/jetpack-assets/': No such file or directory

The contents of /jetpack_vendor/automattic:

jetpack-a8c-mc-stats  jetpack-connection             jetpack-jitm              jetpack-plugins-installer
jetpack-abtest        jetpack-connection-ui          jetpack-lazy-images       jetpack-redirect
jetpack-admin-ui      jetpack-constants              jetpack-licensing         jetpack-roles
jetpack-assets        jetpack-device-detection       jetpack-logo              jetpack-search
jetpack-backup        jetpack-error                  jetpack-my-jetpack        jetpack-status
jetpack-blocks        jetpack-google-fonts-provider  jetpack-options           jetpack-sync
jetpack-compat        jetpack-heartbeat              jetpack-partner           jetpack-tracking
jetpack-config        jetpack-identity-crisis        jetpack-password-checker

Being jetpack-assets a symlink to /mnt/e/_repositories/jetpack/projects/packages/assets, which indeed exists. So, it looks like an issue with WSL.

[jeherve]

Ah, interesting. Just to be sure, are you using WSL version 2?

[github-actions]

Support References

This comment is automatically generated. Please do not edit it.

• 3558150-zen
• -zen