Add plugin capability

[muenzpraeger]

Shiki would benefit from providing a pluggable plugin architecture.

Why?

• Modifications to the current HTML structure are only possible if someone uses renderHTML directly and provides their element definition.
• Modifications can be shared, but there are not built-in merge mechanisms if someone wants to apply their own modifications.
• By having plugins it frees project owners/contributors from work/requests - others can built themselves. ;-)
• It builds an ecosystem.

How?
Good question. I started to design an (opinionated) approach, you can look at it in #381

[Snapstromegon]

I'm not quite sure wether this comment is best placed here or under #381, but I tend to here, so I'll place it here.

I agree that a plugin feature is really missing from shiki, but the current plugin proposal doesn't fit all usecases that I have for plugins.

My Usecases

1. Embedding into SSGs

I want to use shiki in a SSG where I add it as a wrapped plugin that is configured once to my likings and then called many times (once for each codeblock) and I want to keep config between calls.

2. Wrapping Output

I want to add additional custom html around what shiki renders to e.g. add a header that shows the language and file name. This might be based on content of the lang and code.

3. Modify input code as text

My usecase uses something similar to the #5 decoration API where I add markers for enhancement or file name to the first line of a code block like this:

```js{!sh! numbered}:test.js
// ...
```

This currently breaks tokenization, because js{!sh! numbered}:test.js is not a valid language.
In the same type I might want to do something like this:

  function greet(name) {
-   console.log(`Hello ${name}`);
+   console.log(`Hello ${name || "World"}`);
  }

to show diffs or for line numbers this:

15 function greet(name) {
16   console.log(`Hello ${name || "World"}`);
...
25 }

In both cases a plugin would need to modify the code before it is passed into tokenization.
In the line number case it's also important that the plugin is able to set arbritrary CSS on the line element, so a css var can be used to create purely CSS based line numbers.

4. Change tokenized data

I currently use inline comments to add modifiers to lines (similar to how [torchlight.dev] does this):

function greet(name) {
  console.log(`Hello ${name || "World"}`); // [sh! highlight]
}

This would be rendered like the following with the second line highlighted via a class:

function greet(name) {
  console.log(`Hello ${name || "World"}`);
}

This requires that a plugin is able to alter the tokenized code (so comments are correctly detected and can be analyzed).

5. Remove classes / attributes

I came across the issue where the name of my theme was already used in my CSS, so I didn't want the theme name to appear as a class.
In another project shiki was also already used.

Requirements from these usecases

At least three hooks are needed for plugins:

1. before tokenization to modify code and lang
2. after tokenization, before render to modify tokens
3. after render to wrap the existing html -> unlike the current proposal I think this should receive the whole HTML at once, maybe there can be a separate hbook for when a tag was rendered

Also unlike in the current proposal plugins shouldn't be cleared between each render call. I think in those cases a second instance of highlighter should be used. This might require some API changes though.

For usecase number 5, the current proposal has a search and replace based solution, but I think that one is really not good, especially when you keep potential bugs with replacements in mind like this plugin:

module.exports = config => {
  return {
    name: 'pluginHookAfter',
    hooks: {
      after: context => {
        return context.elementType === 'pre'
          ? { html: context.html.replace(/class="shiki/, 'class="highlighter') }
          : { html: context.html }
      }
    }
  }
}

would make this:

Let's say you have an attribute class="shiki" and you want to rename it to class="highlighter".

into this:

Let's say you have an attribute class="highlighter" and you want to rename it to class="highlighter".

This could be fixed, if the tags are functions instead of objects, which can modify the tag. But I don't know if that's the best option.

---

As you can see, I have some thoughts and I hope the final solution can match my usecases.