Add controller

[ianstormtaylor]

This is a structural change to the architecture of some of the core parts of Slate. Specifically it replaces the old "pseudo-models", in the Stack, Schema and History with a new Editor controller. In doing so it upgrades the plugin middleware stack to be more flexible. All with the goal of simplifying the core architecture, making Slate easier to use server-side, in tests and in other view layers, and making more of the core behaviors of Slate more customizable by plugins.

---

NEW

Introducing the Editor controller. Previously there was a vague editor concept, that was the React component itself. This was helpful, but because it was tightly coupled to React and the browser, it didn't lend itself to non-browser use cases well. This meant that the line between "model" and "controller/view" was blurred, and some concepts lived in both places at once, in inconsistent ways.

A new Editor controller now makes this relationship clear. It borrows many of its behaviors from the React <Editor> component. And the component actually just instantiates its own plain JavaScript Editor under the covers to delegate the work to.

This new concept powers a lot of the thinking in this new version, unlocking a lot of changes that bring a clearer separation of responsibilities to Slate. It allows us to create editors in any environment, which makes server-side use cases easier, brings parity to testing, and even opens us up to supporting other view layers like React Native or Vue.js in the future.

It has a familiar API, based on the existing editor concept:

const editor = new Editor({ plugins, value, onChange })

editor.change(change => {
  ...
})

However it also introduces imperative methods to make testing easier:

editor.run('renderNode', props)

editor.event('onKeyDown', event)

editor.command('addMark', 'bold')

editor.query('isVoid', node)

I'm very excited about it, so I hope you like it!

The <Editor> can now choose to not normalize on mount. A nice side effect of splitting out the Editor logic into a reusable place is that it's easier to implement customizable behaviors for normalization. You can now pass an options={{ normalize: false }} prop to the React <Editor> which will disable the default normalization that takes place when the editor is constructed. This is helpful in cases where you are guaranteed to have an already normalized value, and don't want to incur the performance cost of normalizing it again.

Introducing the "commands" concept. Previously, "change methods" were treated in a first-class way, but plugins had no easy way to add their own change methods that were reusable elsewhere. And they had no way to override the built-in logic for certain commands, for example splitBlock or insertText. However, now this is all customizable by plugins, with the core Slate plugin providing all of the previous default commands.

const plugin = {
  commands: {
    wrapQuote(change) {
      change.wrapBlock('quote')
    }
  }
}

Those commands are then available directly on the change objects, which are now editor-specific:

change.wrapQuote()

This allows you to define all of your commands in a single, easily-testable place. And then "behavioral" plugins can simply take command names as options, so that you have full control over the logic they trigger.

Introducing the "queries" concept. Similarly to the commands, queries allow plugins to define specific behaviors that the editor can be queried for in a reusable way, to be used when rendering buttons, or deciding on command behaviors, etc.

For example, you might define an getActiveList query:

const plugin = {
  queries: {
    getActiveList(value) {
     
    }
  }
}

And then be able to re-use that logic easily in different places in your codebase, or pass in the query name to a plugin that can use your custom logic itself:

const { value } = change
const list = change.getActiveList(value)

if (list) {
  ...
} else {
  ...
}

Taken together, commands and queries offer a better way for plugins to manage their inter-dependencies. They can take in command or query names as options to change their behaviors, or they can export new commands and queries that you can reuse in your codebase.

The middleware stack is now deferrable. With the introduction of the Editor controller, the middleware stack in Slate has also been upgraded. Each middleware now receives a next function (similar to Express or Koa) that allows you to choose whether to iterating the stack or not.

// Previously, you'd return `undefined` to continue.
function onKeyDown(event, change, editor) {
  if (event.key !== 'Enter') return
  ...
}

// Now, you call `next()` to continue...
function onKeyDown(event, change, next) {
  if (event.key !== 'Enter') return next()
  ...
}

While that may seem inconvenient, it opens up an entire new behavior, which is deferring to the plugins later in the stack to see if they "handle" a specific case, and if not, handling it yourself:

function onKeyDown(event, change, next) {
  if (event.key === 'Enter') {
    const handled = next()
    if (handled) return handled

    // Otherwise, handle `Enter` yourself...
  }
}

This is how all of the core logic in slate-react is now implemented, eliminating the need for a "before" and an "after" plugin that duplicate logic.

Under the covers, the schema, commands and queries concept are all implemented as plugins that attach varying middleware as well. For example, commands are processed using the onCommand middleware under the covers:

const plugin = {
  onCommand(command, change, next) {
    ...
  }
}

This allows you to actually listen in to all commands, and override individual behaviors if you choose to do so, without having to override the command itself. This is a very advanced feature, which most people won't need, but it shows the flexibility provided by migrating all of the previously custom internal logic to be based on the new middleware stack.

Plugins can now be defined in nested arrays. This is a small addition, but it means that you no longer need to differentiate between individual plugins and multiple plugins in an array. This allows plugins to be more easily composed up from multiple other plugins themselves, without the end user having to change how they use them. Small, but encourages reuse just a little bit more.

BREAKING

The Value object is no longer tied to changes. Previously, you could create a new Change by calling value.change() and retrieve a new value. With the re-architecture to properly decouple the schema, commands, queries and plugins from the core Slate data models, this is no longer possible. Instead, changes are always created via an Editor instance, where those concepts live.

// Instead of...
const { value } = this.state
const change = value.change()
...
this.onChange(change)

// You now would do...
this.editor.change(change => {
  const { value } = change
  ...
})

Sometimes this means you will need to store the React ref of the editor to be able to access its editor.change method in your React components.

Remove the Stack "model", in favor of the Editor. Previously there was a pseudo-model called the Stack that was very low level, and not really a model. This concept has now been rolled into the new Editor controller, which can be used in any environment because it's just plain JavaScript. There was almost no need to directly use a Stack instance previously, so this change shouldn't affect almost anyone.

Remove the Schema "model", in favor of the Editor. Previously there was another pseudo-model called the Schema, that was used to contain validation logic. All of the same validation features are still available, but the old Schema model is now rolled into the Editor controller as well, in the form of an internal SchemaPlugin that isn't exposed.

Remove the schema.isVoid and schema.isAtomic in favor of queries. Previously these two methods were used to query the schema about the behavior of a specific node or decoration. Now these same queries as possible using the "queries" concept, and are available directly on the change object:

if (change.isVoid(node)) {
  ...
}

The middleware stack must now be explicitly continued, using next. Previously returning undefined from a middleware would (usually) continue the stack onto the next middleware. Now, with middleware taking a next function argument you must explicitly decide to continue the stack by call next() yourself.

Remove the History model, in favor of commands. Previously there was a History model that stored the undo/redo stacks, and managing saving new operations to those stacks. All of this logic has been folded into the new "commands" concept, and the undo/redo stacks now live in value.data. This has the benefit of allowing the history behavior to be completely overridable by userland plugins, which was not an easy feat to manage before.

The editor object is no longer passed to event handlers. Previously, the third argument to event handlers would be the React editor instance. However, now that Change objects contain a direct reference to the editor, you can access this on change.editor instead.

function onKeyDown(event, change, next) {
  const { editor } = change
  ...
}

In its place is the new next argument, which allows you to choose to defer to the plugins further on the stack before handling the event yourself.

Values can no longer be normalized on creation. With the decoupling of the data model and the plugin layer, the schema rules are no longer available inside the Value model. This means that you can no longer receive a "normalized" value without having access to the Editor and its plugins.

// While previously you could attach a `schema` to a value...
const normalized = Value.create({ ..., schema })

// Now you'd need to do that with the `editor`...
const value = Value.create({ ... })
const editor = new Editor({ value, plugins: [{ schema }] })
const normalized = editor.value

While this seems inconvenient, it makes the boundaries in the API much more clear, and keeps the immutable and mutable concepts separated. This specific code sample gets longer, but the complexities elsewhere in the library are removed.

The Change class is no longer exported. Changes are now editor-specific, so exporting the Change class no longer makes sense. Instead, you can use the editor.change() API to receive a new change object with the commands and queries specific to your editor's plugins.

slate-hyperscript no longer normalizes values. This behavior was very problematic because it meant that you could not determine exactly what output you'd receive from any given hyperscript creation. The logic for creating child nodes was inconsistent, relying on the built-in normalization to help keep it "normal". While this is sometimes helpful, it makes writing tests for invalid states very tricky, if not impossible.

Now, slate-hyperscript does not do any normalization, meaning that you can create any document structure with it. For example, you can create a block node inside an inline node, even though a Slate editor wouldn't allow it. Or, if you don't create leaf text nodes, they won't exist in the output.

For example these are no longer equivalent:

<document>
  <paragraph>
    <link>word</link>
  </paragraph>
</document>

<document>
  <paragraph>
    <text />
    <link>word</link>
    <text />
  </paragraph>
</document>

Similarly, these are no longer equivalent either:

<document>
  <paragraph />
</document>

<document>
  <paragraph>
    <text />
  </paragraph>
</document>

This allows you to much more easily test invalid states and transition states. However, it means that you need to be more explicit in the "normal" states than previously.

The <text> and <mark> creators now return useful objects. This is a related change that makes the library more useful. Previously you could expect to receive a value from the <value> creator, but the others were less consistent. For example, the <text> creator would actually return an array, instead of the Text node that you expect.

// Previously you had to do...
const text = <text>word</text>[0]

// But now it's more obvious...
const text = <text>word</text>

Similarly, the mark creator used to return a Text node. Now it returns a list of Leaf objects, which can be passed directly as children to the <text> creator.

The findRange, findPoint, cloneFragment, and getEventRange utils now take an editor. Previously these utility functions took a schema argument, but this has been replaced with the new editor controller instead now that the Schema model has been removed.

The getClosestVoid, getDecorations and hasVoidParent method now take an editor. Previously these Node methods took a schema argument, but this has been replaced with the new editor controller instead now that the Schema model has been removed.

The slate-simulator is deprecated. Previously this was used as a pseudo-controller for testing purposes. However, now with the new Editor controller as a first-class concept, everything the simulator could do can now be done directly in the library. This should make testing in non-browser environments much easier to do.

---

Fixes: #2045
Fixes: #2152
Fixes: #2066
Fixes: #2214
Fixes: #837
Fixes: #2199
Fixes: #2176
Fixes: #2206

Addresses: #1013
Addresses: #1411
Addresses: #2098 (comment)
Addresses: #1456
Addresses: #1730
Addresses: #1311
Addresses: #2190

[ericedem]

Haven't looked through everything yet, just some questions around rendering and which values correspond to what within the editor.

[ericedem]

Maybe I'm not understanding, but shouldn't this be checking against the value being passed in? Like: value === this.tmp.lastValue?

[ianstormtaylor]

Yup, you're right! Good catch.

[ericedem]

The editor will only rerender if there is a props change, should we only render prop.value here instead? Or maybe tmp.change should be state.change if we want a render cycle to happen.

[ianstormtaylor]

@ericedem not sure I understand what you mean. I believe here tmp.change should only be present if we needed to normalize the value before the component was mounted and ready to receive changes. So it's just saying that if there's an interim value, use it, since it will soon be flushed via onChange.

[ianstormtaylor]

Op! Sorry, I just got what you mean now. You're right, since we're already rendering via editor.value we're already achieving this, and the tmp.change logic here isn't doing anything!

[ianstormtaylor]

@ericedem thanks, that is super helpful!

[bryanph]

Damn that's quick! Looking forward to a writeup

[codecov]

Codecov Report

Merging #2221 into master will increase coverage by 0.85%.
The diff coverage is 92.45%.

@@            Coverage Diff            @@
##           master   #2221      +/-   ##
=========================================
+ Coverage   82.65%   83.5%   +0.85%     
=========================================
  Files          40      42       +2     
  Lines        4035    4092      +57     
=========================================
+ Hits         3335    3417      +82     
+ Misses        700     675      -25

Impacted Files	Coverage Δ
packages/slate/src/models/operation.js	47.57% <ø> (+0.45%)	⬆️
packages/slate/src/models/node.js	55% <ø> (+2.5%)	⬆️
packages/slate/src/utils/is-object.js	100% <ø> (ø)	⬆️
packages/slate/src/interfaces/model.js	33.33% <ø> (ø)	⬆️
packages/slate/src/models/point.js	64.89% <ø> (ø)	⬆️
packages/slate/src/models/text.js	92.05% <ø> (+0.93%)	⬆️
packages/slate/src/controllers/change.js	90.75% <100%> (ø)
packages/slate/src/commands/at-current-range.js	99.13% <100%> (ø)
packages/slate/src/models/leaf.js	93.75% <100%> (+5.43%)	⬆️
packages/slate/src/interfaces/element.js	75.58% <100%> (+0.08%)	⬆️
... and 25 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e6372d8...72ebb39. Read the comment docs.

[ericedem]

This is really great, looking forward to this!

On the logistics side, is there anything you'd like from the community in terms of testing before this gets rolled out? Given how major this change is, it seems likely there will be a lot of bugs in the semver this lands in (though this will should fix a bunch too).

[ianstormtaylor]

@ericedem thanks! Nothing special compared to normal releases with breaking changes I think.

[justinweiss]

Would it be possible or worthwhile to be able to generate a new editor, cloning the same properties as an old one?

(new Editor(this.editor)).change(c => c.insertText()).value wouldn't be much worse than value.change().insertText().value, if you just want to modify a value with some changes.

[ianstormtaylor]

@justinweiss I'm not sure what you mean. Could you give me a code sample or use case?

[justinweiss]

@ianstormtaylor Never mind! I thought I was using value.change() more than I actually was. Nothing looks like it would be too difficult to move to Editor.

[amitm02]

Look awesome @ianstormtaylor!
On my current project, the Value is always stored in a Redux state container. Any change to the Value state in the store results in a setState() that re-renders the React editor.
The triggers for changing the Value can come from either the React Editor, or from other external sources.
So, while some of the changes to the Value are made via the react editor, others are made outside of the react editor, via value.change() and the result is stored in the Redux directly.

What is the best approach to handle it with the new version now that value.change() is no longer available?
Do i need to create a second "mocked" editor for the externally triggered changes?

[ianstormtaylor]

@amitm02 yup, I'd do it exactly as you said. Create another non-React Editor instance with the same plugins if you need to edit it outside of the React editor. (Or with only the core plugins if you don't care about mismatches.)

[skogsmaskin]

@ianstormtaylor - Man! After having refactored our code during the weekend for the new release, I just want to say that this change is really awesome! There were a lot to refactor though (we relied on Value.change() a lot for instance), but everything is getting so much cleaner now. I especially love the fact that I can override built in stuff like splitBlock which was a real troublemaker for us (various hacks can now be removed). Also love the new query-interface which let me remove a lot of code around the codebase, and have in one central place. Great work 👍

[ianstormtaylor]

@skogsmaskin thank you, that means a lot to me. I get a bit nervous sometimes to make breaking changes, but it’s really nice to hear that they laid off like we thought!