Unrecognized Blocks: Preserve content of invalid and missing blocks

[dmsnell]

Description

There are multiple places where in the normal flow of editing that the editor might corrupt or erase content from unrecognized blocks (block implementation is missing or it's "invalid" content), even from unrelated edits in separate blocks.

When the editor finds content it can't recognize or interact with the default mode of operation should be to preserve that content so that it doesn't unintentionally corrupt documents.

Currently there are a number of contexts in which this data corruption or data loss occurs and they revolve around two related issues:

• Instead of using the parsed block information from load-time we run the working attributes for a given block through its block implementation to re-generate the block output.
• In cases where we're interacting with validation and transformation we overlook any possible inner blocks a block might have.

The implications of the first issue, re-generation of block content, is that there are times when we have a set of invalidly-parsed attribute that when in turn passed back into the block implementation produce corrupt output. Consider a list block whose <ul> tag is missing. Despite loading in the entire set of bullet points and <li> tags the editor will have sourced '' as the values attribute and so will erase the existing list and write an empty one in its place. If a block is invalid or unrecognized then we should not feed the bad data into the block and expect to get good output, GIGO.

The implications of the second are that we produce a lot of false block validations and treat certain broken blocks as valid while invalidating certain valid blocks. By ignoring a block's inner blocks during this process we get around problems introduced by updates to inner blocks, which while nested inside of the parent, should be independent and unrelated to the parent's correctness. However, some blocks do base their output HTML on the presence or content of its inner blocks, making these changes meaningful. Further, some blocks upgrade and change the way they interact with inner blocks and a failure to acknowledge this will lead to the aforementioned data loss.

Issues

• Thread block source information into blocks for preservation.
  - [Blocks] Preserve source markup in invalid blocks #38460 sourceMarkup is insufficient, so this is being replaced by a better system
  - Blocks: Remember raw source block for invalid blocks. #38923 adds source attribute on invalid blocks only
• Show source markup in editor for invalid blocks (block-list/block)
• Update serializeBlock to output the source markup if block is invalid
• Fix compared markup in block comparison to make a reasonable and meaningful comparison before and after block resolution.
• Fix resolution conversion functions to make sense and incorporate innerBlocks
• Deprecate isValidBlockContent since it can't account for inner blocks
  - Blocks: Deprecate isValidBlockContent #38794
• Fix validateBlock so that it accounts for inner blocks
• Make sure that missing and invalid blocks preserve original markup on save
• Make sure that code editor uses and preserves original markup and doesn't break the post by flipping in and out of the code editor without making any edits.
• Audit and fix uses of originalContent vs. inner-block-aware sources for downloading missing block implementations from the register.

[mcsf]

Thanks for laying out your plan like this, it makes it easier to discuss and later review. There's a lot to unpack and ponder, and I can't address everything in one sitting (and I won't really be available for the next fortnight).

But I can comment on the following (emphasis mine):

When the editor finds content it can't recognize or interact with the default mode of operation should be to preserve that content so that it doesn't unintentionally corrupt documents.

I agree with most of the issues and some of the solutions, but I also wanted to stress that the compromise in Gutenberg's block-parsing pipeline has been to isolate blocks, so that corrupt/invalid markup in a block won't corrupt the rest of the document. Now, this isolation can come at the cost of further corrupting the offending block, and this is where we can take a new look at validation and recovery.

Looking at the list of proposed issues, I'm on board with the user-facing features, like providing better visual comparison when recovering blocks, and knowing how to consider the entire block subtree, etc. But I have my doubts on where exactly the API should change and how aggressively. For example:

• Can we find an optimum where a corrupted block remains isolated but we can restore it more intelligently, such as in your example of a List block with li tags but no container ul?
• Can we provide nested block with better integrity without fundamentally changing some our isolation principles?

[dmsnell]

Thank you @mcsf for your thoughts. I too will try and share more later but I want to clear up one thing as I think we aren't talking about the same problems.

isolate blocks, so that corrupt/invalid markup in a block won't corrupt the rest of the document. Now, this isolation can come at the cost of further corrupting the offending block, and this is where we can take a new look at validation and recovery.

without fundamentally changing some our isolation principles?

It reads like you are examining this from the perspective of a broken block interfering with well-understood and validated blocks. That's not the case.

So what I'm talking about doesn't change the isolation principle at all; it upholds it where the current implementation breaks that model.

1. Start with a document containing normal blocks
2. Reach a state where a block is invalidated. Could be due to corruption in the raw HTML or this could come from a version-update on some block implementation.
3. An edit in a normal, validated, recognized block can further corrupt the unrecognized block from step two, utterly and completely violating the isolation principle.

In other words I want to preserve the principle you are talking about. If I change content in a block at the end of the document it should not erase the content from an unrelated block near the beginning. It shouldn't even corrupt its neighbor blocks or its inner blocks. This is the state of things today and that's highlighted when blocks update themselves to use inner blocks where they previously didn't (and the editor only has support for the older one) or also when a block fails to validate in an easy-to-reach way (such as with the list).

The problem doesn't stem from a design principle but rather from the concept of using a block implementation to recreate new output for blocks we actively know we failed to understand. This isn't a necessary side-effect of a core system though and I believe we can resolve it to preserve that unrecognized content without giving up anything and even more, that it will resolve a number of headaches we constantly have to deal with that are due to a current implementation of the editor that violates its own principles.