Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUGFIX: Migration to correct order of removed node events #5363

Merged
merged 5 commits into from
Nov 17, 2024

Conversation

mhsdesign
Copy link
Member

@mhsdesign mhsdesign commented Nov 13, 2024

Resolves partially: #5364 a replay from previous betas works again
Resolves: #5352 by fixing the replay allowing users to upgrade to beta 15

migrateevents:reorderNodeAggregateWasRemoved

Upgrade instructions

Review instructions

The idea to solve the replay for both cases (when structured nodes were repaired, and when a node was moved after deletion) we could just reorder all the NodeAggregateWasRemoved events to the end.

I tested that replaying works again for both bugs.

The only downside is that the removed nodes would temporarily reappear in every workspace as they are now outdated and have to be rebased. But that can be automated as well :)

Checklist

  • Code follows the PSR-2 coding style
  • Tests have been created, run and adjusted as needed
  • The PR is created against the lowest maintained branch
  • Reviewer - PR Title is brief but complete and starts with FEATURE|TASK|BUGFIX
  • Reviewer - The first section explains the change briefly for change-logs
  • Reviewer - Breaking Changes are marked with !!! and have upgrade-instructions

@github-actions github-actions bot added the 9.0 label Nov 13, 2024
reparation for upgrading to beta 15

see also neos#5364
we would have to delete too many events that were legit as well, like a move AFTER a deletion neos#5364
@mhsdesign mhsdesign changed the title WIP: #5352 automated migration to remove invalid events while replaying BUGFIX: Migration to correct order of removed node events Nov 14, 2024
@mhsdesign mhsdesign marked this pull request as ready for review November 14, 2024 10:28
@github-actions github-actions bot added the Bug label Nov 14, 2024
@mhsdesign mhsdesign marked this pull request as draft November 14, 2024 10:36
@mhsdesign mhsdesign marked this pull request as ready for review November 15, 2024 09:38
@mhsdesign
Copy link
Member Author

As discussed with Denny, Christian, Sebastian, Bastian and others this is wrong in the sense of event sourcing as the history is changed but the best solution we have. Lets merge it and have it for beta 16!

@nezaniel
Copy link
Member

Aye, reordering is bad, then again the events never should have happened in the previous order in the first place. So this is actually correcting history.

$this->connection->executeStatement(
'DELETE FROM ' . $eventTableName . ' WHERE sequencenumber IN (:sequenceNumbers)',
[
'sequenceNumbers' => array_map(fn (EventEnvelope $eventEnvelope) => $eventEnvelope->sequenceNumber->value, $eventsToReorder)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can these delete from and below the commit statements get tooo large? I tested it with 800 events to delete and publish at once and that worked.

@nezaniel
Copy link
Member

nezaniel commented Nov 15, 2024

Worked for my project, so +1

Copy link
Member

@nezaniel nezaniel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

works for me

@kitsunet kitsunet merged commit a0a0594 into neos:9.0 Nov 17, 2024
10 checks passed
@mhsdesign mhsdesign deleted the bugfix/5352-migration-to-get-to-beta15 branch November 18, 2024 08:51
@mhsdesign
Copy link
Member Author

We found out that there is a catch after all. Its is not correct for every case to move the removal to the end.
In the case

  • create node abc
  • remove node abc
  • create node abc

the remove node MUST be in the middle position. Otherwise there will be a duplicate node entry in the content graph because we dont have constraints, and the document uri path constraint variant is violated.

This is not a problem with user generated content but for imports with deterministic node aggregate ids (which have been deleted and recreated) this will fail.
For now those projects must not apply this migration then. In case there is a project which still needs this migration but cannot use it we need to offer another way - which will be hard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: Structure adjustments "repaired" orphaned Nodes which is not replayable
3 participants