(Performance) ManagedEntities - Only update an entity if its declaration has changed

[totten]

Overview

Generally, the purpose of ManagedEntities is to keep a database-record up-to-date with the description in code.

This optimization should improve the performance of a typical system-flush/cache-clear. For example, on my local drupal-clean build (with ~65 managed entities), the typical runtime of cv flush drops from 5.1s to 1.6s.

The benefits should be more significant for systems with more managed-entities.

Before

To reconcile, we walk through all the entities and issue API calls (create, update, delete) to ensure that the database is up-to-date with the declaration.

However, during most flushes, there has been no real change in either the entity-declaration or the live-entity. Consequently, the update (usually) isn't doing anything useful.

After

We only perform an update if the entity-declaration has changed in some way.

Comments

• While working on Upgrader - Ensure rebuildMenuAndCaches() runs with fresh container #32339 with the wmff build, I noticed that rebuildMenuAndCaches() was slow -- and that the configuration declared almost 1,000 managed-entities. That's a huge number -- much more than ManagedEntities was originally designed for. This kind of configuration should see bigger gains.

• I haven't done very deep testing. I wouldn't be surprised if there are some test-failures. I've only done the quick benchmark.

• This does change the interpretation of the update policy:

  Policy	Old Interpretation	New Interpretation
  always	Re-save whenever there is a system-flush	Re-save whenever there is a change in the canonical declaration
  unmodified	Re-save whenever there is a system-flush and the user has not edited the live record.	Re-save whenever there is a change in the canonical declaration and the user has not edited the live record.
  never	Never try to re-save. Lock-in the original value.	(Unchanged)

  The new interpretation still follows the same configuration precedence (e.g. always prioritizes canonical-declarations; never prioritizes local-configuration) -- but it applies enforcement less frequently.

• What if you want strict enforcement -- i.e. your intent is to strictly prevent edits to the managed-entity?

  • Well, update=>always was the closest to doing that, but it was never strict. (You might say it was 66% of the way there.)
  • With this formulation, it still gives precedence, but it's even less strict. (You might say it's only 33% of the way there.)
  • To get this use-case to 100%, we probably need some kind of read-only flag on the record -- whenever someone tries to edit the live entity, we first ensure that it is not protected by a read-only flag.

• On a gut-level, I expect there will be some scenarios where one wants to forcibly re-save. But I'm not exactly sure where that ends-up. An advanced flush-option? Something you do for full core-upgrades... but not for intermediate cv flushes? I'm not sure For the moment, there's just a discreet PHP flag as a nod in that direction. (This has been re-worked. It does a full re-save whenever you upgrade the system, and it does a targeted re-save when you (re)enable an extension.)

[civibot]

🤖 Thank you for contributing to CiviCRM! ❤️ We will need to test and review this PR. 👷

Introduction for new contributors...

• If this is your first PR, an admin will greenlight automated testing with the command ok to test or add to whitelist.
• A series of tests will automatically run. You can see the results at the bottom of this page (if there are any problems, it will include a link to see what went wrong).
• A demo site will be built where anyone can try out a version of CiviCRM that includes your changes.
• If this process needs to be repeated, an admin will issue the command test this please to rerun tests and build a new demo site.
• Before this PR can be merged, it needs to be reviewed. Please keep in mind that reviewers are volunteers, and their response time can vary from a few hours to a few weeks depending on their availability and their knowledge of this particular part of CiviCRM.
• A great way to speed up this process is to "trade reviews" with someone - find an open PR that you feel able to review, and leave a comment like "I'm reviewing this now, could you please review mine?" (include a link to yours). You don't have to wait for a response to get started (and you don't have to stop at one!) the more you review, the faster this process goes for everyone 😄
• To ensure that you are credited properly in the final release notes, please add yourself to contributor-key.yml
• For more information about contributing, see CONTRIBUTING.md.

Quick links for reviewers...

• ➡️ PR Demo site (can be rebuilt with the command test this please).
• 📖 Review standards
• 🗒️ Review template (brief or verbose)

➡️ Online demo of this PR 🔗

[totten]

So far, new test-run is looking better for all scenarios that previously failed. Removing the "Draft" flag.