Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

!!! FEATURE: Introduce flushByTags to taggable cache backends #2718

Merged
merged 5 commits into from
Mar 18, 2022

Conversation

Sebobo
Copy link
Member

@Sebobo Sebobo commented Mar 2, 2022

What I did

This change introduces the flushByTags method to taggable cache backends. This allows each backend to be further optimised in the future to find the best way to flush multiple tags from its storage.

Resolves: #2717

How I did it

The TaggableBackendInterface and FrontendInterface were extended with the method flushByTags and each provided backend in the core got a simple implementation for it to fulfil the interface.

This change introduces the `flushByTags` method to taggable cache backends. This allows each backend to be further optimised in the future to find the best way to flush multiple tags from its storage.

Resolves: neos#2717
@Sebobo
Copy link
Member Author

Sebobo commented Mar 2, 2022

Still have to update the tests

@Sebobo Sebobo marked this pull request as ready for review March 2, 2022 10:39
Sebobo added a commit to Sebobo/flow-development-collection that referenced this pull request Mar 2, 2022
With this change the `flushByTags` is optimised to run
in batches. The maximum batch size can and should be
configured based on the used data source.

This change relies on the interface changes in neos#2718
Sebobo added a commit to Sebobo/neos-development-collection that referenced this pull request Mar 2, 2022
Instead of calling the cache backend for each tag to flush
individually, the list of tags is passed to the backend with
the newly introduced `flushByTags` method in
neos/flow-development-collection#2718.

This allows each type of backend to optimise the flushing
of all tags, which can lead to huge performance improvements.
Especially when content is published to the live workspace
which leads to large numbers of cache tags that will be flushed.
@mficzel mficzel requested review from mficzel and albe March 14, 2022 17:44
Copy link
Member

@mficzel mficzel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine by reading. Pretty safe aswell as no functionality is changed yet.

Copy link
Member

@bwaidelich bwaidelich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 by reading, but 3 nitpicks/questions:

  1. somewhere @mficzel suggested to introduce the trivial implementation of flushByTags() in the AbstractBackend for a greater backwards compatibility. Was that idea discarded intentionally?
  2. I'd suggest not to change the RouterCachingService with this PR (see inline comment)
  3. (nitpick): I'd suggest to remove the checklist from the PR description so that it doesn't end up in the merge commit (and thus in the changelog)

@mficzel mficzel requested a review from kdambekalns March 18, 2022 15:25
@mficzel
Copy link
Member

mficzel commented Mar 18, 2022

@bwaidelich i scrapped the idea of a trivial flushByTags implementation myself because there is no base class for taggable Backends that could implement this. So this would require quite a bit of restructuring that is not required for this pr.

Copy link
Member

@kdambekalns kdambekalns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two return type errors incoming… 💥

Either make nullable as commented it change the return type description for the methods to no longer suggest NULL can be returned.

@bwaidelich
Copy link
Member

i scrapped the idea of a trivial flushByTags implementation myself because there is no base class for taggable Backends that could implement this

I thought something like

class AbstractBackend {
   // ...
    public function flushByTags(array $tags): int
    {
        if (!$this instanceof TaggableBackendInterface) {
           // throw or return 0
        }
        $flushed = 0;
        foreach ($tags as $tag) {
            $flushed += $this->flushByTag($tag);
        }
        return $flushed;
    }

but.. Not sure if the B/C is worth the weird construct in this case

The current return non nullable int type was already set in 2017.
@Sebobo
Copy link
Member Author

Sebobo commented Mar 18, 2022

i scrapped the idea of a trivial flushByTags implementation myself because there is no base class for taggable Backends that could implement this

I thought something like

class AbstractBackend {
   // ...
    public function flushByTags(array $tags): int
    {
        if (!$this instanceof TaggableBackendInterface) {
           // throw or return 0
        }
        $flushed = 0;
        foreach ($tags as $tag) {
            $flushed += $this->flushByTag($tag);
        }
        return $flushed;
    }

but.. Not sure if the B/C is worth the weird construct in this case

I don't feel good about adding tagging related functionality to the AbstractBackend. I think the handful of custom backend implementations can adjust their code without much effort. We anyway have a super simple "sample implementation" in the existing backends and can add it to the upgrade guide. And packages like Sandstorms optimised Redis backend can already be updated without the need to introduce a major version.

@mficzel mficzel merged commit 2315f2d into neos:master Mar 18, 2022
@Sebobo Sebobo deleted the feature/flush-by-tags branch March 18, 2022 21:18
@Sebobo
Copy link
Member Author

Sebobo commented Mar 18, 2022

Thx everyone!

Sebobo added a commit to Sebobo/flow-development-collection that referenced this pull request Mar 18, 2022
With this change the `flushByTags` is optimised to run
in batches. The maximum batch size can and should be
configured based on the used data source.

This change relies on the interface changes in neos#2718
neos-bot pushed a commit to neos/neos that referenced this pull request Mar 24, 2022
Instead of calling the cache backend for each tag to flush
individually, the list of tags is passed to the backend with
the newly introduced `flushByTags` method in
neos/flow-development-collection#2718.

This allows each type of backend to optimise the flushing
of all tags, which can lead to huge performance improvements.
Especially when content is published to the live workspace
which leads to large numbers of cache tags that will be flushed.
neos-bot pushed a commit to neos/fusion that referenced this pull request Mar 24, 2022
Instead of calling the cache backend for each tag to flush
individually, the list of tags is passed to the backend with
the newly introduced `flushByTags` method in
neos/flow-development-collection#2718.

This allows each type of backend to optimise the flushing
of all tags, which can lead to huge performance improvements.
Especially when content is published to the live workspace
which leads to large numbers of cache tags that will be flushed.
neos-bot pushed a commit to neos/neos that referenced this pull request Oct 7, 2022
Instead of calling the cache backend for each tag to flush
individually, the list of tags is passed to the backend with
the newly introduced `flushByTags` method in
neos/flow-development-collection#2718.

This allows each type of backend to optimise the flushing
of all tags, which can lead to huge performance improvements.
Especially when content is published to the live workspace
which leads to large numbers of cache tags that will be flushed.
neos-bot pushed a commit to neos/fusion that referenced this pull request Oct 7, 2022
Instead of calling the cache backend for each tag to flush
individually, the list of tags is passed to the backend with
the newly introduced `flushByTags` method in
neos/flow-development-collection#2718.

This allows each type of backend to optimise the flushing
of all tags, which can lead to huge performance improvements.
Especially when content is published to the live workspace
which leads to large numbers of cache tags that will be flushed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

!!! FEATURE: Introduce flushByTags to taggable cache backends
4 participants