Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle map representation of cache in SafeCaches #14732

Merged
merged 3 commits into from
Oct 26, 2022

Conversation

kokosing
Copy link
Member

Handle map representation of cache in SafeCaches

@findepi
Copy link
Member

findepi commented Oct 25, 2022

@kokosing the CI is unhappy

Comment on lines 90 to 129
cache.invalidateAll();
cache.asMap().clear();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add assertion that cache gets emptied by each of these

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#14653 (comment)

I don't think so. SafeCaches provides forwarding implementation of cache, so it is up to a delegate cache to clear things. So testing if methods are properly implemented should be on a different level, where a delegate cache implementation is.

I would like to implement this anyway, but the challenge is that it is not clear how to put/load data into the cache. Without it I cannot verify if clear clears.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So testing if methods are properly implemented should be on a different level

we don't have tests for SafeCaches's wrappers, which was a mistake. You're adding them now.

where a delegate cache implementation is.

We could want a "smoke test" to verify things generally work. I.e. what if SafeCaches wrapper missed a call to delegate?

Other than that, I agree there is no point in fine grained tests, we don't want to test underlying implementation here, nor concurrency, etc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but the challenge is that it is not clear how to put/load data into the cache.

maybe we can have something like

private static void verifyClearIsPossible(Cache<Object, Object> cache)
        throws Exception
{
    Object key = new Object();
    Object firstValue = new Object();
    cache.get(key, () -> firstValue);
    assertSame(cache.getIfPresent(key), firstValue);

    cache.invalidateAll();
    assertNull(cache.getIfPresent(key));

    Object secondValue = new Object();
    cache.get(key, () -> secondValue);
    assertSame(cache.getIfPresent(key), secondValue);
    cache.asMap().clear();
    assertNull(cache.getIfPresent(key));
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your example assumes that loading is possible which is no always a case.

Anyway I follow you suggestion, and I will change it later if needed (NonLoadableCache, however it is a different story)

}

@Override
public boolean replace(K key, V oldValue, V newValue)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one may actually be safe? 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we forbid without being sure, we will appear as if being sure.

I am fine with forbidding this, but let's have a comment that we don't know what we're doing.

@kokosing kokosing force-pushed the origin/master/154_safe_caches branch from bfcc204 to b4a6ee9 Compare October 25, 2022 13:12
Copy link
Member Author

@kokosing kokosing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AC

Please see Expose NonKeyEvictableLoadingCache::unsafeInvalidate which I had to add as there was actually map used to by bypass forbidden invalidation.

Comment on lines 90 to 129
cache.invalidateAll();
cache.asMap().clear();
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#14653 (comment)

I don't think so. SafeCaches provides forwarding implementation of cache, so it is up to a delegate cache to clear things. So testing if methods are properly implemented should be on a different level, where a delegate cache implementation is.

I would like to implement this anyway, but the challenge is that it is not clear how to put/load data into the cache. Without it I cannot verify if clear clears.

}

@Override
public boolean replace(K key, V oldValue, V newValue)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure.

}

private static void verifyClearIsImpossible(Cache<Object, Object> cache)
{
assertThatThrownBy(cache::invalidateAll)
.isInstanceOf(UnsupportedOperationException.class)
.hasMessage("invalidateAll does not invalidate ongoing loads, so a stale value may remain in the cache for ever. Use EvictableCache if you need invalidation, or use SafeCaches.buildNonEvictableCacheWithWeakInvalidateAll() if invalidateAll is not required for correctness");
// TODO test asMap().clear()
Map<Object, Object> map = cache.asMap();
assertThatThrownBy(map::clear).isInstanceOf(UnsupportedOperationException.class);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assert on the message

That method allows invalidation for a key where
it is known that there are no ongoing loads for it.
Also makes error messages to be consistent
@kokosing kokosing force-pushed the origin/master/154_safe_caches branch from b4a6ee9 to fa11f90 Compare October 26, 2022 11:03
@kokosing kokosing merged commit 82baea7 into trinodb:master Oct 26, 2022
@github-actions github-actions bot added this to the 401 milestone Oct 26, 2022
@colebow colebow added the no-release-notes This pull request does not require release notes entry label Oct 26, 2022
@kokosing kokosing deleted the origin/master/154_safe_caches branch February 20, 2023 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed no-release-notes This pull request does not require release notes entry
Development

Successfully merging this pull request may close these issues.

3 participants