Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to have different instances LocalMemoryMetadataStore that share the same state #12390

Merged
merged 1 commit into from
Oct 25, 2021

Conversation

merlimat
Copy link
Contributor

Motivation

Added the possibility to specify a name when creating instances of LocalMemoryMetadataStore so that they can refer to the same memory.

This is useful when setting up tests for which the components are instantiating the MetadataStore on their own, but we still want them to share the state (eg: brokers and bookies).

@merlimat merlimat added type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages area/test labels Oct 16, 2021
@merlimat merlimat added this to the 2.10.0 milestone Oct 16, 2021
@merlimat merlimat self-assigned this Oct 16, 2021
@merlimat merlimat added the doc-not-needed Your PR changes do not impact docs label Oct 16, 2021
return FutureUtils.exception(new MetadataStoreException(""));
private static final Map<String, NavigableMap<String, Value>> STATIC_MAPS = new MapMaker()
.weakValues().makeMap();
private static final Map<String, AtomicLong> STATIC_ID_GEN_MAP = new MapMaker()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why weak values?
This will make tests possible unpredictable, because eviction will depend on GC

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a static map, so it would be never cleaned up otherwise.

It will not make the test unpredictable because there will be hard references to the map value for each instance of LocalMemoryMetadataStore that will keep it from getting GCed.

return FutureUtils.exception(new MetadataStoreException(""));
private static final Map<String, NavigableMap<String, Value>> STATIC_MAPS = new MapMaker()
.weakValues().makeMap();
private static final Map<String, AtomicLong> STATIC_ID_GEN_MAP = new MapMaker()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why weak values?
This will make tests possible unpredictable, because eviction will depend on GC

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@eolivelli eolivelli merged commit 9f25843 into apache:master Oct 25, 2021
zeo1995 pushed a commit to zeo1995/pulsar that referenced this pull request Oct 26, 2021
* up/master:
  [C++] Fixed connection read error logging (apache#12492)
  [Pulsar SQL] Pulsar SQL support query big entry data (apache#12448)
  [Java Client] Remove data race in MultiTopicsConsumerImpl to ensure correct message order (apache#12456)
  Allow to have different instances LocalMemoryMetadataStore that share the same state (apache#12390)
  Remove unused ConsumerImpl.isTxnMessage (apache#12472)

store2.delete("/test", Optional.empty()).join();

assertFalse(store1.exists("/test").join());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@merlimat Not sure why here can get passed, 2 instance only shared the data but not the cache, after deleted data from one instance, the cache data of another instance will not be removed, so the test will get failed.

eolivelli pushed a commit to eolivelli/pulsar that referenced this pull request Nov 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/test doc-not-needed Your PR changes do not impact docs type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants