Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] ShardedLRUCache may coredump when destructor was called #10991

Closed
3 tasks done
liaoxin01 opened this issue Jul 19, 2022 · 0 comments · Fixed by #10995
Closed
3 tasks done

[Bug] ShardedLRUCache may coredump when destructor was called #10991

liaoxin01 opened this issue Jul 19, 2022 · 0 comments · Fixed by #10995

Comments

@liaoxin01
Copy link
Contributor

liaoxin01 commented Jul 19, 2022

Search before asking

  • I had searched in the issues and found no similar issues.

Version

master

What's Wrong?

==360464==ERROR: AddressSanitizer: heap-use-after-free on address 0x612000521148 at pc 0x55cc6f2ae7e7 bp 0x7f184c5f5640 sp 0x7f184c5f5630
READ of size 8 at 0x612000521148 thread T4 (calculate_metri)
#0 0x55cc6f2ae7e6 in doris::LRUCache::get_capacity() const /mnt/data1/yixiu/selectdb-core/be/src/olap/lru_cache.h:326
#1 0x55cc6f2a90f9 in doris::ShardedLRUCache::update_cache_metrics() const /mnt/data1/yixiu/selectdb-core/be/src/olap/lru_cache.cpp:534
#2 0x55cc6f2b110d in void std::_invoke_impl<void, void (doris::ShardedLRUCache::&)() const, doris::ShardedLRUCache&>(std::_invoke_memfun_deref, void (doris::ShardedLRUCache::&)() const, doris::ShardedLRUCache&) /mnt/data1/ldb_toolchain/include/c++/11/bits/invoke.h:74
#3 0x55cc6f2b0f7a in std::_invoke_result<void (doris::ShardedLRUCache::&)() const, doris::ShardedLRUCache&>::type std::_invoke<void (doris::ShardedLRUCache::&)() const, doris::ShardedLRUCache&>(void (doris::ShardedLRUCache::&)() const, doris::ShardedLRUCache&) /mnt/data1/ldb_toolchain/include/c++/11/bits/invoke.h:96
#4 0x55cc6f2b0eb1 in void std::Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>::_call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /mnt/data1/ldb_toolchain/include/c++/11/functional:420
#5 0x55cc6f2b0d62 in void std::_Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>::operator()<, void>() /mnt/data1/ldb_toolchain/include/c++/11/functional:503
#6 0x55cc6f2b0a7f in void std::_invoke_impl<void, std::_Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>&>(std::_invoke_other, std::_Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>&) /mnt/data1/ldb_toolchain/include/c++/11/bits/invoke.h:61
#7 0x55cc6f2b04db in std::enable_if<is_invocable_r_v<void, std::Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>&>, void>::type std::_invoke_r<void, std::_Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>&>(std::_Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const>&) /mnt/data1/ldb_toolchain/include/c++/11/bits/invoke.h:111
#8 0x55cc6f2b0057 in std::_Function_handler<void (), std::_Bind<void (doris::ShardedLRUCache::(doris::ShardedLRUCache))() const> >::_M_invoke(std::_Any_data const&) /mnt/data1/ldb_toolchain/include/c++/11/bits/std_function.h:291
#9 0x55cc7017fa75 in std::function<void ()>::operator()() const /mnt/data1/ldb_toolchain/include/c++/11/bits/std_function.h:560
#10 0x55cc708293ba in doris::MetricEntity::trigger_hook_unlocked(bool) const /mnt/data1/yixiu/selectdb-core/be/src/util/metrics.cpp:262
#11 0x55cc7082a0be in doris::MetricRegistry::trigger_all_hooks(bool) const /mnt/data1/yixiu/selectdb-core/be/src/util/metrics.cpp:311
#12 0x55cc6e893cf2 in doris::Daemon::calculate_metrics_thread() /mnt/data1/yixiu/selectdb-core/be/src/common/daemon.cpp:122
#13 0x55cc6e895d89 in operator() /mnt/data1/yixiu/selectdb-core/be/src/common/daemon.cpp:294
#14 0x55cc6e89809f in __invoke_impl<void, doris::Daemon::start()::<lambda()>&> /mnt/data1/ldb_toolchain/include/c++/11/bits/invoke.h:61
#15 0x55cc6e897e43 in __invoke_r<void, doris::Daemon::start()::<lambda()>&> /mnt/data1/ldb_toolchain/include/c++/11/bits/invoke.h:111
#16 0x55cc6e897ad9 in _M_invoke /mnt/data1/ldb_toolchain/include/c++/11/bits/std_function.h:291
#17 0x55cc7017fa75 in std::function<void ()>::operator()() const /mnt/data1/ldb_toolchain/include/c++/11/bits/std_function.h:560
#18 0x55cc7098f2f1 in doris::Thread::supervise_thread(void*) /mnt/data1/yixiu/selectdb-core/be/src/util/thread.cpp:408
#19 0x7f18529f1608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8608)
#20 0x7f1852b2b132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132)

What You Expected?

fix

How to Reproduce?

ShardedLRUCache::~ShardedLRUCache() {
if (_shards) {
for (int s = 0; s < _num_shards; s++) {
delete _shards[s];
}
delete[] _shards;
}
_entity->deregister_hook(_name);
DorisMetrics::instance()->metric_registry()->deregister_entity(_entity);
}

delete _shards has a competition with MetricRegistry::trigger_all_hooks.
when called in the following order, it will coredump.
delete _shards -> MetricRegistry::trigger_all_hooks->deregister

Anything Else?

All ShardedLRUCache are global in our code now, so has a small probability of coredump.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant