-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fix](schema scan) Fix invalid pointer access #48370
Conversation
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
run buildall |
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
TPC-H: Total hot run time: 31692 ms
|
TPC-DS: Total hot run time: 184015 ms
|
ClickBench: Total hot run time: 30.91 s
|
TeamCity be ut coverage result: |
==14365==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002873c70 at pc 0x5559a290ec05 bp 0x7f6c40385e90 sp 0x7f6c40385e88 READ of size 2 at 0x613002873c70 thread T4019 (FragmentMgrAsyn) #0 0x5559a290ec04 in std::__atomic_base::load(std::memory_order) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9 #1 0x5559a290ec04 in doris::AtomicStatus::update(doris::Status const&) /home/zcp/repo_center/doris_master/doris/be/src/common/status.h:601:44 #2 0x5559d8f289c0 in doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5 #3 0x5559d8f289c0 in void std::__invoke_impl(std::__invoke_other, doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #4 0x5559d8f289c0 in std::enable_if, void>::type std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #5 0x5559d8f289c0 in std::_Function_handler::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #6 0x5559a4140e4a in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24 #7 0x5559a4117877 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 #8 0x7f7912f84ac2 in start_thread nptl/pthread_create.c:442:8 #9 0x7f791301684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
==14365==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002873c70 at pc 0x5559a290ec05 bp 0x7f6c40385e90 sp 0x7f6c40385e88 READ of size 2 at 0x613002873c70 thread T4019 (FragmentMgrAsyn) #0 0x5559a290ec04 in std::__atomic_base::load(std::memory_order) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9 #1 0x5559a290ec04 in doris::AtomicStatus::update(doris::Status const&) /home/zcp/repo_center/doris_master/doris/be/src/common/status.h:601:44 #2 0x5559d8f289c0 in doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5 #3 0x5559d8f289c0 in void std::__invoke_impl(std::__invoke_other, doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #4 0x5559d8f289c0 in std::enable_if, void>::type std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #5 0x5559d8f289c0 in std::_Function_handler::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #6 0x5559a4140e4a in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24 #7 0x5559a4117877 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 #8 0x7f7912f84ac2 in start_thread nptl/pthread_create.c:442:8 #9 0x7f791301684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
==14365==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002873c70 at pc 0x5559a290ec05 bp 0x7f6c40385e90 sp 0x7f6c40385e88 READ of size 2 at 0x613002873c70 thread T4019 (FragmentMgrAsyn) #0 0x5559a290ec04 in std::__atomic_base::load(std::memory_order) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9 #1 0x5559a290ec04 in doris::AtomicStatus::update(doris::Status const&) /home/zcp/repo_center/doris_master/doris/be/src/common/status.h:601:44 apache#2 0x5559d8f289c0 in doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5 apache#3 0x5559d8f289c0 in void std::__invoke_impl(std::__invoke_other, doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 apache#4 0x5559d8f289c0 in std::enable_if, void>::type std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 apache#5 0x5559d8f289c0 in std::_Function_handler::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 apache#6 0x5559a4140e4a in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24 apache#7 0x5559a4117877 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 apache#8 0x7f7912f84ac2 in start_thread nptl/pthread_create.c:442:8 apache#9 0x7f791301684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
==14365==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002873c70 at pc 0x5559a290ec05 bp 0x7f6c40385e90 sp 0x7f6c40385e88 READ of size 2 at 0x613002873c70 thread T4019 (FragmentMgrAsyn) #0 0x5559a290ec04 in std::__atomic_base::load(std::memory_order) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9 #1 0x5559a290ec04 in doris::AtomicStatus::update(doris::Status const&) /home/zcp/repo_center/doris_master/doris/be/src/common/status.h:601:44 apache#2 0x5559d8f289c0 in doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5 apache#3 0x5559d8f289c0 in void std::__invoke_impl(std::__invoke_other, doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 apache#4 0x5559d8f289c0 in std::enable_if, void>::type std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 apache#5 0x5559d8f289c0 in std::_Function_handler::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 apache#6 0x5559a4140e4a in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24 apache#7 0x5559a4117877 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 apache#8 0x7f7912f84ac2 in start_thread nptl/pthread_create.c:442:8 apache#9 0x7f791301684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
==14365==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002873c70 at pc 0x5559a290ec05 bp 0x7f6c40385e90 sp 0x7f6c40385e88 READ of size 2 at 0x613002873c70 thread T4019 (FragmentMgrAsyn) #0 0x5559a290ec04 in std::__atomic_base::load(std::memory_order) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9 #1 0x5559a290ec04 in doris::AtomicStatus::update(doris::Status const&) /home/zcp/repo_center/doris_master/doris/be/src/common/status.h:601:44 #2 0x5559d8f289c0 in doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5 #3 0x5559d8f289c0 in void std::__invoke_impl(std::__invoke_other, doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #4 0x5559d8f289c0 in std::enable_if, void>::type std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #5 0x5559d8f289c0 in std::_Function_handler::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #6 0x5559a4140e4a in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24 #7 0x5559a4117877 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 #8 0x7f7912f84ac2 in start_thread nptl/pthread_create.c:442:8 #9 0x7f791301684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
What problem does this PR solve?
Follow-up: #48313
when task could not be locked successfully, the status is also deconstructed.
==14365==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002873c70 at pc 0x5559a290ec05 bp 0x7f6c40385e90 sp 0x7f6c40385e88
READ of size 2 at 0x613002873c70 thread T4019 (FragmentMgrAsyn)
#0 0x5559a290ec04 in std::__atomic_base::load(std::memory_order) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x5559a290ec04 in doris::AtomicStatus::update(doris::Status const&) /home/zcp/repo_center/doris_master/doris/be/src/common/status.h:601:44
#2 0x5559d8f289c0 in doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5
#3 0x5559d8f289c0 in void std::__invoke_impl(std::__invoke_other, doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
#4 0x5559d8f289c0 in std::enable_if, void>::type std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
#5 0x5559d8f289c0 in std::_Function_handler::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
#6 0x5559a4140e4a in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24
#7 0x5559a4117877 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
#8 0x7f7912f84ac2 in start_thread nptl/pthread_create.c:442:8
#9 0x7f791301684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)