[DocDB] WaitForTermination core dump occured in snapshot isolation bank workload test with network nemesis and VM graceful restarts

[qvad]

Jira Link: DB-14763

Description

During our nemesis snapshot isolation testing with bank workload, we observed the following core dump on 2.25.0.0-b398
Does not reproduced since then

Network package loss and VM restart nemesis were used.

(lldb) target create "/home/yugabyte/master/bin/yb-master" --core "/home/yugabyte/cores/core_yb.1733321120.yb-master.1123.14688"
Core file '/home/yugabyte/cores/core_yb.1733321120.yb-master.1123.14688' (aarch64) was loaded.
(lldb) bt all
* thread #1, name = 'yb-master', stop reason = signal SIGABRT
  * frame #0: 0x0000ffff9677dc60 libpthread.so.0`pthread_cond_wait@@GLIBC_2.17 + 528
    frame #1: 0x0000aaaaaecc3bf0 yb-master`yb::TerminationMonitor::WaitForTermination() [inlined] void std::__1::condition_variable::wait<yb::TerminationMonitor::WaitForTermination()::$_0>(this=0x000015213fdb4338, __lk=<unavailable>, __pred=(unnamed class) @ x19) at condition_variable.h:148:5
    frame #2: 0x0000aaaaaecc3bd8 yb-master`yb::TerminationMonitor::WaitForTermination(this=0x000015213fdb4300) at termination_monitor.cc:68:16
    frame #3: 0x0000aaaaae8a3440 yb-master`main [inlined] yb::master::MasterMain(argc=<unavailable>, argv=<unavailable>) at master_main.cc:162:24
    frame #4: 0x0000aaaaae8a2920 yb-master`main(argc=<unavailable>, argv=<unavailable>) at master_main.cc:190:12
    frame #5: 0x0000ffff967d4384 libc.so.6`__libc_start_main + 220
    frame #6: 0x0000aaaaae715034 yb-master`_start + 52
  thread #2, stop reason = signal 0
    frame #0: 0x0000ffff96884220 libc.so.6`epoll_pwait + 136
    frame #1: 0x0000aaaaaecc4c58 yb-master`epoll_poll + 88
    frame #2: 0x0000aaaaaecc73dc yb-master`ev_run + 1596
    frame #3: 0x0000aaaaafc6a2bc yb-master`yb::rpc::Reactor::RunThread() [inlined] ev::loop_ref::run(this=0x000015213f869bb8, flags=0) at ev++.h:211:7
    frame #4: 0x0000aaaaafc6a2b4 yb-master`yb::rpc::Reactor::RunThread(this=0x000015213f869b80) at reactor.cc:733:9
    frame #5: 0x0000aaaab05a38d8 yb-master`yb::Thread::SuperviseThread(void*) [inlined] std::__1::__function::__value_func<void ()>::operator()[abi:ue170006](this=0x000015213fde3950) const at function.h:517:16
    frame #6: 0x0000aaaab05a38c4 yb-master`yb::Thread::SuperviseThread(void*) [inlined] std::__1::function<void ()>::operator()(this=0x000015213fde3950) const at function.h:1168:12
    frame #7: 0x0000aaaab05a38c4 yb-master`yb::Thread::SuperviseThread(arg=0x000015213fde38f0) at thread.cc:895:3
    frame #8: 0x0000ffff967778b8 libpthread.so.0`start_thread + 392
    frame #9: 0x0000ffff967d3afc libc.so.6`thread_start + 12

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

• I confirm this issue does not contain any sensitive information.