Verify with olap + simple_queue (8)

[maximyurchuk]

Given workload (with --mode column)

https://github.com/ydb-platform/ydb/blob/main/ydb/tools/simple_queue/__main__.py

version:

maxim-yurchuk@ydb-sas-testing-0004:~/coredumps/1$ ./kikimr -V
Git info:
    Commit: 5244b5de8198c0d49a1834a2ed3771694578f78f
    Branch: heads/main
    Author: Oleg Doronin <dorooleg@yandex.ru>
    Summary: misspells have been fixed (#11242)

Other info:
    Build by: maxim-yurchuk
    Top src dir: /home/maxim-yurchuk/ydb
    Top build dir: /home/maxim-yurchuk/.ya/build
    Hostname: yurchuk-turbo.vla.yp-c.yandex.net
    Host information: 
        Linux yurchuk-turbo.vla.yp-c.yandex.net 5.4.210-39.1 #1 SMP Thu Jan 12 13:13:41 UTC 2023 x86_64

    
Build info:
    Compiler: /home/maxim-yurchuk/.ya/tools/v4/7599469156/bin/clang++
    Compiler version: 
        clang version 18.1.8
        Target: x86_64-unknown-linux-gnu
        Thread model: posix
        InstalledDir: /home/maxim-yurchuk/.ya/tools/v4/7599469156/bin
    Compile flags: -pipe -m64 -O3 -g -fdebug-default-version=4 -ggnu-pubnames -fdebug-info-for-profiling -fexceptions -fno-common -ffunction-sections -fdata-sections -fuse-init-array -fcolor-diagnostics -faligned-allocation -Werror -Wall -Wextra -Wno-parentheses -Wno-implicit-const-int-float-conversion -Wno-unknown-warning-option -D_THREAD_SAFE -D_PTHREADS -D_REENTRANT -D_LARGEFILE_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_YNDX_LIBUNWIND_ENABLE_EXCEPTION_BACKTRACE -D__LONG_LONG_SUPPORTED -DSSE_ENABLED=1 -DSSE3_ENABLED=1 -DSSSE3_ENABLED=1 -DSSE41_ENABLED=1 -DSSE42_ENABLED=1 -DPOPCNT_ENABLED=1 -DCX16_ENABLED=1 -fno-omit-frame-pointer -D_libunwind_ -DLIBCXX_BUILDING_LIBCXXRT -DNDEBUG -Wno-array-parameter -Wno-deprecate-lax-vec-conv-all -Wno-unqualified-std-cast-call -Wno-unused-but-set-parameter -Wno-implicit-function-declaration -Wno-int-conversion -Wno-incompatible-function-pointer-types -Wno-address-of-packed-member -Wno-deprecated-this-capture -Wno-c++11-narrowing-const-reference -Wno-missing-designated-field-initializers -Wno-packed-non-pod -Wno-format -Wno-vla-cxx-extension -Wno-invalid-offsetof -Wno-include-angled-in-module-purview -Wno-unused-parameter -DCATBOOST_OPENSOURCE=yes -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -mpopcnt -mcx16 -DNDEBUG -Wno-array-parameter -Wno-deprecate-lax-vec-conv-all -Wno-unqualified-std-cast-call -Wno-unused-but-set-parameter -Wno-implicit-function-declaration -Wno-int-conversion -Wno-incompatible-function-pointer-types -Wno-address-of-packed-member -Wno-deprecated-this-capture -Wno-c++11-narrowing-const-reference -Wno-missing-designated-field-initializers -Wno-packed-non-pod -Wno-format -Wno-vla-cxx-extension -Wno-invalid-offsetof -Wno-include-angled-in-module-purview -Wno-unused-parameter -std=c++20 -Wimport-preprocessor-directive-pedantic -Woverloaded-virtual -Wno-ambiguous-reversed-operator -Wno-defaulted-function-deleted -Wno-deprecated-anon-enum-enum-conversion -Wno-deprecated-enum-enum-conversion -Wno-deprecated-enum-float-conversion -Wno-deprecated-volatile -Wno-pessimizing-move -Wno-undefined-var-template -nostdinc++ -DCATBOOST_OPENSOURCE=yes -nostdinc++

GDB output:

(gdb) bt
#0  0x00007f6c740dd00b in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f6c740bc859 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x000000000e3e1b61 in NPrivate::InternalPanicImpl (line=line@entry=754, function=function@entry=0x12ff84e "~TVerifyFormattedRecordWriter", expr=expr@entry=0xe15a30 "false", file=..., 
    errorMessage=0x55f3e7178310 "tablet_id=72075186224051535;tablet_actor_id=[50007:7450091395607496173:76194];this=94502863846464;activity=0;task_id=cd64a2e4-be0011ef-bc8603e4-a5295b64::31;verification=false;fline=actor.h:100;type=N"..., errorMessageSize=232) at /home/maxim-yurchuk/ydb/util/system/yassert.cpp:90
#3  0x000000000e3dcafc in NPrivate::Panic (file=..., line=line@entry=754, function=0x12ff84e "~TVerifyFormattedRecordWriter", expr=0xe15a30 "false", format=0xef9425 " %s")
    at /home/maxim-yurchuk/ydb/util/system/yassert.cpp:55
#4  0x000000000efd2c95 in NActors::TVerifyFormattedRecordWriter::~TVerifyFormattedRecordWriter (this=<optimized out>) at /home/maxim-yurchuk/ydb/ydb/library/actors/core/log.cpp:754
#5  0x0000000015b3375f in NKikimr::NOlap::NDataReader::TActor::StateFunc (this=<optimized out>, ev=...) at /home/maxim-yurchuk/ydb/util/generic/string.h:100
#6  0x000000000efc6147 in NActors::TGenericExecutorThread::Execute (this=this@entry=0x55f2bf44b400, mailbox=mailbox@entry=0x55f3db4a0500, isTailExecution=<optimized out>)
    at /home/maxim-yurchuk/ydb/ydb/library/actors/core/executor_thread.cpp:248
#7  0x000000000efc9911 in NActors::TGenericExecutorThread::ProcessExecutorPool(NActors::IExecutorPool*)::$_0::operator()(NActors::TMailbox*, bool) const (this=this@entry=0x7f6c57e33f90, 
    mailbox=mailbox@entry=0x55f3db4a0500, isTailExecution=false) at /home/maxim-yurchuk/ydb/ydb/library/actors/core/executor_thread.cpp:425
#8  0x000000000efc9463 in NActors::TGenericExecutorThread::ProcessExecutorPool (this=this@entry=0x55f2bf44b400, pool=<optimized out>) at /home/maxim-yurchuk/ydb/ydb/library/actors/core/executor_thread.cpp:479
#9  0x000000000efca066 in NActors::TExecutorThread::ThreadProc (this=0x55f2bf44b400) at /home/maxim-yurchuk/ydb/ydb/library/actors/core/executor_thread.cpp:510
#10 0x000000000e3e55b8 in (anonymous namespace)::TPosixThread::ThreadProxy (arg=0x55f2bd347650) at /home/maxim-yurchuk/ydb/util/system/thread.cpp:244
#11 0x00007f6c74299609 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#12 0x00007f6c741b9353 in clone () from /lib/x86_64-linux-gnu/libc.so.6

See coredump on ydb-sas-testing-0004.search.yandex.net

sudo gdb /home/maxim-yurchuk/coredumps/1/kikimr /home/maxim-yurchuk/coredumps/1/kikimr.User.840203.6

Info from start logs

Dec 19 14:40:18 ydb-sas-testing-0004 kikimr_31003[840203]: GRpc memory quota was set but disabled due to issues with grpc quoter, to enable it use EnableGRpcMemoryQuota option
Dec 19 15:01:48 ydb-sas-testing-0004 kikimr_31003[840203]: VERIFY failed (2024-12-19T15:01:48.957984+0300): tablet_id=72075186224051535;tablet_actor_id=[50007:7450091395607496173:76194];this=94502863846464;activity=0;task_id=cd64a2e4-be0011ef-bc8603e4-a5295b64::31;verification=false;fline=actor.h:100;type=NActors::TEvents::TEvUndelivered;
Dec 19 15:01:48 ydb-sas-testing-0004 kikimr_31003[840203]:   ydb/library/actors/core/log.cpp:754
Dec 19 15:01:48 ydb-sas-testing-0004 kikimr_31003[840203]:   ~TVerifyFormattedRecordWriter(): requirement false failed
Dec 19 15:01:51 ydb-sas-testing-0004 kikimr_31003[840203]: 0. /home/maxim-yurchuk/ydb/util/system/yassert.cpp:83: NPrivate::InternalPanicImpl(int, char const*, char const*, int, int, int, TBasicStringBuf<char, std::__y1::char_traits<char>>, char const*, unsigned long) @ 0xE3E1B5B
Dec 19 15:01:51 ydb-sas-testing-0004 kikimr_31003[840203]: 1. /home/maxim-yurchuk/ydb/util/system/yassert.cpp:55: NPrivate::Panic(NPrivate::TStaticBuf const&, int, char const*, char const*, char const*, ...) @ 0xE3DCAFB
Dec 19 15:01:51 ydb-sas-testing-0004 kikimr_31003[840203]: 2. /home/maxim-yurchuk/ydb/ydb/library/actors/core/log.cpp:754: NActors::TVerifyFormattedRecordWriter::~TVerifyFormattedRecordWriter() @ 0xEFD2C94
Dec 19 15:01:51 ydb-sas-testing-0004 kikimr_31003[840203]: 3. /home/maxim-yurchuk/ydb/ydb/core/tx/columnshard/data_reader/actor.h:100: NKikimr::NOlap::NDataReader::TActor::StateFunc(TAutoPtr<NActors::IEventHandle, TDelete>&) @ 0x15B3375E
Dec 19 15:01:51 ydb-sas-testing-0004 kikimr_31003[840203]: 4. /home/maxim-yurchuk/ydb/ydb/library/actors/core/executor_thread.cpp:248: NActors::TGenericExecutorThread::Execute(NActors::TMailbox*, bool) @ 0xEFC6146
Dec 19 15:01:51 ydb-sas-testing-0004 kikimr_31003[840203]: 5. /home/maxim-yurchuk/ydb/ydb/library/actors/core/executor_thread.cpp:425: NActors::TGenericExecutorThread::ProcessExecutorPool(NActors::IExecutorPool*)::$_0::operator()(NActors::TMailbox*, bool) const @ 0xEFC9910
Dec 19 15:01:51 ydb-sas-testing-0004 kikimr_31003[840203]: 6. /home/maxim-yurchuk/ydb/ydb/library/actors/core/executor_thread.cpp:479: NActors::TGenericExecutorThread::ProcessExecutorPool(NActors::IExecutorPool*) @ 0xEFC9462
Dec 19 15:01:51 ydb-sas-testing-0004 kikimr_31003[840203]: 7. /home/maxim-yurchuk/ydb/ydb/library/actors/core/executor_thread.cpp:510: NActors::TExecutorThread::ThreadProc() @ 0xEFCA065
Dec 19 15:01:51 ydb-sas-testing-0004 kikimr_31003[840203]: 8. /home/maxim-yurchuk/ydb/util/system/thread.cpp:244: (anonymous namespace)::TPosixThread::ThreadProxy(void*) @ 0xE3E55B7
Dec 19 15:01:51 ydb-sas-testing-0004 kikimr_31003[840203]: 9. ??:0: ?? @ 0x7F6C74299608
Dec 19 15:01:51 ydb-sas-testing-0004 kikimr_31003[840203]: 10. ??:0: ?? @ 0x7F6C741B9352