QuillBackend received segfault under multi-thread environment

[sangoblin]

Describe the bug
"QuillBackend" received signal SIGSEGV, Segmentation fault

To Reproduce

Expected Behaviour
A clear and concise description of what you expected to happen.

Environment Details

• Library Version: v7.4
• Linkage: dynamically
• Operating System: Ubuntu 22.04).
• Compiler: GCC-14.2

Additional context
[Switching to Thread 0x7ffff7008640 (LWP 35863)]
quill::v7::detail::BoundedSPSCQueueImpl::_free_aligned (ptr=0x7ffff4cf5080) at /opt/vcpkg/installed/x64-linux/include/quill/core/BoundedSPSCQueue.h:311
311 std::memcpy(&offset, static_caststd::byte*(ptr) - (2u * sizeof(size_t)), sizeof(offset));
(gdb) where
#0 quill::v7::detail::BoundedSPSCQueueImpl::_free_aligned (ptr=0x7ffff4cf5080) at /opt/vcpkg/installed/x64-linux/include/quill/core/BoundedSPSCQueue.h:311
#1 0x00005555555a1e08 in quill::v7::detail::BoundedSPSCQueueImpl::~BoundedSPSCQueueImpl (this=0x7fffec005500, __in_chrg=)
at /opt/vcpkg/installed/x64-linux/include/quill/core/BoundedSPSCQueue.h:112
#2 0x000055555558a372 in quill::v7::detail::UnboundedSPSCQueue::Node::~Node (this=0x7fffec005480, __in_chrg=) at /opt/vcpkg/installed/x64-linux/include/quill/core/UnboundedSPSCQueue.h:45
#3 0x00007ffff66ea9e0 in quill::v7::detail::UnboundedSPSCQueue::_read_next_queue (this=0x7fffec000980, next_node=0x7fffec005)
at /opt/vcpkg/installed/x64-linux/include/quill/core/UnboundedSPSCQueue.h:300
#4 0x00007ffff66ea84b in quill::v7::detail::UnboundedSPSCQueue::prepare_read (this=0x7fffec000980) at /opt/vcpkg/installed/x64-linux/include/quill/core/UnboundedSPSCQueue.h:174
#5 0x00007ffff66f0923 in quill::v7::detail::BackendWorker::_read_unbounded_frontend_queue (this=0x7ffff6806a00, frontend_queue=..., thread_context=0x7fffec000980)
at /opt/vcpkg/installed/x64-linux/include/quill/backend/BackendWorker.h:1076
#6 0x00007ffff6704bcb in quill::v7::detail::BackendWorker::_read_and_decode_frontend_queuequill::v7::detail::UnboundedSPSCQueue (this=0x7ffff6806a00, frontend_queue=...,
thread_context=0x7fffec000980, ts_now=1731336852949553496) at /opt/vcpkg/installed/x64-linux/include/quill/backend/BackendWorker.h:441
#7 0x00007ffff66ee352 in quill::v7::detail::BackendWorker::_populate_transit_events_from_frontend_queues (this=0x7ffff6806a00) at /opt/vcpkg/installed/x64-linux/include/quill/backend/BackendWorker.h:406
#8 0x00007ffff66edab3 in quill::v7::detail::BackendWorker::_poll (this=0x7ffff6806a00) at /opt/vcpkg/installed/x64-linux/include/quill/backend/BackendWorker.h:246
#9 0x00007ffff66ed368 in quill::v7::detail::BackendWorker::run(quill::v7::BackendOptions const&)::{lambda()#1}::operator()() const (__closure=0x55555568ab58)
at /opt/vcpkg/installed/x64-linux/include/quill/backend/BackendWorker.h:167
#10 0x00007ffff678bfb9 in std::__invoke_impl<void, quill::v7::detail::BackendWorker::run(quill::v7::BackendOptions const&)::{lambda()#1}>(std::__invoke_other, quill::v7::detail::BackendWorker::run(quill::v7::BackendOptions const&)::{lambda()#1}&&) (__f=...) at /opt/GCC/v14.2.0/include/c++/14/bits/invoke.h:61
#11 0x00007ffff678b549 in std::__invoke<quill::v7::detail::BackendWorker::run(quill::v7::BackendOptions const&)::{lambda()#1}>(quill::v7::detail::BackendWorker::run(quill::v7::BackendOptions const&)::{lambda()#1}&&) (__fn=...) at /opt/GCC/v14.2.0/include/c++/14/bits/invoke.h:96
#12 0x00007ffff678a0e6 in std::thread::_Invoker<std::tuple<quill::v7::detail::BackendWorker::run(quill::v7::BackendOptions const&)::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) (
this=0x55555568ab58) at /opt/GCC/v14.2.0/include/c++/14/bits/std_thread.h:301
#13 0x00007ffff6784552 in std::thread::_Invoker<std::tuple<quill::v7::detail::BackendWorker::run(quill::v7::BackendOptions const&)::{lambda()#1}> >::operator()() (this=0x55555568ab58)
at /opt/GCC/v14.2.0/include/c++/14/bits/std_thread.h:308
#14 0x00007ffff6783a52 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<quill::v7::detail::BackendWorker::run(quill::v7::BackendOptions const&)::{lambda()#1}> > >::_M_run() (
this=0x55555568ab50) at /opt/GCC/v14.2.0/include/c++/14/bits/std_thread.h:253
#15 0x00007ffff7cf90e4 in std::execute_native_thread_routine (__p=0x55555568ab50) at ../../../../../libstdc++-v3/src/c++11/thread.cc:104
#16 0x00007ffff7961ac3 in start_thread (arg=) at ./nptl/pthread_create.c:442
#17 0x00007ffff79f3850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

[odygrd]

That seems weird, can you share more info please like:

can you reproduce every time, how many frontend threads used, are the threads always alive or are they being spawned and destroyed.. basically anything that can help

How are you using QuillBackend ? Are you starting it via Backend::start() or are you using the ManualBackendWorker ? The ManualBackendWorker is not meant to be used under multiple threads

we have many test cases in regression tests and CI covering multiple frontends logging and it has never crashed in that particular place, never

If you can reproduce it locally, can you try linking statically please to see if that changes it ?

[odygrd]

Are you dynamically loading and unloading a shared library that is issuing the logs ?

[odygrd]

I added extra tests for the UnboundedQueue and cannot reproduce the issue. CI passes on all macOS, Windows, and Linux platforms. I also reviewed the _alloc_aligned and _free_aligned functions, and they appear correct. Are you sure there isn't something else in your application that could be corrupting the logger's memory?

[sangoblin]

Are you dynamically loading and unloading a shared library that is issuing the logs ?

HI, to make it clear, I compiled the header into multiple shared libraries and forgot to export the symbols from the main executable, which leads to multiple resource reclaiming. Now the issue has been fixed after export symbols from the main executable. Sorry for causing the trouble, and thank you so much for the prompt response!

[sangoblin]

Btw, feel free to close this issue. Have a nice one!

[odygrd]

Thank you for letting me know. Happy it is resolved :)