Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LHAPDF::_getQ2CachesMap() is not thread safe #35082

Open
makortel opened this issue Aug 31, 2021 · 13 comments
Open

LHAPDF::_getQ2CachesMap() is not thread safe #35082

makortel opened this issue Aug 31, 2021 · 13 comments

Comments

@makortel
Copy link
Contributor

#35073 (comment) experienced a rare crash

Thread 10 (Thread 0x2ba051c00700 (LWP 11137) "cmsRun"):
#2  0x00002ba009f033f0 in sig_pause_for_stacktrace () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/pluginFWCoreServicesPlugins.so
#3  <signal handler called>
#4  0x00002ba004138a35 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#5  0x00002ba003d0085c in __gthread_cond_wait (__mutex=<optimized out>, __cond=<optimized out>) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_11_1_0_pre6-slc7_amd64_gcc900/build/CMSSW_11_1_0_pre6-build/BUILD/slc7_amd64_gcc900/external/gcc/9.3.0/gcc-9.3.0/obj/x86_64-unknown-linux-gnu/libstdc++-v3/include/x86_64-unknown-linux-gnu/bits/gthr-default.h:865
#6  std::condition_variable::wait (this=<optimized out>, __lock=...) at ../../../../../libstdc++-v3/src/c++11/condition_variable.cc:53
#7  0x00002ba03690968b in OscarMTProducer::produce(edm::Event&, edm::EventSetup const&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/biglib/slc7_amd64_gcc900/pluginSimulation.so
#8  0x00002ba001dd586c in edm::stream::EDProducerAdaptorBase::doEvent(edm::EventTransitionInfo const&, edm::ActivityRegistry*, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#9  0x00002ba001db6a5d in edm::WorkerT<edm::stream::EDProducerAdaptorBase>::implDo(edm::EventTransitionInfo const&, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#10 0x00002ba001d11545 in decltype ({parm#1}()) edm::convertException::wrap<edm::Worker::runModule<edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}>(edm::Worker::runModule<edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#11 0x00002ba001d116fd in bool edm::Worker::runModule<edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::Context const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#12 0x00002ba001d11a06 in std::__exception_ptr::exception_ptr edm::Worker::runModuleAfterAsyncPrefetch<edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1> >(std::__exception_ptr::exception_ptr const*, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::Context const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#13 0x00002ba001d13d96 in edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1> >::execute() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#14 0x00002ba001a852a5 in tbb::detail::d1::function_task<edm::WaitingTaskList::announce()::{lambda()#1}>::execute(tbb::detail::d1::execution_data&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreConcurrency.so

Thread 9 (Thread 0x2ba050cfb700 (LWP 11136) "cmsRun"):
#2  0x00002ba009f033f0 in sig_pause_for_stacktrace () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/pluginFWCoreServicesPlugins.so
#3  <signal handler called>
#4  0x00002ba031eb6e80 in LHAPDF::KnotArray1F::iq2below(double) const@plt () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#5  0x00002ba031ee19dd in LHAPDF::Interpolator::interpolateXQ2(int, double, double) const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#6  0x00002ba031edcadf in LHAPDF::GridPDF::_xfxQ2(int, double, double) const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#7  0x00002ba031ecf807 in LHAPDF::PDF::xfxQ2(int, double, double) const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#8  0x00002ba04d3f2931 in Pythia8::LHAPDF6::xfUpdate(int, double, double) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8lhapdf6.so
#9  0x00002ba0318931b7 in Pythia8::PDF::xf(int, double, double) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#10 0x00002ba031a6d799 in Pythia8::SigmaProcess::sigmaPDF(bool, bool, bool, double, double) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#11 0x00002ba0318db1e7 in Pythia8::PhaseSpace::setupSampling123(bool, bool) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#12 0x00002ba03190f2d0 in Pythia8::ProcessContainer::init(bool, Pythia8::ResonanceDecays*, Pythia8::SLHAinterface*, Pythia8::GammaKinematics*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#13 0x00002ba031928c5b in Pythia8::ProcessLevel::init(bool, Pythia8::SLHAinterface*, std::vector<Pythia8::SigmaProcess*, std::allocator<Pythia8::SigmaProcess*> >&, std::vector<Pythia8::PhaseSpace*, std::allocator<Pythia8::PhaseSpace*> >&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#14 0x00002ba03194e025 in Pythia8::Pythia::init() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#15 0x00002ba02fdf20f0 in Pythia8Hadronizer::initializeForInternalPartons() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/pluginGeneratorInterfacePythia8Filters.so
#16 0x00002ba02fe2e034 in edm::ConcurrentGeneratorFilter<Pythia8Hadronizer, gen::ConcurrentExternalDecayDriver>::initLumi(edm::gen::GenStreamCache<Pythia8Hadronizer, gen::ConcurrentExternalDecayDriver>*, edm::LuminosityBlock const&, edm::EventSetup const&) const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/pluginGeneratorInterfacePythia8Filters.so
#17 0x00002ba001dba720 in edm::global::EDFilterBase::doStreamBeginLuminosityBlock(edm::StreamID, edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#18 0x00002ba001daf280 in edm::WorkerT<edm::global::EDFilterBase>::implDoStreamBegin(edm::StreamID, edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#19 0x00002ba001cacfc2 in decltype ({parm#1}()) edm::convertException::wrap<edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}>(edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#20 0x00002ba001cad20d in bool edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#21 0x00002ba001cad3f0 in edm::Worker::doWorkNoPrefetchingAsync<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::WaitingTaskHolder, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::ServiceToken const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}::operator()() const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#22 0x00002ba001cad56a in tbb::detail::d1::function_task<edm::Worker::doWorkNoPrefetchingAsync<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::WaitingTaskHolder, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::ServiceToken const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}>::execute(tbb::detail::d1::execution_data&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so

Thread 8 (Thread 0x2ba0502fa700 (LWP 11135) "cmsRun"):
#3  0x00002ba009f077bb in sig_dostack_then_abort () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/pluginFWCoreServicesPlugins.so
#4  <signal handler called>
#5  std::local_Rb_tree_rotate_left (__root=@0x2ba031f64710: 0x2ba080fcc440, __x=0x2ba04e84ba40) at ../../../../../libstdc++-v3/src/c++98/tree.cc:138
#6  std::_Rb_tree_insert_and_rebalance (__insert_left=<optimized out>, __x=0x2ba05ec01180, __p=<optimized out>, __header=...) at ../../../../../libstdc++-v3/src/c++98/tree.cc:278
#7  0x00002ba031ee316d in LHAPDF::_getQ2CachesMap() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#8  0x00002ba031ee322d in LHAPDF::LogBicubicInterpolator::_getCacheQ2(LHAPDF::KnotArray1F const&, double, unsigned long) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#9  0x00002ba031ee34a8 in LHAPDF::LogBicubicInterpolator::_interpolateXQ2(LHAPDF::KnotArray1F const&, double, unsigned long, double, unsigned long) const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#10 0x00002ba031ee19ff in LHAPDF::Interpolator::interpolateXQ2(int, double, double) const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#11 0x00002ba031edcadf in LHAPDF::GridPDF::_xfxQ2(int, double, double) const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#12 0x00002ba031ecf807 in LHAPDF::PDF::xfxQ2(int, double, double) const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#13 0x00002ba04d3f28e7 in Pythia8::LHAPDF6::xfUpdate(int, double, double) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8lhapdf6.so
#14 0x00002ba0318931b7 in Pythia8::PDF::xf(int, double, double) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#15 0x00002ba031a6d799 in Pythia8::SigmaProcess::sigmaPDF(bool, bool, bool, double, double) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#16 0x00002ba0318db1e7 in Pythia8::PhaseSpace::setupSampling123(bool, bool) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#17 0x00002ba03190f2d0 in Pythia8::ProcessContainer::init(bool, Pythia8::ResonanceDecays*, Pythia8::SLHAinterface*, Pythia8::GammaKinematics*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#18 0x00002ba031928c5b in Pythia8::ProcessLevel::init(bool, Pythia8::SLHAinterface*, std::vector<Pythia8::SigmaProcess*, std::allocator<Pythia8::SigmaProcess*> >&, std::vector<Pythia8::PhaseSpace*, std::allocator<Pythia8::PhaseSpace*> >&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#19 0x00002ba03194e025 in Pythia8::Pythia::init() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#20 0x00002ba02fdf20f0 in Pythia8Hadronizer::initializeForInternalPartons() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/pluginGeneratorInterfacePythia8Filters.so
#21 0x00002ba02fe2e034 in edm::ConcurrentGeneratorFilter<Pythia8Hadronizer, gen::ConcurrentExternalDecayDriver>::initLumi(edm::gen::GenStreamCache<Pythia8Hadronizer, gen::ConcurrentExternalDecayDriver>*, edm::LuminosityBlock const&, edm::EventSetup const&) const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/pluginGeneratorInterfacePythia8Filters.so
#22 0x00002ba001dba720 in edm::global::EDFilterBase::doStreamBeginLuminosityBlock(edm::StreamID, edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#23 0x00002ba001daf280 in edm::WorkerT<edm::global::EDFilterBase>::implDoStreamBegin(edm::StreamID, edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#24 0x00002ba001cacfc2 in decltype ({parm#1}()) edm::convertException::wrap<edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}>(edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#25 0x00002ba001cad20d in bool edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#26 0x00002ba001cad3f0 in edm::Worker::doWorkNoPrefetchingAsync<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::WaitingTaskHolder, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::ServiceToken const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}::operator()() const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#27 0x00002ba001cad56a in tbb::detail::d1::function_task<edm::Worker::doWorkNoPrefetchingAsync<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::WaitingTaskHolder, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::ServiceToken const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}>::execute(tbb::detail::d1::execution_data&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so

Thread 4 (Thread 0x2ba04a280700 (LWP 11130) "cmsRun"):
#0  CLHEP::MixMaxRng::MULWU (this=this@entry=0x2ba0359f8250, k=k@entry=1839237037704408705) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_12_0_0_pre5-slc7_amd64_gcc900/build/CMSSW_12_0_0_pre5-build/BUILD/slc7_amd64_gcc900/external/clhep/2.4.4.0-llifpc2/clhep-2.4.4.0/Random/src/MixMaxRng.cc:495
#1  0x00002ba00a5f21fb in CLHEP::MixMaxRng::iterate (this=0x2ba0359f8250) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_12_0_0_pre5-slc7_amd64_gcc900/build/CMSSW_12_0_0_pre5-build/BUILD/slc7_amd64_gcc900/external/clhep/2.4.4.0-llifpc2/clhep-2.4.4.0/Random/src/MixMaxRng.cc:312
#2  0x00002ba037133380 in G4UrbanMscModel::SampleScattering(CLHEP::Hep3Vector const&, double) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/biglib/slc7_amd64_gcc900/pluginSimulation.so
#3  0x00002ba03715e5fe in G4VMultipleScattering::AlongStepDoIt(G4Track const&, G4Step const&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/biglib/slc7_amd64_gcc900/pluginSimulation.so
#4  0x00002ba0374cdeca in G4SteppingManager::InvokeAlongStepDoItProcs() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/biglib/slc7_amd64_gcc900/pluginSimulation.so
#5  0x00002ba0374cc3a7 in G4SteppingManager::Stepping() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/biglib/slc7_amd64_gcc900/pluginSimulation.so
#6  0x00002ba036fa8d98 in G4TrackingManager::ProcessOneTrack(G4Track*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/biglib/slc7_amd64_gcc900/pluginSimulation.so
#7  0x00002ba036c77d9b in G4EventManager::DoProcessing(G4Event*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/biglib/slc7_amd64_gcc900/pluginSimulation.so
#8  0x00002ba0368ecfbc in RunManagerMTWorker::produce(edm::Event const&, edm::EventSetup const&, RunManagerMT&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/biglib/slc7_amd64_gcc900/pluginSimulation.so
#9  0x00002ba03690636f in omt::ThreadHandoff::Functor<OscarMTProducer::produce(edm::Event&, edm::EventSetup const&)::{lambda()#1}>::execute() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/biglib/slc7_amd64_gcc900/pluginSimulation.so
#10 0x00002ba0368fd6ea in omt::ThreadHandoff::threadLoop(void*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/biglib/slc7_amd64_gcc900/pluginSimulation.so

Thread 1 (Thread 0x2ba0064c3980 (LWP 11056) "cmsRun"):
#2  0x00002ba009f033f0 in sig_pause_for_stacktrace () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/pluginFWCoreServicesPlugins.so
#3  <signal handler called>
#4  0x00002ba0043880f6 in ____strtod_l_internal () from /lib64/libc.so.6
#5  0x00002ba031edd050 in LHAPDF::GridPDF::_loadData(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#6  0x00002ba031eef9dc in LHAPDF::mkPDF(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#7  0x00002ba04d3f445d in Pythia8::LHAPDF6::init(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8lhapdf6.so
#8  0x00002ba04d3f480f in newPDF () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8lhapdf6.so
#9  0x00002ba0318aab68 in Pythia8::LHAPDF::LHAPDF(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Pythia8::Info*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#10 0x00002ba03197081c in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<Pythia8::LHAPDF, std::allocator<Pythia8::LHAPDF>, int&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, Pythia8::Info*>(Pythia8::LHAPDF*&, std::_Sp_alloc_shared_tag<std::allocator<Pythia8::LHAPDF> >, int&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, Pythia8::Info*&&) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#11 0x00002ba03193e114 in Pythia8::Pythia::getPDFPtr(int, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#12 0x00002ba0319423e1 in Pythia8::Pythia::initPDFs() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#13 0x00002ba03194dce2 in Pythia8::Pythia::init() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#14 0x00002ba02fdf20f0 in Pythia8Hadronizer::initializeForInternalPartons() () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/pluginGeneratorInterfacePythia8Filters.so
#15 0x00002ba02fe2e034 in edm::ConcurrentGeneratorFilter<Pythia8Hadronizer, gen::ConcurrentExternalDecayDriver>::initLumi(edm::gen::GenStreamCache<Pythia8Hadronizer, gen::ConcurrentExternalDecayDriver>*, edm::LuminosityBlock const&, edm::EventSetup const&) const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/pluginGeneratorInterfacePythia8Filters.so
#16 0x00002ba001dba720 in edm::global::EDFilterBase::doStreamBeginLuminosityBlock(edm::StreamID, edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#17 0x00002ba001daf280 in edm::WorkerT<edm::global::EDFilterBase>::implDoStreamBegin(edm::StreamID, edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#18 0x00002ba001cacfc2 in decltype ({parm#1}()) edm::convertException::wrap<edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}>(edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#19 0x00002ba001cad20d in bool edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*) () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so
#20 0x00002ba001cad3f0 in edm::Worker::doWorkNoPrefetchingAsync<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1> >(edm::WaitingTaskHolder, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::ServiceToken const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}::operator()() const () from /cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_X_2021-08-29-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so

that points to a race condition in LHAPDF that is discussed in https://gitlab.com/hepcedar/lhapdf/-/issues/2.

@makortel
Copy link
Contributor Author

assign core, generators

@cmsbuild
Copy link
Contributor

New categories assigned: core,generators

@Dr15Jones,@smuzaffar,@Saptaparna,@mkirsano,@SiewYan,@alberto-sanchez,@makortel,@agrohsje,@GurpreetSinghChahal you have been requested to review this Pull request/Issue and eventually sign? Thanks

@cmsbuild
Copy link
Contributor

A new Issue was created by @makortel Matti Kortelainen.

@Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

@makortel
Copy link
Contributor Author

Do we want to try patch https://gitlab.com/hepcedar/lhapdf/-/merge_requests/3 ? I'm not really sure how we could test it conclusively, given that it took ~1.5 months to hit to the issue after the update to LHAPDF 6.3.0.

@agrohsje
Copy link

agrohsje commented Sep 7, 2021

Hi @mkirsano : I would agree with @makortel and apply the proposed patch.

@mkirsano
Copy link
Contributor

mkirsano commented Sep 8, 2021 via email

@makortel
Copy link
Contributor Author

makortel commented Sep 10, 2021

Apparently the authors released a beta of 6.4.0 that eliminates the cache (I did not look into details), see https://gitlab.com/hepcedar/lhapdf/-/issues/2#note_674891431. Should we try that?

@makortel
Copy link
Contributor Author

makortel commented Sep 13, 2021

If #35251 is connected, we just got a way to test rather conclusively if the beta 6.4.0 would fix the race conditions in our case.

@Dr15Jones
Copy link
Contributor

Dr15Jones commented Sep 23, 2021

It happened here as well

https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc900/CMSSW_12_1_ROOT624_X_2021-09-22-2300/pyRelValMatrixLogs/run/39034.0_TTbar_14TeV+2026D87+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger/step1_TTbar_14TeV+2026D87+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger.log#/599-599

#3  0x00002b69b6b287bb in sig_dostack_then_abort () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/lib/slc7_amd64_gcc900/pluginFWCoreServicesPlugins.so
#4  <signal handler called>
#5  std::local_Rb_tree_rotate_left (__root=@0x2b69e142c750: 0x2b6a27010360, __x=0x2b69fd8e4100) at ../../../../../libstdc++-v3/src/c++98/tree.cc:138
#6  std::_Rb_tree_insert_and_rebalance (__insert_left=<optimized out>, __x=0x2b6a2697c660, __p=<optimized out>, __header=...) at ../../../../../libstdc++-v3/src/c++98/tree.cc:278
#7  0x00002b69e13aad9d in LHAPDF::_getXCachesMap() () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#8  0x00002b69e13aae5d in LHAPDF::LogBicubicInterpolator::_getCacheX(LHAPDF::KnotArray1F const&, double, unsigned long) () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#9  0x00002b69e13ab48f in LHAPDF::LogBicubicInterpolator::_interpolateXQ2(LHAPDF::KnotArray1F const&, double, unsigned long, double, unsigned long) const () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#10 0x00002b69e13a99ff in LHAPDF::Interpolator::interpolateXQ2(int, double, double) const () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#11 0x00002b69e13a4adf in LHAPDF::GridPDF::_xfxQ2(int, double, double) const () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#12 0x00002b69e1397807 in LHAPDF::PDF::xfxQ2(int, double, double) const () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libLHAPDF.so
#13 0x00002b69ff3ab8e7 in Pythia8::LHAPDF6::xfUpdate(int, double, double) () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libpythia8lhapdf6.so
#14 0x00002b69e0d5b1b7 in Pythia8::PDF::xf(int, double, double) () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#15 0x00002b69e0f35799 in Pythia8::SigmaProcess::sigmaPDF(bool, bool, bool, double, double) () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#16 0x00002b69e0da31e7 in Pythia8::PhaseSpace::setupSampling123(bool, bool) () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#17 0x00002b69e0dd72d0 in Pythia8::ProcessContainer::init(bool, Pythia8::ResonanceDecays*, Pythia8::SLHAinterface*, Pythia8::GammaKinematics*) () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#18 0x00002b69e0df0c5b in Pythia8::ProcessLevel::init(bool, Pythia8::SLHAinterface*, std::vector<Pythia8::SigmaProcess*, std::allocator<Pythia8::SigmaProcess*> >&, std::vector<Pythia8::PhaseSpace*, std::allocator<Pythia8::PhaseSpace*> >&) () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#19 0x00002b69e0e16025 in Pythia8::Pythia::init() () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/external/slc7_amd64_gcc900/lib/libpythia8.so
#20 0x00002b69df2ba0f0 in Pythia8Hadronizer::initializeForInternalPartons() () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/lib/slc7_amd64_gcc900/pluginGeneratorInterfacePythia8Filters.so
#21 0x00002b69df2f6034 in edm::ConcurrentGeneratorFilter<Pythia8Hadronizer, gen::ConcurrentExternalDecayDriver>::initLumi(edm::gen::GenStreamCache<Pythia8Hadronizer, gen::ConcurrentExternalDecayDriver>*, edm::LuminosityBlock const&, edm::EventSetup const&) const () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/lib/slc7_amd64_gcc900/pluginGeneratorInterfacePythia8Filters.so
#22 0x00002b69ae8757c0 in edm::global::EDFilterBase::doStreamBeginLuminosityBlock(edm::StreamID, edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/nweek-02699/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_1_ROOT624_X_2021-09-22-2300/lib/slc7_amd64_gcc900/libFWCoreFramework.so

...

Current Modules:

Module: Pythia8ConcurrentGeneratorFilter:generator (crashed)
Module: Pythia8ConcurrentGeneratorFilter:generator
Module: OscarMTProducer:g4SimHits
Module: Pythia8ConcurrentGeneratorFilter:generator

A fatal system signal has occurred: segmentation violation

@agrohsje
Copy link

agrohsje commented Oct 1, 2021

Dear @mkirsano, @SiewYan , lhapdf 6.4.0 including NNPDF 4.0 is now out. I would suggest to update cmsdist. Are you available?
This allows us to study the potential PDFs for Run 3 as well as a possible answer to the problem of thread safety.

@mkirsano
Copy link
Contributor

mkirsano commented Oct 1, 2021

Yes, OK

@makortel
Copy link
Contributor Author

makortel commented Sep 6, 2024

Given that we have updated LHAPDF to 6.4.0 that (presumably) eliminated the cache, and that we don't have any reports from past few years (although that doesn't mean they couldn't have occurred), maybe we should close this issue?

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 6, 2024

cms-bot internal usage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants