Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Erase from correct vector in PATTauHybridProducer #44067

Merged
merged 1 commit into from
Feb 29, 2024

Conversation

makortel
Copy link
Contributor

PR description:

Erasing an element from pfChs using an iterator from pfGammas is undefined behavior. Based on the surrounding code, I guess the intention was to erase the element from pfGammas.

I came across this while investigating an UBSAN warning

/data/cmsbld/jenkins/workspace/build-any-ib/w/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/stl_construct.h:151:22: runtime error: member call on null pointer of type 'struct Ptr'
    #0 0x14e0b6a2131e in void std::_Destroy<edm::Ptr<reco::Candidate> >(edm::Ptr<reco::Candidate>*) /data/cmsbld/jenkins/workspace/build-any-ib/w/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/stl_construct.h:151
    #1 0x14e0b6a2131e in void std::_Destroy_aux<false>::__destroy<edm::Ptr<reco::Candidate>*>(edm::Ptr<reco::Candidate>*, edm::Ptr<reco::Candidate>*) /data/cmsbld/jenkins/workspace/build-any-ib/w/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/stl_construct.h:163
    #2 0x14e0b6a2131e in void std::_Destroy<edm::Ptr<reco::Candidate>*>(edm::Ptr<reco::Candidate>*, edm::Ptr<reco::Candidate>*) /data/cmsbld/jenkins/workspace/build-any-ib/w/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/stl_construct.h:196
    #3 0x14e0b6a2131e in void std::_Destroy<edm::Ptr<reco::Candidate>*, edm::Ptr<reco::Candidate> >(edm::Ptr<reco::Candidate>*, edm::Ptr<reco::Candidate>*, std::allocator<edm::Ptr<reco::Candidate> >&) /data/cmsbld/jenkins/workspace/build-any-ib/w/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/alloc_traits.h:850
    #4 0x14e0b6a2131e in std::vector<edm::Ptr<reco::Candidate>, std::allocator<edm::Ptr<reco::Candidate> > >::~vector() /data/cmsbld/jenkins/workspace/build-any-ib/w/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/stl_vector.h:730
    #5 0x14e0b6a2131e in PATTauHybridProducer::fillTauFromJet(reco::PFTau&, edm::RefToBase<reco::Jet> const&) src/PhysicsTools/PatAlgos/plugins/PATTauHybridProducer.cc:416
    #6 0x14e0b6a2db7e in PATTauHybridProducer::produce(edm::Event&, edm::EventSetup const&) src/PhysicsTools/PatAlgos/plugins/PATTauHybridProducer.cc:269
    #7 0x14e2588c290e in edm::stream::EDProducerAdaptorBase::doEvent(edm::EventTransitionInfo const&, edm::ActivityRegistry*, edm::ModuleCallingContext const*) src/FWCore/Framework/src/stream/EDProducerAdaptorBase.cc:82

(e.g. in https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc12/CMSSW_14_1_UBSAN_X_2024-02-21-2300/pyRelValMatrixLogs/run/11602.0_SingleElectronPt35+2021/step3_SingleElectronPt35+2021.log#/)

I'm not fully certain if this PR would fix that warning, but this line should be fixed anyway.

PR validation:

None

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

The fix impacts physics, but I don't know by how much, nor if it should be backported. I can backport it if needed.

Erasing an element from pfChs using an iterator from pfGammas is undefined behavior. Based on the surrounding code, I guess the intention was to erase the element from pfGammas.
@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 23, 2024

cms-bot internal usage

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44067/39019

  • This PR adds an extra 24KB to repository

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @makortel for master.

It involves the following packages:

  • PhysicsTools/PatAlgos (xpog, reconstruction)

@jfernan2, @mandrenguyen, @hqucms, @vlimant, @cmsbuild can you please review it and eventually sign? Thanks.
@AnnikaStein, @schoef, @jdolen, @seemasharmafnal, @emilbols, @Ming-Yan, @AlexDeMoor, @gkasieczka, @demuller, @gpetruc, @mbluj, @gouskos, @nhanvtran, @Senphy, @mmarionncern, @JyothsnaKomaragiri, @andrzejnovak, @jdamgov, @mariadalfonso, @azotz, @rappoccio, @hatakeyamak, @ahinzmann this is something you requested to watch as well.
@rappoccio, @sextonkennedy, @antoniovilela you are the release manager for this.

cms-bot commands are listed here

@makortel
Copy link
Contributor Author

@cmsbuild, please test

@makortel
Copy link
Contributor Author

@cms-sw/tau-pog-l2 @mbluj Could you please double-check this was the intention (in #41333)?

@mbluj
Copy link
Contributor

mbluj commented Feb 23, 2024

@cms-sw/tau-pog-l2 @mbluj Could you please double-check this was the intention (in #41333)?

Thanks @makortel for this fix, it was indeed the intention.
I think it will be good to backport this to 14_0 for this year operations / nanoAOD production. But, as long as the issue does not cause crashes it is not critical as I suppose that information on tau decay products for taus tagged by PNet, but not reconstructed by HPS, is not used.

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7797a2/37642/summary.html
COMMIT: fbe0e16
CMSSW: CMSSW_14_1_X_2024-02-23-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44067/37642/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7797a2/37642/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7797a2/37642/git-merge-result

Unit Tests

I found 2 errors in the following unit tests:

---> test TestL1ScoutingFormat had ERRORS
---> test TestSDSRawDataCollectionFormat had ERRORS

Comparison Summary

Summary:

@makortel
Copy link
Contributor Author

Looks like better to wait for the next IB before restarting tests

@makortel
Copy link
Contributor Author

Comparisons show only the usual

The differences in 25034.999 seem to be caused by #44051 that was included in the tests (because of being merged already)

@vlimant
Copy link
Contributor

vlimant commented Feb 26, 2024

enable nano

@makortel
Copy link
Contributor Author

@cmsbuild, please test

@makortel
Copy link
Contributor Author

Another thing I noticed in this code is that the loops like

  for (CandPtrs::iterator it = pfGammas.begin(); it != pfGammas.end();) {
    if ((*it)->pt() < 0.5) {
      it = pfGammas.erase(it);
    } else {
      ++it;
    }
  }

have $O(n^2)$ complexity. Linear complexity could be achieved with

  pfGammas.erase(std::remove_if(pfGammas.begin(), pfGammas.end(), [](auto const& cand) {
    return cand->pt() < 0.5;
  }, pfGammas.end());

(with C++20 this would become simpler, but we're not there yet...)

@mbluj
Copy link
Contributor

mbluj commented Feb 26, 2024

@makortel will you apply this "linearization" of complexity or an action is needed from my side?

@makortel
Copy link
Contributor Author

@mbluj I wasn't planning to (even if I was a bit tempted), so it would be great if you (or someone else) could do that.

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7797a2/37694/summary.html
COMMIT: fbe0e16
CMSSW: CMSSW_14_1_X_2024-02-26-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/44067/37694/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

NANO Comparison Summary

Summary:

  • You potentially removed 3 lines from the logs
  • Reco comparison results: 36 differences found in the comparisons
  • DQMHistoTests: Total files compared: 15
  • DQMHistoTests: Total histograms compared: 16430
  • DQMHistoTests: Total failures: 29
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 16401
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
  • Checked 40 log files, 20 edm output root files, 15 DQM output files

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.0 2.551 2.551 -0.000 ( -0.0% ) 5.29 5.27 +0.4% 2.151 2.226
2500.001 2.700 2.700 0.000 ( +0.0% ) 4.70 4.72 -0.3% 2.584 2.644
2500.002 2.641 2.641 0.000 ( +0.0% ) 4.92 4.86 +1.2% 2.581 2.634
2500.01 1.322 1.322 -0.000 ( -0.0% ) 9.61 9.72 -1.1% 2.251 2.245
2500.011 1.745 1.745 0.000 ( +0.0% ) 5.17 5.28 -2.0% 2.398 2.420
2500.012 1.586 1.586 0.000 ( +0.0% ) 7.39 7.43 -0.5% 2.374 2.421
2500.1 2.194 2.194 0.000 ( +0.0% ) 5.35 5.32 +0.4% 1.989 2.060
2500.2 2.310 2.310 0.000 ( +0.0% ) 6.09 6.04 +0.7% 1.888 1.976
2500.21 1.185 1.185 0.000 ( +0.0% ) 4.37 4.36 +0.2% 2.207 2.255
2500.211 1.549 1.549 0.000 ( +0.0% ) 3.79 3.74 +1.2% 2.229 2.331
2500.3 2.060 2.060 0.000 ( +0.0% ) 12.68 12.66 +0.2% 1.955 1.958
2500.31 1.258 1.258 0.000 ( +0.0% ) 20.02 20.10 -0.4% 2.349 2.357
2500.311 1.644 1.644 0.000 ( +0.0% ) 13.39 13.39 +0.1% 2.324 2.434
2500.312 7.159 7.159 0.000 ( +0.0% ) 1.41 1.41 +0.2% 1.681 1.685
2500.313 1.564 1.564 0.000 ( +0.0% ) 6.39 6.14 +4.0% 1.039 1.035
2500.314 1.157 1.157 0.000 ( +0.0% ) 13.88 14.16 -2.0% 2.169 2.175
2500.4 2.060 2.060 0.000 ( +0.0% ) 12.42 12.54 -1.0% 1.903 1.926
2500.401 1.817 1.817 0.000 ( +0.0% ) 10.53 10.15 +3.7% 1.807 1.779
2500.5 19.575 19.575 0.000 ( +0.0% ) 1.23 1.02 +20.4% 1.338 1.346

@vlimant
Copy link
Contributor

vlimant commented Feb 27, 2024

+1

@jfernan2
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @rappoccio, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

@antoniovilela
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit ea5de5d into cms-sw:master Feb 29, 2024
13 checks passed
@makortel makortel deleted the patch-6 branch February 29, 2024 16:53
@makortel
Copy link
Contributor Author

The 14_0_X backport is in #44272

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants