Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix to "Update SiStrip and TkAlignment ALCARECO event content for Run3" #38590

Merged

Conversation

mmusich
Copy link
Contributor

@mmusich mmusich commented Jul 4, 2022

PR description:

Unfortunately PR #38415 aimed to update the event content of the input ALCARECO event contents to use S/W FED 1023 information instead of SCAL for all the relevant SiStrip and TkAlignment alcas contained two bugs:

  • in python, when a variable A is assigned to another variable B, if an operation is done on B, the attributes of A are changed as well. This has resulted in applying the removal of the content of the _run3_common_removedCommands list even for eras <= Run2.
  • the ConfigBuilder uses the *_noDrop version of the ALCARECO event content to build the list of products to keep, so the instructions gated via the run3_common modifier were not effective.

self.executeAndRemember('process.ALCARECOEventContent.outputCommands.extend(process.OutALCARECO'+shortName+'_noDrop.outputCommands)')

As a result of the two bugs, the list of event products to be kept is now wrong for all eras.
This PR resolves the issue.

PR validation:

Test running:

runTheMatrix.py -l 1000.0, 136.874, 138.4 --ibeos

and checked the event content of the SiStripCalMinBias ALCARECO file, in order to test the Run1, Run2 and Run3 eras.
In addition I've also produced ALCARECO data, starting from RAW and the ConfigDP machinery using the command:

python3 Configuration/DataProcessing/test/RunExpressProcessing.py --nThreads 8 --scenario ppEra_Run3 --global-tag 124X_dataRun3_Express_v3 --lfn root://eoscms.cern.ch//eos/cms/tier0/store/data//Run2022A/MinimumBias/RAW/v1/000/352/929/00000/18df5589-67c8-45e5-ac84-7fd2472de50a.root --fevt --dqmio --alcarecos SiStripCalMinBias

and obtained the right event content:

File output_inALCARECO.root Events 13
Branch Name | Average Uncompressed Size (Bytes/Event) | Average Compressed Size (Bytes/Event) 
SiStripClusteredmNewDetSetVector_siStripClusters__RECO. 51212.1 13165.5
SiPixelClusteredmNewDetSetVector_siPixelClusters__RECO. 16359.7 6596.46
recoTrackExtras_ALCARECOSiStripCalMinBias__RECO. 1641.15 1066.77
SiStripClusteredmNewDetSetVector_ALCARECOSiStripCalMinBias__RECO. 3236.77 871.923
TrackingRecHitsOwned_ALCARECOSiStripCalMinBias__RECO. 4918.23 813.692
recoTracks_ALCARECOSiStripCalMinBias__RECO. 1793.77 800.615
SiPixelClusteredmNewDetSetVector_ALCARECOSiStripCalMinBias__RECO. 1216.54 559.385
DetIdedmEDCollection_siStripDigis__RECO. 2081.62 280.231
edmTriggerResults_TriggerResults__HLT. 1847.85 245.231
OnlineLuminosityRecord_onlineMetaDataDigis__RECO. 230.538 210.846
L1AcceptBunchCrossings_scalersRawToDigi__RECO. 209.769 198.462
L1GlobalTriggerReadoutRecord_gtDigis__RECO. 261.923 196.385
edmTriggerResults_TriggerResults__RECO. 263.846 191.846
DCSRecord_onlineMetaDataDigis__RECO. 278.538 184.077
EventProductProvenance 2428.31 162
EventAuxiliary 160.385 43.4615
EventSelections 120.769 17.6154
BranchListIndexes 68.5385 12.2308

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport, but will need to be backported to 12.4.X.

@mmusich
Copy link
Contributor Author

mmusich commented Jul 4, 2022

type trk,bugfix

@mmusich
Copy link
Contributor Author

mmusich commented Jul 4, 2022

urgent

@cmsbuild cmsbuild added the urgent label Jul 4, 2022
@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 4, 2022

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38590/30847

  • This PR adds an extra 16KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 4, 2022

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

  • Alignment/CommonAlignmentProducer (alca)
  • Calibration/TkAlCaRecoProducers (alca)

@malbouis, @yuanchao, @cmsbuild, @francescobrivio, @ChrisMisan, @tvami can you please review it and eventually sign? Thanks.
@pakhotin, @adewit, @tocheng, @tlampen, @mmusich, @threus this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@mmusich
Copy link
Contributor Author

mmusich commented Jul 4, 2022

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 4, 2022

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7733c9/25972/summary.html
COMMIT: cfc5852
CMSSW: CMSSW_12_5_X_2022-07-04-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38590/25972/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 10 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3654771
  • DQMHistoTests: Total failures: 19
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 3654729
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.004 KiB( 49 files compared)
  • DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@mmusich
Copy link
Contributor Author

mmusich commented Jul 5, 2022

please test

  • let's try

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 5, 2022

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7733c9/25976/summary.html
COMMIT: cfc5852
CMSSW: CMSSW_12_5_X_2022-07-04-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38590/25976/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3654771
  • DQMHistoTests: Total failures: 2
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3654747
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@tvami
Copy link
Contributor

tvami commented Jul 5, 2022

+alca

  • test pass, the event content is correct now

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 5, 2022

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

perrotta commented Jul 5, 2022

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants