AlCaLumiPixelsCountsExpress and AlCaLumiPixelsCountsPrompt PCL and PromptReco

[benitezj]

PR description:

Code updates in Calibration/LumiAlCaRecoProducers for processing the AlCaLumiPixelsCountsExpress and AlCaLumiPixelsCountsPrompt streams for PCC luminosity in Run 3.

Related to this JIRA, where the HLT streams were defined:
https://its.cern.ch/jira/browse/CMSHLT-2114

PR validation:

The code was tested with

• CMSSW_12_1_0_pre4

• and the October 2021 Beam Test data:
  /eos/cms/store/data/Commissioning2021/AlCaLumiPixelsCounts*

• Using the following cmsDriver steps:
  cmsDriver.py step2 --conditions auto:run3_data -s ALCAPRODUCER:AlCaPCCRandom --process RECO --data --eventcontent ALCARECO --scenario pp --datatier ALCARECO --era Run3 --filein file:/eos/cms/store/data/Commissioning2021/AlCaLumiPixelsCountsExpress/RAW/v1/000/346/512/00000/fc13125c-7f0a-4140-9e26-18eb085e7c4c.root --no_exec -n 1000

cmsDriver.py step3 -s ALCAOUTPUT:AlCaPCCRandom,ALCA:PromptCalibProdLumiPCC --conditions auto:run3_data --era Run3 --datatier ALCARECO --eventcontent ALCARECO --triggerResultsProcess RECO --filein file:/afs/cern.ch/user/b/benitezj/output/BRIL/CMSSW_12_1_0_pre4/src/step2_ALCAPRODUCER.root -n 100 --no_exec

cmsDriver.py step4 -s ALCAHARVEST:LumiPCC --conditions auto:run3_data --era Run3 --scenario pp --datatier DQM
--eventcontent DQM --data --filein file:/afs/cern.ch/user/b/benitezj/output/BRIL/CMSSW_12_1_0_pre4/src/PromptCalibProdLumiPCC.root -n 100 --no_exec

cmsDriver.py step5 --process RECO --conditions auto:run3_data --scenario pp -s ALCAPRODUCER:AlCaPCCZeroBias+RawPCCProducer --datatier ALCARECO --filein file:/eos/cms/store/data/Commissioning2021/AlCaLumiPixelsCountsPrompt/RAW/v1/000/346/512/00000/237b5f81-adab-4529-a571-41b3a26ab24c.root --data --eventcontent ALCARECO --era Run3 --no_exec -n 1000

• obtained from:
  runTheMatrix.py --showMatrix --ext -l 1020.0

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36952/28304

• This PR adds an extra 32KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36952/28304/code-format.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36952/28304/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36952/28331

• This PR adds an extra 32KB to repository

[cmsbuild]

A new Pull Request was created by @benitezj (Jose) for master.

It involves the following packages:

• Calibration/LumiAlCaRecoProducers (alca)

@cmsbuild, @malbouis, @tvami, @yuanchao, @francescobrivio can you please review it and eventually sign? Thanks.
@mmusich, @tocheng this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[tvami]

Please also either modify the meaning of 1020.0 so it runs on the 2021 data with this new producer, or create a new wf, let's say 1020.1, which has these things. Thanks!

[tvami]

Hi @benitezj please address the comments from 4 days ago.

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36952/28401

ERROR: Build errors found during clang-tidy run.

Calibration/LumiAlCaRecoProducers/plugins/RawPCCProducer.cc:76:13: error: no viable overloaded '=' [clang-diagnostic-error]
  pccToken_ = consumes<reco::PixelClusterCounts, edm::InLumi>(edm::InputTag(pccSource, prodInst));
            ^
/cvmfs/cms-ib.cern.ch/nweek-02720/slc7_amd64_gcc10/cms/cmssw-patch/CMSSW_12_3_X_2022-02-18-1100/src/FWCore/Utilities/interface/EDGetToken.h:75:28: note: candidate function not viable: 'this' argument has type 'const edm::EDGetTokenT<reco::PixelClusterCounts>', but method is not marked const
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:129: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9a973b/22726/summary.html
COMMIT: ab32d50
CMSSW: CMSSW_12_3_X_2022-02-28-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36952/22726/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-9a973b/1020.0_AlCaLumiPixels2021+AlCaLumiPixels2021+TIER0EXPLP+ALCAEXPLP+ALCAHARVLP+TIER0PROMPTLP

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 4001207
• DQMHistoTests: Total failures: 2
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 4001183
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 204 log files, 45 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[tvami]

+alca

• same results as earlier but now it's squashed

[tvami]

urgent

• needs to be in pre6

[tvami]

@cms-sw/pdmv-l2 @cms-sw/upgrade-l2 could you please have a look at this PR?

[jordan-martins]

+pdmv

[srimanob]

+Upgrade

The code related to the upgrade is on Configuration/PyReleaseValidation. It is an update of the workflow. The PR test shows that the new workflow run fine.

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1

• Tested running in a dedicated workflow

[mmusich]

I think this broke the IB: https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc10/CMSSW_12_3_X_2022-03-02-2300/pyRelValMatrixLogs/run/1020.0_AlCaLumiPixels2021+AlCaLumiPixels2021+TIER0EXPLP+ALCAEXPLP+ALCAHARVLP+TIER0PROMPTLP/step2_AlCaLumiPixels2021+AlCaLumiPixels2021+TIER0EXPLP+ALCAEXPLP+ALCAHARVLP+TIER0PROMPTLP.log#/

[tvami]

Hi @mmusich thanks for pointing that out. I thought the ALCARECOs are forced to be done on a single thread, (and that's actually the wf that we tested in this PR), is that not the case? Can you confirm that I understood correctly that the issue is due to the multi-threaded use (which quite obviously then has issues with reading two streams)?

[mmusich]

I thought the ALCARECOs are forced to be done on a single thread, (and that's actually the wf that we tested in this PR), is that not the case?

What is done at Tier-0 doesn't necessarily reflect what is run in IBs and PR tests.
In this PR was run single-threaded, while in the IB likely it runs multi-threaded.

[makortel]

IB tests run multithreaded by default, so if something absolutely has to be run with a single thread only, it needs to be specified explicitly.