Fixed the PPS alignment relval - 1042

[MatiXOfficial]

PR description:

Following #35631 (and #35764), this PR fixes the PPS alignment relval by adding EDMtoMEConverter to the 4th, harvesting step. Now, the harvester can read the plots booked by the worker in the 3rd step.

The results of the matrix test still contain corrections set to 0 though. The reason for that is insufficient (100) number of events. So, is there any way to increase the number of events to be more than 100?

PR validation:

The 1042 relval behaves as expected.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35874/26271

• This PR adds an extra 28KB to repository

[cmsbuild]

A new Pull Request was created by @MatiXOfficial (Mateusz Kocot) for master.

It involves the following packages:

• CalibPPS/AlignmentGlobal (alca)
• Configuration/StandardSequences (operations)

@perrotta, @malbouis, @yuanchao, @tvami, @cmsbuild, @qliphy, @francescobrivio, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@fabiocos, @makortel, @felicepantaleo, @fabferro, @GiacomoSguazzoni, @JanFSchulte, @tocheng, @VinInn, @Martin-Grunewald, @missirol, @rovere, @lecriste, @mtosi, @ebrondol, @mmusich, @dgulhan, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[perrotta]

type bug-fix

fixes #35764

[perrotta]

The results of the matrix test still contain corrections set to 0 though. The reason for that is insufficient (100) number of events. So, is there any way to increase the number of events to be more than 100?

Please run the workflow yourself on a larger number of events

[perrotta]

please test

[tvami]

@perrotta is 1042 part of the standard relvals?

[tvami]

@cmsbuild , please abort

• it's not part of it

[tvami]

test parameters:

• workflows = 1042

[tvami]

@cmsbuild , please test

[MatiXOfficial]

Please run the workflow yourself on a larger number of events

@perrotta I'm not sure I understand what you mean. I can run the worker and the harvester on another dataset with a larger number of events (like here), but I don't know how to increase the number of events in the workflow.

[perrotta]

Please run the workflow yourself on a larger number of events

@perrotta I'm not sure I understand what you mean. I can run the worker and the harvester on another dataset with a larger number of events (like here), but I don't know how to increase the number of events in the workflow.

runTheMatrix.py -l 1042 --command "-n 1000" > & out &

[tvami]

@MatiXOfficial if you want your wf to have 1000 events in the RelVals that you can do by adding
-n 1000
in
https://github.com/cms-sw/cmssw/blob/master/Configuration/PyReleaseValidation/python/relval_steps.py#L2137
(or the equivalent 'TIER0EXPPPSCAL','ALCASPLITPPSALIG','ALCAHARVDPPSALIG' depending on which one you want to change)

I'm not sure this is the preferred way to go I'm just saying this is the technical way to do it

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7758b1/20058/summary.html
COMMIT: 03db82d
CMSSW: CMSSW_12_2_X_2021-10-28-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35874/20058/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-7758b1/1042.0_RunExpressPhy2017F+RunExpressPhy2017F+TIER0EXPPPSCALALIG+ALCASPLITPPSALIG+ALCAHARVDPPSALIG

Summary:

• No significant changes to the logs found
• Reco comparison results: 2 differences found in the comparisons
• DQMHistoTests: Total files compared: 42
• DQMHistoTests: Total histograms compared: 2901440
• DQMHistoTests: Total failures: 5
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 2901412
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.004 KiB( 41 files compared)
• DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
• Checked 177 log files, 37 edm output root files, 42 DQM output files
• TriggerResults: no differences found

[tvami]

Nice, I see non-empty plots, as expected after all!

[tvami]

+alca

• see Fixed the PPS alignment relval - 1042 #35874 (comment)
• Jenkins tests pass

[tvami]

@cms-sw/pdmv-l2 @cms-sw/upgrade-l2
this PR changes the number of events in this wf, but it still runs very fast, please consider signing

[srimanob]

+Upgrade

[bbilin]

@MatiXOfficial @tvami I think we would like to drop the -n 1000 option in two places. Otherwise it looks ok.

I take it back, there are other examples, it shall be kept as it is.

[bbilin]

+1

[tvami]

@perrotta @qliphy this is essentially fully signed now

[qliphy]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.