HBHEGPU: added taggedBadByDb by ChannelQuality in GPU [12_0_X] #35355

mariadalfonso · 2021-09-21T13:27:20Z

This PR add a functionality to the Mahi on GPU that take into account the HcalChannelQuality to resynch with what is done on CPU

On CPU the rechit is dropped, while for the GPU energy is set to 0.
Tested on Run3 MC, and disable the ieta=18, depth1. (Those channel are present only at DIGI, but the fiber is not connected to the scintillator, so energy is not meaningful.)

Help in solving some differences observed between the Run3 HLT menu

backport of #35357

@fwyzard @abdoulline @silviodonato

cmsbuild · 2021-09-21T13:27:47Z

A new Pull Request was created by @mariadalfonso for CMSSW_12_0_X.

It involves the following packages:

CondFormats/HcalObjects (db, alca)
RecoLocalCalo/HcalRecProducers (reconstruction)

@malbouis, @yuanchao, @cmsbuild, @jpata, @slava77, @ggovi, @francescobrivio, @tvami can you please review it and eventually sign? Thanks.
@apsallid, @tocheng, @bsunanda, @mmusich, @abdoulline, @seemasharmafnal this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Backported from HBHEGPU: Channel quality gpu [master] #35357

tvami · 2021-09-21T13:34:44Z

enable gpu

tvami · 2021-09-21T13:35:11Z

@mariadalfonso is there a specific relval wf that tests this?

mariadalfonso · 2021-09-21T13:39:09Z

@mariadalfonso is there a specific relval wf that tests this?

11634.522 this is TTbar_14TeV , 2021 , HCALOnlyGPU

tvami · 2021-09-21T13:43:07Z

test parameters:

workflows = 11634.522

tvami · 2021-09-21T13:43:44Z

@cmsbuild , please test

abdoulline · 2021-09-21T14:01:31Z

@mariadalfonso Thank you, Maria.
There is small typo in the introduction:
<...> On CPU the rechit is dropped, while for the CPU energy is set to 0 <...>
the latter is apparently GPU

francescobrivio · 2021-09-21T14:02:26Z

CondFormats/HcalObjects/interface/HcalChannelQualityGPU.h

+
+  cms::cuda::ESProduct<Product> product_;
+#endif  // __CUDACC__
+};


Sorry for the (maybe) naive question, but should this object be COND_SERIALIZABLE ?

@mariadalfonso @fwyzard please have a look at this comment

CondFormats/HcalObjects/interface/HcalChannelQuality.h is already COND_SERIALIZABLE
HcalChannelQualityGPU.h is not, but usually include the main .h
This is a common pattern for all conditions if need to do this change should be done in synch for all ECAL/HCAL/TRK ... and outside of this PR

Indeed, I don't think these conditions need to be declared COND_SERIALIZABLE.
They are never persisted or read from the database - they are only transient conditions, derived from the persistent ones and "repackaged" for use on GPUs.

ok thanks for the explanation @mariadalfonso and @fwyzard, makes sense!

cmsbuild · 2021-09-21T15:29:47Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4950a8/18791/summary.html
COMMIT: 25187e9
CMSSW: CMSSW_12_0_X_2021-09-21-1100/slc7_amd64_gcc900
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35355/18791/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19735
DQMHistoTests: Total failures: 10
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 19725
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

tvami · 2021-09-21T15:34:09Z

@mariadalfonso would have been better to start with the master, or at least wait with the second PR until tests and review finishes.
Anyway,

it seems 11634.522 is not a real wf? https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4950a8/18791/runTheMatrix-results/matrixTests.log
please modify the PR title and include [12_0_X] in it, please also modify the PR description and mention that this is a backport
is this needed for the test beams in October?

mariadalfonso · 2021-09-21T15:37:33Z

-1

Failed Tests: RelVals

Not sure why, I tested locally 11634.522 (GPU ) and compared with 11634.521 (CPU)

locally is discoverable like this: runTheMatrix.py -w gpu -l 11634.522

Traceback (most recent call last):
File "/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35355/18791/CMSSW_12_0_X_2021-09-21-1100/bin/slc7_amd64_gcc900/runTheMatrix.py", line 545, in
ret = runSelected(opt)
File "/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35355/18791/CMSSW_12_0_X_2021-09-21-1100/bin/slc7_amd64_gcc900/runTheMatrix.py", line 31, in runSelected
if len(undefSet)>0: raise ValueError('Undefined workflows: '+', '.join(map(str,list(undefSet))))
ValueError: Undefined workflows: 11634.522

mariadalfonso · 2021-09-21T15:38:02Z

is this needed for the test beams in October?

please ask @fwyzard

fwyzard · 2021-09-21T16:35:59Z

is this needed for the test beams in October?

Yes.

fwyzard · 2021-09-21T16:36:49Z

test parameters:
* workflows = 11634.522

I think you need to enable the gpu tests

tvami · 2021-09-21T16:45:46Z

test parameters:
* workflows = 11634.522
I think you need to enable the gpu tests

I thought I did that here: #35355 (comment)

mmusich · 2021-09-21T16:49:23Z

I thought I did that here: #35355 (comment)

I think the matrix should be told to use the gpu part of it. relvals_options = --what gpu or something like that.

tvami · 2021-09-21T17:01:01Z

test parameters:

workflows_gpu = 11634.522
enable_test = gpu
relvals_options = --what gpu

cmsbuild · 2021-09-28T21:54:03Z

Pull request #35355 was updated. @malbouis, @yuanchao, @missirol, @Martin-Grunewald, @jpata, @cmsbuild, @slava77, @ggovi, @francescobrivio, @tvami can you please check and sign again.

tvami · 2021-09-28T21:57:30Z

@cmsbuild , please test

cmsbuild · 2021-09-29T01:18:26Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4950a8/19215/summary.html
COMMIT: 82a7d35
CMSSW: CMSSW_12_0_X_2021-09-28-1100/slc7_amd64_gcc900
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35355/19215/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19735
DQMHistoTests: Total failures: 10
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 19725
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 2998564
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2998536
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

tvami · 2021-09-29T01:25:20Z

+alca

differences seen for HCAL GPU wf are expected

tvami · 2021-09-29T01:25:26Z

+db

missirol · 2021-09-29T05:18:20Z

+hlt

jpata · 2021-09-29T07:52:09Z

+reconstruction

previous reco sig HBHEGPU: added taggedBadByDb by ChannelQuality in GPU [12_0_X] #35355 (comment)
missing HLT customizations were added to sync with HBHEGPU: Channel quality gpu [master] #35357 (technically outside of reco, so we didn't bring this up)

cmsbuild · 2021-09-29T07:52:45Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-09-30T00:53:11Z

+1

cmsbuild added this to the CMSSW_12_0_X milestone Sep 21, 2021

cmsbuild added alca-pending db-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Sep 21, 2021

cmsbuild added tests-started and removed tests-pending labels Sep 21, 2021

francescobrivio reviewed Sep 21, 2021

View reviewed changes

mariadalfonso mentioned this pull request Sep 21, 2021

HBHEGPU: Channel quality gpu [master] #35357

Merged

cmsbuild added tests-rejected and removed tests-started labels Sep 21, 2021

mariadalfonso changed the title ~~HBHEGPU: added taggedBadByDb by ChannelQuality in GPU~~ HBHEGPU: added taggedBadByDb by ChannelQuality in GPU [12_0_X] Sep 21, 2021

cmsbuild added hlt-pending reconstruction-pending tests-pending and removed tests-approved labels Sep 28, 2021

cmsbuild added tests-started and removed tests-pending labels Sep 28, 2021

cmsbuild added tests-approved and removed tests-started labels Sep 29, 2021

cmsbuild added alca-approved db-approved and removed alca-pending db-pending labels Sep 29, 2021

cmsbuild added hlt-approved and removed hlt-pending labels Sep 29, 2021

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Sep 29, 2021

cmsbuild added orp-approved and removed orp-pending labels Sep 30, 2021

cmsbuild merged commit 91c9ba0 into cms-sw:CMSSW_12_0_X Sep 30, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBHEGPU: added taggedBadByDb by ChannelQuality in GPU [12_0_X] #35355

HBHEGPU: added taggedBadByDb by ChannelQuality in GPU [12_0_X] #35355

mariadalfonso commented Sep 21, 2021 •

edited

Loading

cmsbuild commented Sep 21, 2021 •

edited

Loading

tvami commented Sep 21, 2021

tvami commented Sep 21, 2021

mariadalfonso commented Sep 21, 2021

tvami commented Sep 21, 2021

tvami commented Sep 21, 2021

abdoulline commented Sep 21, 2021

francescobrivio Sep 21, 2021

tvami Sep 22, 2021

mariadalfonso Sep 23, 2021

fwyzard Sep 23, 2021

francescobrivio Sep 23, 2021

cmsbuild commented Sep 21, 2021

tvami commented Sep 21, 2021

mariadalfonso commented Sep 21, 2021 •

edited

Loading

mariadalfonso commented Sep 21, 2021

fwyzard commented Sep 21, 2021

fwyzard commented Sep 21, 2021

tvami commented Sep 21, 2021

mmusich commented Sep 21, 2021

tvami commented Sep 21, 2021 •

edited

Loading

cmsbuild commented Sep 28, 2021

tvami commented Sep 28, 2021

cmsbuild commented Sep 29, 2021

tvami commented Sep 29, 2021

tvami commented Sep 29, 2021

missirol commented Sep 29, 2021

jpata commented Sep 29, 2021

cmsbuild commented Sep 29, 2021

qliphy commented Sep 30, 2021

HBHEGPU: added taggedBadByDb by ChannelQuality in GPU [12_0_X] #35355

HBHEGPU: added taggedBadByDb by ChannelQuality in GPU [12_0_X] #35355

Conversation

mariadalfonso commented Sep 21, 2021 • edited Loading

cmsbuild commented Sep 21, 2021 • edited Loading

tvami commented Sep 21, 2021

tvami commented Sep 21, 2021

mariadalfonso commented Sep 21, 2021

tvami commented Sep 21, 2021

tvami commented Sep 21, 2021

abdoulline commented Sep 21, 2021

francescobrivio Sep 21, 2021

Choose a reason for hiding this comment

tvami Sep 22, 2021

Choose a reason for hiding this comment

mariadalfonso Sep 23, 2021

Choose a reason for hiding this comment

fwyzard Sep 23, 2021

Choose a reason for hiding this comment

francescobrivio Sep 23, 2021

Choose a reason for hiding this comment

cmsbuild commented Sep 21, 2021

RelVals

GPU Comparison Summary

tvami commented Sep 21, 2021

mariadalfonso commented Sep 21, 2021 • edited Loading

mariadalfonso commented Sep 21, 2021

fwyzard commented Sep 21, 2021

fwyzard commented Sep 21, 2021

tvami commented Sep 21, 2021

mmusich commented Sep 21, 2021

tvami commented Sep 21, 2021 • edited Loading

cmsbuild commented Sep 28, 2021

tvami commented Sep 28, 2021

cmsbuild commented Sep 29, 2021

GPU Comparison Summary

Comparison Summary

tvami commented Sep 29, 2021

tvami commented Sep 29, 2021

missirol commented Sep 29, 2021

jpata commented Sep 29, 2021

cmsbuild commented Sep 29, 2021

qliphy commented Sep 30, 2021

mariadalfonso commented Sep 21, 2021 •

edited

Loading

cmsbuild commented Sep 21, 2021 •

edited

Loading

mariadalfonso commented Sep 21, 2021 •

edited

Loading

tvami commented Sep 21, 2021 •

edited

Loading