Add Alpaka Implementation of PFClusterProducer #43130

jsamudio · 2023-10-27T06:47:53Z

PR description:

This pull request is a follow up to #42807. It adds SoA formats for PFClusters and PFRecHitFractions as well as an Alpaka based producer for them. The module relies on the usage of the PFRecHitProducer introduced in the #42807. ~~Included is an alpaka process modifier to use for cmsDriver commands. --procModifiers alpaka will replace the existing HCAL PF clustering with a chain using alpaka modules.~~ Once the changes in #43229 are merged, we will use --procModifiers alpaka to replace existing HCAL PF clustering with the alpaka equivalents. We plan an additional follow up to read HCAL thresholds from DB in this module once the changes in #43025 converge.

PR validation:

We have performed extensive validation of the PF Clusters and their use in the full PF Validation outlined in Validation/RecoParticleFlow/.

We observe only minor precision based energy differences in the clusters which have no significant impact to relevant PF outputs.

Validation plots are linked here: https://hep.baylor.edu/jsamudio/pfClusterProducerAlpakaValidation/

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

PR is not a backport.

@hatakeyamak

cmsbuild · 2023-10-27T06:56:05Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43130/37401

This PR adds an extra 108KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/ProcessModifiers/python/alpaka_cff.py modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File Configuration/StandardSequences/python/Services_cff.py modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File DataFormats/ParticleFlowReco/BuildFile.xml modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File DataFormats/ParticleFlowReco/interface/CaloRecHitHostCollection.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File DataFormats/ParticleFlowReco/interface/CaloRecHitSoA.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File DataFormats/ParticleFlowReco/interface/PFRecHitHostCollection.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File DataFormats/ParticleFlowReco/interface/PFRecHitSoA.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File DataFormats/ParticleFlowReco/interface/alpaka/CaloRecHitDeviceCollection.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File DataFormats/ParticleFlowReco/interface/alpaka/PFRecHitDeviceCollection.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File DataFormats/ParticleFlowReco/src/alpaka/classes_cuda.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File DataFormats/ParticleFlowReco/src/alpaka/classes_cuda_def.xml modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File DataFormats/ParticleFlowReco/src/alpaka/classes_rocm.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File DataFormats/ParticleFlowReco/src/alpaka/classes_rocm_def.xml modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File DataFormats/ParticleFlowReco/src/classes_serial.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File DataFormats/ParticleFlowReco/src/classes_serial_def.xml modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/BuildFile.xml modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/interface/PFRecHitParamsHostCollection.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/interface/PFRecHitParamsRecord.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/interface/PFRecHitParamsSoA.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/interface/PFRecHitTopologyHostCollection.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/interface/PFRecHitTopologyRecord.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/interface/PFRecHitTopologySoA.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/interface/alpaka/PFRecHitParamsDeviceCollection.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/interface/alpaka/PFRecHitTopologyDeviceCollection.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/plugins/BuildFile.xml modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/plugins/LegacyPFRecHitProducer.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/plugins/alpaka/CaloRecHitSoAProducer.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/plugins/alpaka/CalorimeterDefinitions.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/plugins/alpaka/PFRecHitECALParamsESProducer.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/plugins/alpaka/PFRecHitHCALParamsESProducer.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/plugins/alpaka/PFRecHitProducerKernel.dev.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/plugins/alpaka/PFRecHitProducerKernel.h modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/plugins/alpaka/PFRecHitSoAProducer.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/plugins/alpaka/PFRecHitTopologyESProducer.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/src/PFRecHitParamsHostCollection.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/src/PFRecHitParamsRecord.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/src/PFRecHitTopologyHostCollection.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/src/PFRecHitTopologyRecord.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/src/alpaka/PFRecHitParamsDeviceCollection.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/src/alpaka/PFRecHitTopologyDeviceCollection.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/test/BuildFile.xml modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/test/PFRecHitProducerTest.cc modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/test/drawValidationPlot.C modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/test/harvestValidationPlot.py modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807
- File RecoParticleFlow/PFRecHitProducer/test/pfRecHitProducerAlpakaValidation.py modified in PR(s): Add Alpaka implementation of PFRecHitProducer #42807

cmsbuild · 2023-10-27T06:56:37Z

A new Pull Request was created by @jsamudio for master.

It involves the following packages:

Configuration/ProcessModifiers (operations)
Configuration/StandardSequences (operations)
DataFormats/ParticleFlowReco (reconstruction)
RecoParticleFlow/PFClusterProducer (reconstruction)
RecoParticleFlow/PFClusterProducerAlpaka (****)
RecoParticleFlow/PFRecHitProducer (reconstruction)
Validation/RecoParticleFlow (dqm)

The following packages do not have a category, yet:

RecoParticleFlow/PFClusterProducerAlpaka
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@syuvivida, @antoniovilela, @cmsbuild, @fabiocos, @tjavaid, @rappoccio, @jfernan2, @antoniovagnerini, @rvenditti, @davidlange6, @mandrenguyen, @nothingface0 can you please review it and eventually sign? Thanks.
@thomreis, @ReyerBand, @slomeo, @mmusich, @dgulhan, @sameasy, @mtosi, @lgray, @VinInn, @makortel, @felicepantaleo, @argiro, @rchatter, @JanFSchulte, @ebrondol, @rovere, @GiacomoSguazzoni, @hatakeyamak, @VourMa, @mmarionncern, @missirol, @seemasharmafnal, @fabiocos, @Martin-Grunewald, @wang0jin this is something you requested to watch as well.
@sextonkennedy, @antoniovilela, @rappoccio you are the release manager for this.

cms-bot commands are listed here

swagata87 · 2023-10-27T07:04:12Z

type pf

makortel

Cursory look up to RecoParticleFlow/PFClusterProducerAlpaka/plugins/alpaka/PFClusterProducerAlpaka.cc.

One question I have is if the new package is really necessary? (I haven't digested the code enough to form an opinion though)

DataFormats/ParticleFlowReco/interface/CaloRecHitSoA.h

DataFormats/ParticleFlowReco/interface/PFClusterSoA.h

DataFormats/ParticleFlowReco/interface/PFRecHitFractionSoA.h

DataFormats/ParticleFlowReco/interface/PFRecHitSoA.h

DataFormats/ParticleFlowReco/interface/alpaka/CaloRecHitDeviceCollection.h

RecoParticleFlow/PFClusterProducerAlpaka/plugins/alpaka/PFClusterProducerAlpaka.cc

RecoParticleFlow/PFClusterProducerAlpaka/plugins/alpaka/PFClusterProducerKernel.dev.cc

RecoParticleFlow/PFClusterProducerAlpaka/plugins/AlpakaPFCommon.h

RecoParticleFlow/PFClusterProducerAlpaka/interface/alpaka/PFClusterProducerKernel.h

RecoParticleFlow/PFClusterProducerAlpaka/plugins/alpaka/PFClusterProducerAlpaka.cc

cmsbuild · 2023-10-28T21:36:29Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43130/37422

This PR adds an extra 92KB to repository
Found files with invalid states:
- RecoParticleFlow/PFClusterProducerAlpaka/interface/alpaka/PFClusterProducerKernel.h:
  - Added: e2b13b5
  - Modified: 048cd51
  - Deleted: 1e6a16d
- RecoParticleFlow/PFClusterProducerAlpaka/plugins/AlpakaPFCommon.h:
  - Added: e2b13b5
  - Modified: 048cd51
  - Deleted: 1e6a16d
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/ProcessModifiers/python/alpaka_cff.py modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File Configuration/StandardSequences/python/Services_cff.py modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43130/37422/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43130/37422/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

fwyzard · 2023-10-30T13:30:04Z

assign heterogeneous

hatakeyamak · 2023-10-30T14:58:27Z

@jsamudio re-run the code-format?

cmsbuild · 2023-10-30T15:44:18Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43130/37438

This PR adds an extra 32KB to repository
Found files with invalid states:
- RecoParticleFlow/PFClusterProducerAlpaka/interface/tmpEdgeSoA.h:
  - Added: e2b13b5
  - Modified: 048cd51
  - Deleted: a83a686
- RecoParticleFlow/PFClusterProducerAlpaka/interface/alpaka/tmpEdgeDeviceCollection.h:
  - Added: e2b13b5
  - Modified: 048cd51
  - Deleted: a83a686
- RecoParticleFlow/PFClusterProducerAlpaka/interface/tmpSoA.h:
  - Added: e2b13b5
  - Deleted: a83a686
- RecoParticleFlow/PFClusterProducerAlpaka/interface/alpaka/PFClusterProducerKernel.h:
  - Added: e2b13b5
  - Modified: 048cd51
  - Deleted: 1e6a16d
- RecoParticleFlow/PFClusterProducerAlpaka/plugins/AlpakaPFCommon.h:
  - Added: e2b13b5
  - Modified: 048cd51
  - Deleted: 1e6a16d
- RecoParticleFlow/PFClusterProducerAlpaka/interface/alpaka/tmpDeviceCollection.h:
  - Added: e2b13b5
  - Deleted: a83a686
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/ProcessModifiers/python/alpaka_cff.py modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File Configuration/StandardSequences/python/Services_cff.py modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117

cmsbuild · 2023-10-30T15:44:52Z

New categories assigned: heterogeneous

@fwyzard,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks

cmsbuild · 2023-12-11T15:11:22Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43130/38127

This PR adds an extra 20KB to repository

cmsbuild · 2023-12-11T15:11:55Z

Pull request #43130 was updated. @rvenditti, @antoniovagnerini, @fwyzard, @jfernan2, @mandrenguyen, @tjavaid, @nothingface0, @cmsbuild, @makortel, @syuvivida can you please check and sign again.

fwyzard · 2023-12-11T17:27:27Z

+heterogeneous

fwyzard · 2023-12-11T17:27:32Z

please test

cmsbuild · 2023-12-11T20:05:02Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b729dc/36418/summary.html
COMMIT: e381569
CMSSW: CMSSW_14_0_X_2023-12-11-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43130/36418/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 345 lines to the logs
Reco comparison results: 13 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3430794
DQMHistoTests: Total failures: 79
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3430693
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 214 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

mandrenguyen · 2023-12-11T20:10:30Z

+reconstruction

fwyzard · 2023-12-13T10:49:00Z

@cms-sw/dqm-l2 , could you please review or sign this PR ?

tjavaid · 2023-12-13T11:10:57Z

+1

cmsbuild · 2023-12-13T11:11:25Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @rappoccio, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

rappoccio · 2023-12-14T13:27:34Z

+1

cmsbuild added this to the CMSSW_13_3_X milestone Oct 27, 2023

cmsbuild added reconstruction-pending dqm-pending operations-pending pending-signatures tests-pending orp-pending new-package-pending code-checks-pending labels Oct 27, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 27, 2023

cmsbuild added the pf label Oct 27, 2023

makortel reviewed Oct 27, 2023

View reviewed changes

cmsbuild added code-checks-pending and removed code-checks-approved labels Oct 28, 2023

cmsbuild added code-checks-rejected and removed code-checks-pending labels Oct 28, 2023

swagata87 mentioned this pull request Oct 30, 2023

Adapt PF code to read Hcal thresholds from GT #43025

Merged

cmsbuild added the heterogeneous-pending label Oct 30, 2023

cmsbuild added code-checks-pending and removed code-checks-rejected labels Oct 30, 2023

cmsbuild removed the code-checks-pending label Oct 30, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Dec 11, 2023

cmsbuild added tests-started heterogeneous-approved and removed tests-pending heterogeneous-pending labels Dec 11, 2023

cmsbuild added tests-approved and removed tests-started labels Dec 11, 2023

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Dec 11, 2023

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Dec 13, 2023

cmsbuild added orp-approved and removed orp-pending labels Dec 14, 2023

cmsbuild merged commit 18fc414 into cms-sw:master Dec 14, 2023
11 checks passed

jsamudio mentioned this pull request Dec 14, 2023

Adapt Alpaka-based PF Clustering to read Hcal thresholds from GT #43574

Merged

cmsbuild mentioned this pull request Dec 15, 2023

[Do not merge] testing bot #43568

Closed

makortel mentioned this pull request Feb 14, 2024

Add {Copy,Move}ToDeviceCache<T> class templates and moveToDeviceAsync function template #43969

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Alpaka Implementation of PFClusterProducer #43130

Add Alpaka Implementation of PFClusterProducer #43130

jsamudio commented Oct 27, 2023 •

edited

Loading

cmsbuild commented Oct 27, 2023

cmsbuild commented Oct 27, 2023 •

edited

Loading

swagata87 commented Oct 27, 2023

makortel left a comment

cmsbuild commented Oct 28, 2023

fwyzard commented Oct 30, 2023

hatakeyamak commented Oct 30, 2023

cmsbuild commented Oct 30, 2023

cmsbuild commented Oct 30, 2023

cmsbuild commented Dec 11, 2023

cmsbuild commented Dec 11, 2023

fwyzard commented Dec 11, 2023

fwyzard commented Dec 11, 2023

cmsbuild commented Dec 11, 2023

mandrenguyen commented Dec 11, 2023

fwyzard commented Dec 13, 2023

tjavaid commented Dec 13, 2023

cmsbuild commented Dec 13, 2023

rappoccio commented Dec 14, 2023

Add Alpaka Implementation of PFClusterProducer #43130

Add Alpaka Implementation of PFClusterProducer #43130

Conversation

jsamudio commented Oct 27, 2023 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Oct 27, 2023

cmsbuild commented Oct 27, 2023 • edited Loading

swagata87 commented Oct 27, 2023

makortel left a comment

Choose a reason for hiding this comment

cmsbuild commented Oct 28, 2023

fwyzard commented Oct 30, 2023

hatakeyamak commented Oct 30, 2023

cmsbuild commented Oct 30, 2023

cmsbuild commented Oct 30, 2023

cmsbuild commented Dec 11, 2023

cmsbuild commented Dec 11, 2023

fwyzard commented Dec 11, 2023

fwyzard commented Dec 11, 2023

cmsbuild commented Dec 11, 2023

Comparison Summary

mandrenguyen commented Dec 11, 2023

fwyzard commented Dec 13, 2023

tjavaid commented Dec 13, 2023

cmsbuild commented Dec 13, 2023

rappoccio commented Dec 14, 2023

jsamudio commented Oct 27, 2023 •

edited

Loading

cmsbuild commented Oct 27, 2023 •

edited

Loading