Removing CUDA/`gpu` from Pixel code configs and dropping all CUDA wfs #46853

AdrianoDee · 2024-12-03T14:08:55Z

PR description:

This PR proposes:

the removal of all the CUDA modules from pixel-related configs and of all the CUDA Patatrack wfs;
the removal of pixelNtupletFit_cff modifier.

A subsequent step would be to remove the gpu modifier, but since this involves code from many parties, I prefer to have it separated.

cmsbuild · 2024-12-03T14:09:22Z

cms-bot internal usage

AdrianoDee · 2024-12-03T14:10:02Z

Well, apparently already these changes involve many parties. So maybe I'll push the drop of the gpu modifier already here.

AdrianoDee · 2024-12-03T14:10:25Z

enable gpu

AdrianoDee · 2024-12-03T14:10:28Z

please test

cmsbuild · 2024-12-03T14:10:38Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46853/42877

There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/README.md modified in PR(s): Add run3 tracking low pu era #33532, ECAL RecHit producer Alpaka migration #46453, Addition of single iteration Patatrack and LST HLT configurations and workflows #46828
- File Configuration/PyReleaseValidation/python/relval_gpu.py modified in PR(s): ECAL RecHit producer Alpaka migration #46453
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Add run3 tracking low pu era #33532, ECAL RecHit producer Alpaka migration #46453, Allowing Cluster Splitting with HLT Vertices #46694, Addition of single iteration Patatrack and LST HLT configurations and workflows #46828
- File EventFilter/SiPixelRawToDigi/python/SiPixelRawToDigi_cfi.py modified in PR(s): Remove legacy CUDA modules for pixel track and vertex reconstruction #45853, Remove the configuration of the legacy CUDA workflows #46076
- File EventFilter/SiPixelRawToDigi/python/siPixelDigis_cff.py modified in PR(s): Remove legacy CUDA modules for pixel track and vertex reconstruction #45853, Remove the configuration of the legacy CUDA workflows #46076
- File RecoHI/HiTracking/python/HILowPtConformalPixelTracks_cfi.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
- File RecoLocalTracker/SiPixelClusterizer/python/SiPixelClusterizerPreSplitting_cfi.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
- File RecoLocalTracker/SiPixelClusterizer/python/siPixelClustersPreSplitting_cff.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
- File RecoLocalTracker/SiPixelRecHits/python/PixelCPEESProducers_cff.py modified in PR(s): Remove legacy CUDA modules for pixel track and vertex reconstruction #45853
- File RecoLocalTracker/SiPixelRecHits/python/SiPixelRecHits_cfi.py modified in PR(s): New Heterogeneous Memory Pool #37952, Remove the configuration of the legacy CUDA workflows #46076
- File RecoTracker/PixelTrackFitting/python/PixelTracks_cff.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
- File RecoVertex/BeamSpotProducer/python/BeamSpot_cff.py modified in PR(s): Remove legacy CUDA modules for pixel track and vertex reconstruction #45853, Remove the configuration of the legacy CUDA workflows #46076
- File RecoVertex/Configuration/python/RecoPixelVertexing_cff.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076

cmsbuild · 2024-12-03T14:10:58Z

A new Pull Request was created by @AdrianoDee for master.

It involves the following packages:

Configuration/ProcessModifiers (operations)
Configuration/PyReleaseValidation (pdmv, upgrade)
EventFilter/SiPixelRawToDigi (reconstruction)
RecoHI/HiTracking (reconstruction)
RecoLocalTracker/SiPixelClusterizer (reconstruction)
RecoLocalTracker/SiPixelRecHits (reconstruction)
RecoTracker/PixelTrackFitting (reconstruction)
RecoVertex/BeamSpotProducer (reconstruction, alca)
RecoVertex/Configuration (reconstruction)

@AdrianoDee, @Moanwar, @antoniovilela, @atpathak, @consuegs, @davidlange6, @DickyChant, @fabiocos, @jfernan2, @mandrenguyen, @miquork, @perrotta, @rappoccio, @srimanob, @subirsarkar can you please review it and eventually sign? Thanks.
@GiacomoSguazzoni, @Martin-Grunewald, @VinInn, @VourMa, @dgulhan, @dkotlins, @fabiocos, @felicepantaleo, @ferencek, @francescobrivio, @gpetruc, @jazzitup, @kurtejung, @makortel, @mandrenguyen, @martinamalberti, @missirol, @mmusich, @mroguljic, @mtosi, @rovere, @rsreds, @slomeo, @threus, @tocheng, @tsusa, @tvami, @yenjie, @yetkinyilmaz, @yuanchao this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

jfernan2 · 2024-12-03T14:40:19Z

assign heterogeneous

cmsbuild · 2024-12-03T14:40:42Z

New categories assigned: heterogeneous

@fwyzard,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks

cmsbuild · 2024-12-03T15:08:09Z

-1

Failed Tests: UnitTests RelVals RelVals-GPU RelVals-INPUT AddOn
Size: This PR adds an extra 56KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-693680/43213/summary.html
COMMIT: 5d626de
CMSSW: CMSSW_15_0_X_2024-12-03-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46853/43213/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 34 errors in the following unit tests:

---> test TestDQMOnlineClient-es_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beamhlt_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-csc_dqm_sourceclient had ERRORS
and more ...

RelVals

135.4135.4_ZEEFS_13/step1_ZEEFS_13.log
1306.01306.0_SingleMuPt1_UP15/step1_SingleMuPt1_UP15.log
7.37.3_CosmicsSPLoose2018/step1_CosmicsSPLoose2018.log

Expand to see more relval errors ...

RelVals-GPU

12834.42312834.423_TTbar_14TeV+2024_Patatrack_HCALOnlyGPUandAlpaka_Validation/step1_TTbar_14TeV+2024_Patatrack_HCALOnlyGPUandAlpaka_Validation.log
12834.42212834.422_TTbar_14TeV+2024_Patatrack_HCALOnlyAlpaka_Validation/step1_TTbar_14TeV+2024_Patatrack_HCALOnlyAlpaka_Validation.log
12834.40612834.406_TTbar_14TeV+2024_Patatrack_PixelOnlyTripletsAlpaka/step1_TTbar_14TeV+2024_Patatrack_PixelOnlyTripletsAlpaka.log

Expand to see more relval errors ...

RelVals-INPUT

159.01159.01_HydjetQ_reminiaodPbPb2022_INPUT/step2_HydjetQ_reminiaodPbPb2022_INPUT.log
136.875136.875_RunDoubleMuon2018C/step2_RunDoubleMuon2018C.log
2500.2252500.225_jmeNANOrePuppimc140X/step2_jmeNANOrePuppimc140X.log

Expand to see more relval errors ...

AddOn Tests

fastsim

UNKNOWN

fastsim1

UNKNOWN

fastsim2

UNKNOWN

Expand to see more addon errors ...

AdrianoDee · 2025-01-09T09:06:42Z

Thanks @mmusich

@cms-sw/alca-l2 @cms-sw/reconstruction-l2 @cms-sw/heterogeneous-l2 @cms-sw/upgrade-l2 this should be ready to go (the last couple of commits were only rebases).

fwyzard · 2025-01-09T09:09:24Z

+1

perrotta · 2025-01-09T09:10:42Z

+alca

Signing again, as in Removing CUDA/gpu from Pixel code configs and dropping all CUDA wfs #46853 (comment)

Moanwar · 2025-01-09T09:16:38Z

+Upgrade

jfernan2 · 2025-01-09T09:32:24Z

+1

cmsbuild · 2025-01-09T09:32:45Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @sextonkennedy, @rappoccio, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2025-01-09T10:20:09Z

+1

makortel · 2025-01-10T01:16:28Z

RecoLocalTracker/SiPixelRecHits/python/SiPixelRecHits_cfi.py

            pixelRecHitSrc = cms.InputTag('siPixelRecHitsPreSplittingAlpaka'),
            src = cms.InputTag('siPixelClustersPreSplitting'))
 )

-(alpaka & pp_on_AA & ~phase2_tracker).toModify(siPixelRecHitsPreSplitting,
-    cpu = _siPixelRecHitFromSoAAlpakaHIonPhase1.clone(
+(alpaka & pp_on_AA & ~phase2_tracker).toModify(siPixelRecHitsPreSplitting, _siPixelRecHitFromSoAAlpakaHIonPhase1.clone(


This line should have been

(alpaka & pp_on_AA & ~phase2_tracker).toReplaceWith(siPixelRecHitsPreSplitting, _siPixelRecHitFromSoAAlpakaHIonPhase1.clone(

Workflow 160.401 fails in the IB with

Traceback (most recent call last): File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_X_2025-01-08-2300/bin/el8_amd64_gcc12/cmsDriver.py", line 40, in <module> run() File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_X_2025-01-08-2300/bin/el8_amd64_gcc12/cmsDriver.py", line 16, in run configBuilder.prepare() File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_X_2025-01-08-2300/src/Configuration/Applications/python/ConfigBuilder.py", line 2284, in prepare self.addStandardSequences() File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_X_2025-01-08-2300/src/Configuration/Applications/python/ConfigBuilder.py", line 826, in addStandardSequences getattr(self,"prepare_"+stepName)(stepSpec = getattr(self,stepName+"DefaultSeq")) File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_X_2025-01-08-2300/src/Configuration/Applications/python/ConfigBuilder.py", line 1743, in prepare_RECO _,_recoSeq,_ = self.loadDefaultOrSpecifiedCFF(stepSpec,self.RECODefaultCFF) File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_X_2025-01-08-2300/src/Configuration/Applications/python/ConfigBuilder.py", line 1275, in loadDefaultOrSpecifiedCFF l=self.loadAndRemember(_cff) File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_X_2025-01-08-2300/src/Configuration/Applications/python/ConfigBuilder.py", line 354, in loadAndRemember self.process.load(includeFile) File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_X_2025-01-08-2300/src/FWCore/ParameterSet/python/Config.py", line 760, in load module = __import__(moduleName) File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_0_X_2025-01-09-2300/src/Configuration/StandardSequences/python/Reconstruction_cff.py", line 8, in <module> from RecoTracker.Configuration.RecoTracker_cff import * File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_0_X_2025-01-09-2300/src/RecoTracker/Configuration/python/RecoTracker_cff.py", line 5, in <module> from RecoTracker.IterativeTracking.iterativeTk_cff import * File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_0_X_2025-01-09-2300/src/RecoTracker/IterativeTracking/python/iterativeTk_cff.py", line 4, in <module> from RecoTracker.IterativeTracking.InitialStepPreSplitting_cff import * File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_0_X_2025-01-09-2300/src/RecoTracker/IterativeTracking/python/InitialStepPreSplitting_cff.py", line 245, in <module> from RecoLocalTracker.SiPixelRecHits.SiPixelRecHits_cfi import siPixelRecHits File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_0_X_2025-01-09-2300/src/RecoLocalTracker/SiPixelRecHits/python/SiPixelRecHits_cfi.py", line 62, in <module> (alpaka & pp_on_AA & ~phase2_tracker).toModify(siPixelRecHitsPreSplitting, _siPixelRecHitFromSoAAlpakaHIonPhase1.clone( File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_X_2025-01-08-2300/src/FWCore/ParameterSet/python/Config.py", line 1783, in toModify Modifier._toModify(obj,func,**kw) File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02871/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_X_2025-01-08-2300/src/FWCore/ParameterSet/python/Config.py", line 1866, in _toModify func(obj) TypeError: 'EDProducer' object is not callable

https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc12/CMSSW_15_0_X_2025-01-09-2300/pyRelValMatrixLogs/run/160.401_HydjetQ_MinBias_5362GeV_2023_ppReco/step4_HydjetQ_MinBias_5362GeV_2023_ppReco.log#/

Thanks Matti, I opened this #47078 to fix it.

cmsbuild added this to the CMSSW_15_0_X milestone Dec 3, 2024

cmsbuild added reconstruction-pending alca-pending operations-pending pending-signatures tests-pending orp-pending pdmv-pending upgrade-pending code-checks-pending tracking trk labels Dec 3, 2024

cmsbuild added tests-started and removed tests-pending labels Dec 3, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Dec 3, 2024

cmsbuild added the heterogeneous-pending label Dec 3, 2024

cmsbuild added tests-rejected and removed tests-started labels Dec 3, 2024

AdrianoDee force-pushed the remove_cuda_wfs_and_pixel_cfgs branch from 5d626de to 77de940 Compare December 3, 2024 15:12

cmsbuild removed the tests-rejected label Dec 3, 2024

cmsbuild added the tests-approved label Jan 9, 2025

cmsbuild added heterogeneous-approved and removed heterogeneous-pending labels Jan 9, 2025

cmsbuild added alca-approved and removed alca-pending labels Jan 9, 2025

cmsbuild added upgrade-approved and removed upgrade-pending labels Jan 9, 2025

cmsbuild added reconstruction-approved fully-signed and removed reconstruction-pending pending-signatures labels Jan 9, 2025

cmsbuild added orp-approved and removed orp-pending labels Jan 9, 2025

cmsbuild merged commit 04cdcf2 into cms-sw:master Jan 9, 2025
14 checks passed

AdrianoDee deleted the remove_cuda_wfs_and_pixel_cfgs branch January 9, 2025 10:21

makortel reviewed Jan 10, 2025

View reviewed changes

AdrianoDee mentioned this pull request Jan 10, 2025

Fix for HI Alpaka Pixel Configs #47078

Merged

mmusich mentioned this pull request Jan 10, 2025

speedup SiStripClusterizer(FromRaw) using ThreeThresholdAlgorithm #47061

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Removing CUDA/`gpu` from Pixel code configs and dropping all CUDA wfs #46853

Removing CUDA/`gpu` from Pixel code configs and dropping all CUDA wfs #46853

AdrianoDee commented Dec 3, 2024 •

edited

Loading

cmsbuild commented Dec 3, 2024 •

edited

Loading

AdrianoDee commented Dec 3, 2024 •

edited

Loading

AdrianoDee commented Dec 3, 2024

AdrianoDee commented Dec 3, 2024

cmsbuild commented Dec 3, 2024

cmsbuild commented Dec 3, 2024

jfernan2 commented Dec 3, 2024

cmsbuild commented Dec 3, 2024

cmsbuild commented Dec 3, 2024

AdrianoDee commented Jan 9, 2025

fwyzard commented Jan 9, 2025

perrotta commented Jan 9, 2025

Moanwar commented Jan 9, 2025

jfernan2 commented Jan 9, 2025

cmsbuild commented Jan 9, 2025

mandrenguyen commented Jan 9, 2025

makortel Jan 10, 2025

AdrianoDee Jan 10, 2025

Removing CUDA/gpu from Pixel code configs and dropping all CUDA wfs #46853

Removing CUDA/gpu from Pixel code configs and dropping all CUDA wfs #46853

Conversation

AdrianoDee commented Dec 3, 2024 • edited Loading

PR description:

cmsbuild commented Dec 3, 2024 • edited Loading

AdrianoDee commented Dec 3, 2024 • edited Loading

AdrianoDee commented Dec 3, 2024

AdrianoDee commented Dec 3, 2024

cmsbuild commented Dec 3, 2024

cmsbuild commented Dec 3, 2024

jfernan2 commented Dec 3, 2024

cmsbuild commented Dec 3, 2024

cmsbuild commented Dec 3, 2024

Unit Tests

RelVals

RelVals-GPU

RelVals-INPUT

AddOn Tests

AdrianoDee commented Jan 9, 2025

fwyzard commented Jan 9, 2025

perrotta commented Jan 9, 2025

Moanwar commented Jan 9, 2025

jfernan2 commented Jan 9, 2025

cmsbuild commented Jan 9, 2025

mandrenguyen commented Jan 9, 2025

makortel Jan 10, 2025

Choose a reason for hiding this comment

AdrianoDee Jan 10, 2025

Choose a reason for hiding this comment

Removing CUDA/`gpu` from Pixel code configs and dropping all CUDA wfs #46853

Removing CUDA/`gpu` from Pixel code configs and dropping all CUDA wfs #46853

AdrianoDee commented Dec 3, 2024 •

edited

Loading

cmsbuild commented Dec 3, 2024 •

edited

Loading

AdrianoDee commented Dec 3, 2024 •

edited

Loading