Patatrack integration - trivial changes (1/N) #29109

fwyzard · 2020-03-05T11:03:31Z

PR description:

Part of the overall Patatrack integration.

Add code rule exception for HeterogeneousCore.
Update tests following #28537.
Other trivial changes.

PR validation:

Compiles.

fwyzard · 2020-03-05T11:03:45Z

@cmsbuild, please test

cmsbuild · 2020-03-05T11:03:56Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-03-05T11:09:24Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29109/14010

This PR adds an extra 16KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Calibration/LumiAlCaRecoProducers/python/ALCARECOAlCaPCCRandom_cff.py modified in PR(s): PCC Event Producer 11_1_X target #29069, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File Calibration/LumiAlCaRecoProducers/python/ALCARECOAlCaPCCZeroBias_cff.py modified in PR(s): PCC Event Producer 11_1_X target #29069, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File Calibration/LumiAlCaRecoProducers/python/ALCARECOLumiPixels_cff.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File DataFormats/DetId/test/BuildFile.xml modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File DataFormats/DetId/test/test_detid.cu modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDACore/interface/ESProduct.h modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAServices/scripts/cmsCudaRebuild.sh modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAServices/scripts/cmsCudaSetup.sh modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAServices/scripts/cudaPreallocate.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File HeterogeneousCore/CUDAServices/scripts/nvprof-remote modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File Utilities/ReleaseScripts/python/cmsCodeRules/config.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983

cmsbuild · 2020-03-05T11:09:53Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5003/console Started: 2020/03/05 12:12

cmsbuild · 2020-03-05T11:09:54Z

A new Pull Request was created by @fwyzard (Andrea Bocci) for master.

It involves the following packages:

Calibration/LumiAlCaRecoProducers
DataFormats/DetId
HeterogeneousCore/CUDACore
HeterogeneousCore/CUDAServices
Utilities/ReleaseScripts

@smuzaffar, @civanch, @Dr15Jones, @makortel, @fwyzard, @christopheralanwest, @mdhildreth, @cmsbuild, @franzoni, @tocheng, @tlampen, @pohsun can you please review it and eventually sign? Thanks.
@makortel, @mmusich, @rovere, @wddgit, @tocheng this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2020-03-05T12:08:57Z

-1

Tested at: bd816fa

CMSSW: CMSSW_11_1_X_2020-03-04-2300
SCRAM_ARCH: slc7_amd64_gcc820
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f6110a/5003/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

RelVals:

When I ran the RelVals I found an error in the following workflows:
4.22 step2

runTheMatrix-results/4.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log

140.56 step2

runTheMatrix-results/140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log

4.53 step3

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step3_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

136.793 step3

runTheMatrix-results/136.793_RunDoubleEG2017C+RunDoubleEG2017C+HLTDR2_2017+RECODR2_2017reHLT_skimDoubleEG_Prompt+HARVEST2017/step3_RunDoubleEG2017C+RunDoubleEG2017C+HLTDR2_2017+RECODR2_2017reHLT_skimDoubleEG_Prompt+HARVEST2017.log

136.874 step3

runTheMatrix-results/136.874_RunEGamma2018C+RunEGamma2018C+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM/step3_RunEGamma2018C+RunEGamma2018C+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM.log

8.0 step4

runTheMatrix-results/8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step4_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

7.3 step3

runTheMatrix-results/7.3_CosmicsSPLoose_UP18+CosmicsSPLoose_UP18+DIGICOS_UP18+RECOCOS_UP18+ALCACOS_UP18+HARVESTCOS_UP18/step3_CosmicsSPLoose_UP18+CosmicsSPLoose_UP18+DIGICOS_UP18+RECOCOS_UP18+ALCACOS_UP18+HARVESTCOS_UP18.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

158.0 step3

runTheMatrix-results/158.0_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO/step3_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO.log

10042.0 step5

runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017/step5_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017.log

10024.0 step5

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017/step5_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017.log

10824.0 step5

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018/step5_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018.log

11634.0 step5

runTheMatrix-results/11634.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021/step5_TTbar_14TeV+TTbar_14TeV_TuneCP5_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021.log

12434.0 step5

runTheMatrix-results/12434.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023/step5_TTbar_14TeV+TTbar_14TeV_TuneCP5_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023.log

cmsbuild · 2020-03-05T12:08:58Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

Calibration/LumiAlCaRecoProducers/python/ALCARECOAlCaPCCRandom_cff.py

makortel · 2020-03-05T14:53:39Z

Utilities/ReleaseScripts/python/cmsCodeRules/config.py

@@ -63,7 +63,7 @@

 Configuration[ruleName]['description'] = 'Search for "catch(...)" statements in *.cc, *.cxx files'
 Configuration[ruleName]['filesToMatch'] = ['*.cc', '*.cxx']
-Configuration[ruleName]['exceptPaths'] = ['FWCore/*', 'EventFilter/*', '*/*/test/*', '*/*/bin/*']
+Configuration[ruleName]['exceptPaths'] = ['FWCore/*', 'EventFilter/*', 'HeterogeneousCore/*', '*/*/test/*', '*/*/bin/*']


By quick look we no longer have any catch(...) inside HeterogeneousCore. If there is no other reason, could this addition be removed?

We have this:

cmssw/HeterogeneousCore/CUDACore/src/ScopedContext.cc

Lines 27 to 34 in 6dcf685

try {

auto error = cudaGetErrorName(status);

auto message = cudaGetErrorString(status);

throw cms::Exception("CUDAError") << "Callback of CUDA stream " << streamId << " in device " << device

<< " error " << error << ": " << message;

} catch (cms::Exception&) {

waitingTaskHolder.doneWaiting(std::current_exception());

}

If the rule checker does not complain, I can drop the exception.

As far as I can tell the check is for catch (...) and catch (cms::Exception&) should be fine.

Ah - I see.
OK, that is in the main releases since a while, so if there were no complaints, we are fine.

fwyzard · 2020-03-06T11:03:15Z

@cmsbuild, please test

Clean up the Patatrack code base following the comments received during the integration into the upstream release. Currently tracks the changes introduced due to - cms-sw#29109: Patatrack integration - trivial changes (1/N) - cms-sw#29110: Patatrack integration - common tools (2/N) List of changes: * Remove unused files * Fix compilation warnings * Fix AtomicPairCounter unit test * Rename the cudaCompat namespace to cms::cudacompat * Remove extra semicolon * Move SimpleVector and VecArray to the cms::cuda namespace * Add missing dependency * Move HistoContainer, AtomicPairCounter, prefixScan and radixSort to the cms::cuda namespace * Remove rule exception for HeterogeneousCore * Fix code rule violations: - replace using namespace cms::cuda in test/OneToManyAssoc_t.h . - add an exception for cudaCompat.h: cudaCompat relies on defining equivalent symbols to the CUDA intrinsics in the cms::cudacompat namespace, and pulling them in the global namespace when compiling device code without CUDA. * Protect the headers to compile only with a CUDA compiler

cmsbuild added this to the CMSSW_11_1_X milestone Mar 5, 2020

cmsbuild added alca-pending code-checks-pending comparison-pending core-pending heterogeneous-pending orp-pending pending-signatures simulation-pending tests-pending labels Mar 5, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 5, 2020

cmsbuild added tests-started and removed tests-pending labels Mar 5, 2020

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Mar 5, 2020

makortel reviewed Mar 5, 2020

View reviewed changes

fwyzard force-pushed the Patatrack_1_N_trivial_changes branch from bd816fa to 67ebd2e Compare March 6, 2020 11:02

cmsbuild removed alca-pending code-checks-approved labels Mar 6, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Patatrack integration - trivial changes (1/N) #29109

Patatrack integration - trivial changes (1/N) #29109

fwyzard commented Mar 5, 2020

fwyzard commented Mar 5, 2020

cmsbuild commented Mar 5, 2020

cmsbuild commented Mar 5, 2020

cmsbuild commented Mar 5, 2020 •

edited

Loading

cmsbuild commented Mar 5, 2020

cmsbuild commented Mar 5, 2020

cmsbuild commented Mar 5, 2020

makortel Mar 5, 2020

fwyzard Mar 6, 2020

makortel Mar 6, 2020

fwyzard Mar 6, 2020

fwyzard commented Mar 6, 2020

	try {
	auto error = cudaGetErrorName(status);
	auto message = cudaGetErrorString(status);
	throw cms::Exception("CUDAError") << "Callback of CUDA stream " << streamId << " in device " << device
	<< " error " << error << ": " << message;
	} catch (cms::Exception&) {
	waitingTaskHolder.doneWaiting(std::current_exception());
	}

Patatrack integration - trivial changes (1/N) #29109

Patatrack integration - trivial changes (1/N) #29109

Conversation

fwyzard commented Mar 5, 2020

PR description:

PR validation:

fwyzard commented Mar 5, 2020

cmsbuild commented Mar 5, 2020

cmsbuild commented Mar 5, 2020

cmsbuild commented Mar 5, 2020 • edited Loading

cmsbuild commented Mar 5, 2020

cmsbuild commented Mar 5, 2020

cmsbuild commented Mar 5, 2020

makortel Mar 5, 2020

Choose a reason for hiding this comment

fwyzard Mar 6, 2020

Choose a reason for hiding this comment

makortel Mar 6, 2020

Choose a reason for hiding this comment

fwyzard Mar 6, 2020

Choose a reason for hiding this comment

fwyzard commented Mar 6, 2020

cmsbuild commented Mar 5, 2020 •

edited

Loading