Cleaning of several Analysis packages #32917

silviodonato · 2021-02-15T11:44:12Z

PR description:

This PR removes several obsolete Analysis packages. The cleaning has been already discussed with PAG/POG, but it will be further discussed with Physics Coordination.
The list of the obsolete modules is here https://docs.google.com/spreadsheets/d/10jq3dyvypt0axQ_l_ZL3GQQxkkyKek3VDQM2DrowhNI/edit#gid=0

This PR requires #32916, #32914, and it is also related to #32915.

The packages which are removed are

DiffractiveForwardAnalysis/Configuration
DiffractiveForwardAnalysis/Skimming
ElectroWeakAnalysis/Configuration
ElectroWeakAnalysis/Skimming
ElectroWeakAnalysis/Utilities
ElectroWeakAnalysis/WENu
ElectroWeakAnalysis/WMuNu
ElectroWeakAnalysis/ZEE
ElectroWeakAnalysis/ZMuMu
QCDAnalysis/ChargedHadronSpectra
QCDAnalysis/Configuration
QCDAnalysis/Skimming
QCDAnalysis/UEAnalysis
HeavyFlavorAnalysis/Configuration
HeavyFlavorAnalysis/Skimming
HiggsAnalysis/HiggsToGammaGamma
HiggsAnalysis/Configuration
HiggsAnalysis/Skimming
TopQuarkAnalysis/Examples
TopQuarkAnalysis/TopPairBSM
SUSYBSMAnalysis/HSCP
MuonAnalysis/Configuration

In SUSYBSMAnalysis/HSCP/plugins/Skim_HSCPFilter.cc
there is the definition of HSCPFilter that is used in
Configuration/Skimming/python/PDWG_EXOHSCP_cff.py
which is eventually used in
Configuration/Skimming/python/Skims_PDWG_cff.py

So this PR removes also Configuration/Skimming/python/PDWG_EXOHSCP_cff.py and few lines in Configuration/Skimming/python/Skims_PDWG_cff.py (see commit 1c74159)
The last PR changing Configuration/Skimming/python/PDWG_EXOHSCP_cff.py was done in 2015 (#7518)
Configuration/Skimming/python/Skims_PDWG_cff.py is not changed since we moved to github (CMSSW_6_2_0_pre8, 8 years ago) - at least.

PR validation:

I checked the dependencies of these packages in the rest of CMSSW code
(https://github.com/silviodonato/usercode/blob/master/checkDependencies.py)

if this PR is a backport please specify the original PR and why you need to backport that PR:

No backport expected.

silviodonato · 2021-02-15T11:44:53Z

test parameters:

pull_requests: Cleaning of Configuration/StandardSequences/python/Analysis_cff.py and FastSimulation/Calorimetry/python/TreeWriterForEcalCorrection_cfg.py #32916, Remove HLTriggerOffline/Egamma/test/testPatRecoElectrons.py #32914

silviodonato · 2021-02-15T11:45:43Z

hold
(this needs to be discussed a bit further with PC)

cmsbuild · 2021-02-15T11:51:08Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32917/21145

This PR adds an extra 8KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File ElectroWeakAnalysis/WENu/interface/GenPurposeSkimmer.h modified in PR(s): delete unused headers #31870
- File ElectroWeakAnalysis/WENu/interface/GenPurposeSkimmerAcceptance.h modified in PR(s): delete unused headers #31870
- File ElectroWeakAnalysis/ZEE/interface/AnalysisErsatz.h modified in PR(s): delete unused headers #31870
- File ElectroWeakAnalysis/ZMuMu/bin/csa08IsoBkg.cpp modified in PR(s): delete unused headers #31870
- File ElectroWeakAnalysis/ZMuMu/bin/csa08ZFit.cpp modified in PR(s): delete unused headers #31870
- File ElectroWeakAnalysis/ZMuMu/bin/zMassFit.cpp modified in PR(s): delete unused headers #31870
- File ElectroWeakAnalysis/ZMuMu/bin/zMassHSFit.cpp modified in PR(s): delete unused headers #31870
- File ElectroWeakAnalysis/ZMuMu/bin/zMassMCFit.cpp modified in PR(s): delete unused headers #31870
- File ElectroWeakAnalysis/ZMuMu/bin/zMassResFit.cpp modified in PR(s): delete unused headers #31870
- File ElectroWeakAnalysis/ZMuMu/bin/zMuMuExpFit.cpp modified in PR(s): delete unused headers #31870
- File ElectroWeakAnalysis/ZMuMu/bin/zMuMuFit.cpp modified in PR(s): delete unused headers #31870
- File ElectroWeakAnalysis/ZMuMu/bin/zToMuMuMassMCFit.cpp modified in PR(s): delete unused headers #31870

cmsbuild · 2021-02-15T11:51:33Z

Pull request has been put on hold by @silviodonato
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

cmsbuild · 2021-02-15T11:51:36Z

A new Pull Request was created by @silviodonato (Silvio Donato) for master.

It involves the following packages:

DiffractiveForwardAnalysis/Configuration
DiffractiveForwardAnalysis/Skimming
ElectroWeakAnalysis/Configuration
ElectroWeakAnalysis/Skimming
ElectroWeakAnalysis/Utilities
ElectroWeakAnalysis/WENu
ElectroWeakAnalysis/WMuNu
ElectroWeakAnalysis/ZEE
ElectroWeakAnalysis/ZMuMu
HeavyFlavorAnalysis/Configuration
HeavyFlavorAnalysis/Skimming
HiggsAnalysis/Configuration
HiggsAnalysis/HiggsToGammaGamma
HiggsAnalysis/Skimming
MuonAnalysis/Configuration
QCDAnalysis/Configuration
QCDAnalysis/Skimming
QCDAnalysis/UEAnalysis
SUSYBSMAnalysis/HSCP
TopQuarkAnalysis/Examples
TopQuarkAnalysis/TopPairBSM

@cmsbuild, @santocch can you please review it and eventually sign? Thanks.
@bellan, @jdolen this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

silviodonato · 2021-02-15T11:52:58Z

test parameters:

pull_requests = Cleaning of Configuration/StandardSequences/python/Analysis_cff.py and FastSimulation/Calorimetry/python/TreeWriterForEcalCorrection_cfg.py #32916, Remove HLTriggerOffline/Egamma/test/testPatRecoElectrons.py #32914

silviodonato · 2021-02-15T11:53:35Z

please test

silviodonato · 2021-02-15T11:57:08Z

@cms-sw/core-l2

ElectroWeakAnalysis/Utilities appears in Utilities/ReleaseScripts/scripts/CMSSWReleaseDependency.rules, shall I remove that line?
all packages appear in Utilities/ReleaseScripts/scripts/git-publish should I modify it? (I'd say no... it sounds something related to https://github.com/cms-sw/cms-bot/blob/master/categories_map.py)

cmsbuild · 2021-02-15T15:06:42Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-565b99/12889/summary.html
COMMIT: ffb762c
CMSSW: CMSSW_11_3_X_2021-02-15-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/32917/12889/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

4.764.76_ZMuSkim2012D+ZMuSkim2012D+HLTDSKIM2+RECODR1reHLT2+HARVESTDR1reHLT/step2_ZMuSkim2012D+ZMuSkim2012D+HLTDSKIM2+RECODR1reHLT2+HARVESTDR1reHLT.log

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-565b99/11634.911_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA

Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2750846
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2750821
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 36 files compared)
Checked 156 log files, 37 edm output root files, 37 DQM output files

silviodonato · 2021-02-15T15:54:15Z

The error is not related to this PR

----- Begin Fatal Exception 15-Feb-2021 13:15:41 CET-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://cms-xrd-global.cern.ch//store/data/Run2012D/SingleMu/RAW-RECO/ZMu-15Apr2014-v1/00000/007A3BD1-02CD-E311-853B-002590D0AFC8.root'
   Additional Info:
      [a] Calling RootInputFileSequence::initTheFile(): fail to open the file with name root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/data/Run2012D/SingleMu/RAW-RECO/ZMu-15Apr2014-v1/00000/007A3BD1-02CD-E311-853B-002590D0AFC8.root
      [b] Input file root://cms-xrd-global.cern.ch//store/data/Run2012D/SingleMu/RAW-RECO/ZMu-15Apr2014-v1/00000/007A3BD1-02CD-E311-853B-002590D0AFC8.root could not be opened.
      [c] XrdCl::File::Open(name='root://cms-xrd-global.cern.ch//store/data/Run2012D/SingleMu/RAW-RECO/ZMu-15Apr2014-v1/00000/007A3BD1-02CD-E311-853B-002590D0AFC8.root', flags=0x10, permissions=0660) => error '[ERROR] Server responded with an error: [3011] No servers are available to read the file.�
' (errno=3011, code=400). No additional data servers were found.
      [d] Last URL tried: root://cms-xrd-global.cern.ch:1094//store/data/Run2012D/SingleMu/RAW-RECO/ZMu-15Apr2014-v1/00000/007A3BD1-02CD-E311-853B-002590D0AFC8.root?tried=&xrdcl.requuid=6bf4650c-fe39-4d9b-b4d0-51868268f097
      [e] Problematic data server: cms-xrd-global.cern.ch:1094
      [f] Disabled source: cms-xrd-global.cern.ch:1094
----- End Fatal Exception -------------------------------------------------

silviodonato · 2021-02-15T15:59:13Z

please test

cmsbuild · 2021-02-23T13:59:34Z

Pull request #32917 was updated. @chayanit, @wajidalikhan, @santocch, @jordan-martins can you please check and sign again.

cmsbuild · 2021-02-23T16:38:30Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-565b99/13047/summary.html
COMMIT: 1c74159
CMSSW: CMSSW_11_3_X_2021-02-22-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32917/13047/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2750983
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2750960
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 36 files compared)
Checked 156 log files, 37 edm output root files, 37 DQM output files

silviodonato · 2021-02-25T09:28:41Z

unhold

silviodonato · 2021-02-25T09:29:38Z

It requires #32916

silviodonato · 2021-02-25T09:35:54Z

@cms-sw/pdmv-l2 I've added some lines in the PR description about the changes in Configuration/Skimming. Please let me know if you have further comments

chayanit · 2021-02-25T09:41:44Z

Let me include @kfjack to double check on skimming. Thanks in advance.

silviodonato · 2021-02-26T14:28:24Z

@chayanit @kfjack I wrote to Todd Adams about a week ago who forwarded the request to the HSCP group, and he did not get any objection about the deletion of PDWG_EXOHSCP and the removal of it from Skims_PDWG_cff

silviodonato · 2021-03-01T08:36:41Z

@cms-sw/pdmv-l2 @kfjack do you have any comment/objection on this PR?

chayanit · 2021-03-01T08:41:16Z

+1

I think it is OK.

kfjack · 2021-03-01T08:51:51Z

+1
Ok too. These skims are not active now.

silviodonato · 2021-03-01T08:55:04Z

merge

santocch · 2021-03-07T08:14:45Z

+1

cmsbuild · 2021-03-07T08:15:14Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

Cleaning of some Analysis packages

ffb762c

cmsbuild added this to the CMSSW_11_3_X milestone Feb 15, 2021

cmsbuild added analysis-pending code-checks-pending orp-pending pending-signatures tests-pending labels Feb 15, 2021

cmsbuild added the hold label Feb 15, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 15, 2021

cmsbuild added the requires-external label Feb 15, 2021

cmsbuild added tests-started and removed tests-pending labels Feb 15, 2021

cmsbuild added tests-rejected and removed tests-started labels Feb 15, 2021

cmsbuild removed the tests-rejected label Feb 15, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 23, 2021

cmsbuild added tests-approved and removed tests-started labels Feb 23, 2021

cmsbuild removed the hold label Feb 25, 2021

cmsbuild mentioned this pull request Feb 28, 2021

Remove not needed RooFit dependencies from BuildFiles #33019

Merged

cmsbuild added pdmv-approved and removed pdmv-pending labels Mar 1, 2021

cmsbuild added orp-approved and removed orp-pending labels Mar 1, 2021

cmsbuild merged commit 733bf09 into cms-sw:master Mar 1, 2021

cmsbuild added analysis-approved fully-signed and removed analysis-pending pending-signatures labels Mar 7, 2021

This was referenced Mar 9, 2021

Removal of git-publish #33112

Merged

Cleaning of ElectroWeakAnalysis in CMSSWReleaseDependency.rules #33113

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cleaning of several Analysis packages #32917

Cleaning of several Analysis packages #32917

silviodonato commented Feb 15, 2021 •

edited

Loading

silviodonato commented Feb 15, 2021

silviodonato commented Feb 15, 2021

cmsbuild commented Feb 15, 2021

cmsbuild commented Feb 15, 2021

cmsbuild commented Feb 15, 2021

silviodonato commented Feb 15, 2021

silviodonato commented Feb 15, 2021

silviodonato commented Feb 15, 2021

cmsbuild commented Feb 15, 2021

silviodonato commented Feb 15, 2021

silviodonato commented Feb 15, 2021

cmsbuild commented Feb 23, 2021

cmsbuild commented Feb 23, 2021

silviodonato commented Feb 25, 2021

silviodonato commented Feb 25, 2021

silviodonato commented Feb 25, 2021

chayanit commented Feb 25, 2021

silviodonato commented Feb 26, 2021

silviodonato commented Mar 1, 2021

chayanit commented Mar 1, 2021 •

edited

Loading

kfjack commented Mar 1, 2021

silviodonato commented Mar 1, 2021

santocch commented Mar 7, 2021

cmsbuild commented Mar 7, 2021

Cleaning of several Analysis packages #32917

Cleaning of several Analysis packages #32917

Conversation

silviodonato commented Feb 15, 2021 • edited Loading

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

silviodonato commented Feb 15, 2021

silviodonato commented Feb 15, 2021

cmsbuild commented Feb 15, 2021

cmsbuild commented Feb 15, 2021

cmsbuild commented Feb 15, 2021

silviodonato commented Feb 15, 2021

silviodonato commented Feb 15, 2021

silviodonato commented Feb 15, 2021

cmsbuild commented Feb 15, 2021

RelVals-INPUT

Comparison Summary

silviodonato commented Feb 15, 2021

silviodonato commented Feb 15, 2021

cmsbuild commented Feb 23, 2021

cmsbuild commented Feb 23, 2021

Comparison Summary

silviodonato commented Feb 25, 2021

silviodonato commented Feb 25, 2021

silviodonato commented Feb 25, 2021

chayanit commented Feb 25, 2021

silviodonato commented Feb 26, 2021

silviodonato commented Mar 1, 2021

chayanit commented Mar 1, 2021 • edited Loading

kfjack commented Mar 1, 2021

silviodonato commented Mar 1, 2021

santocch commented Mar 7, 2021

cmsbuild commented Mar 7, 2021

silviodonato commented Feb 15, 2021 •

edited

Loading

chayanit commented Mar 1, 2021 •

edited

Loading