Technical update to the DeepJet and DeepDoubleX TF models #27

hqucms · 2020-03-10T20:39:23Z

A few technical updates to the TF models (e.g., remove nodes that are only used for training like cpf_input_batchnorm/keras_learning_phase from the graph, fix the DeepDoubleX models for batch inference). No change in the output is expected. Small (5-10%) speedup and slightly reduced model size.

The models are converted from the original Keras models using
https://github.com/hqucms/keras_to_tensorflow/blob/master/keras_to_tensorflow.py

[Updated on backward compatibility]

These models are intended to be used together w/ cms-sw/cmssw#29172, though it is OK to merge this PR (the DNN models) before cms-sw/cmssw#29172, because currently the inference in CMSSW is using ONNX (since cms-sw/cmssw#28112), so these TF models are not really used.

These models are NOT compatible w/ older releases (before cms-sw/cmssw#28112) -- one needs to remove the learning_phase input in https://github.com/cms-sw/cmssw/blob/8a7ee7592d72cc40cc0a95c371787ccc49f520b1/RecoBTag/TensorFlow/plugins/DeepDoubleXTFJetTagsProducer.cc#L136 and https://github.com/cms-sw/cmssw/blob/8a7ee7592d72cc40cc0a95c371787ccc49f520b1/RecoBTag/TensorFlow/plugins/DeepFlavourTFJetTagsProducer.cc#L141 to use these new models.

https://github.com/hqucms/keras_to_tensorflow/blob/master/keras_to_tensorflow.py

cmsbuild · 2020-03-10T20:39:49Z

A new Pull Request was created by @hqucms (Huilin Qu) for branch master.

@perrotta, @smuzaffar, @mrodozov, @cmsbuild, @slava77, @tulamor can you please review it and eventually sign? Thanks.
@emilbols, @smoortga, @riga, @JyothsnaKomaragiri, @mverzett, @ferencek, @andrzejnovak this is something you requested to watch as well.
cms-bot commands are listed here

slava77 · 2020-03-10T20:47:04Z

@cmsbuild please test

cmsbuild · 2020-03-10T20:47:28Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5115/console Started: 2020/03/10 21:58

cmsbuild · 2020-03-10T20:59:23Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmssw#29172
Test Parameters:

ARCHITECTURE_FILTER = slc7_ppc64le_gcc820
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5117/console Started: 2020/03/10 22:00

cmsbuild · 2020-03-10T22:30:36Z

+1
Tested at: 9f5e916
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b9ddee/5115/summary.html
CMSSW: CMSSW_11_1_X_2020-03-10-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-03-10T22:30:39Z

Comparison job queued.

cmsbuild · 2020-03-10T22:40:04Z

-1

Tested at: 9f5e916

CMSSW: CMSSW_11_1_X_2020-03-09-2300
SCRAM_ARCH: slc7_ppc64le_gcc820

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

#28902 @hqucms: MXNet-based implementation of the ParticleNet tagger

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-323b31/5117/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-323b31/5117/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-323b31/5117/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

When I ran the RelVals I found an error in the following workflows:
140.53 step2

runTheMatrix-results/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/step2_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI.log

5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

140.56 step2

runTheMatrix-results/140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log

4.53 step3

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step3_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

158.0 step2

runTheMatrix-results/158.0_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO/step2_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO.log

136.793 step3

runTheMatrix-results/136.793_RunDoubleEG2017C+RunDoubleEG2017C+HLTDR2_2017+RECODR2_2017reHLT_skimDoubleEG_Prompt+HARVEST2017/step3_RunDoubleEG2017C+RunDoubleEG2017C+HLTDR2_2017+RECODR2_2017reHLT_skimDoubleEG_Prompt+HARVEST2017.log

136.874 step3

runTheMatrix-results/136.874_RunEGamma2018C+RunEGamma2018C+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM/step3_RunEGamma2018C+RunEGamma2018C+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM+NANOUP15/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM+NANOUP15.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

10042.0 step3

runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017/step3_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017.log

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

11634.0 step2

runTheMatrix-results/11634.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021/step2_TTbar_14TeV+TTbar_14TeV_TuneCP5_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021.log

12434.0 step2

runTheMatrix-results/12434.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023/step2_TTbar_14TeV+TTbar_14TeV_TuneCP5_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

10824.0 step3

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15_PU25.log

10224.0 step3

runTheMatrix-results/10224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU+NanoFull_2017PU/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU+NanoFull_2017PU.log

20034.0 step3

runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D35+RecoFullGlobal_2026D35+HARVESTFullGlobal_2026D35/step3_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D35+RecoFullGlobal_2026D35+HARVESTFullGlobal_2026D35.log

20434.0 step3

runTheMatrix-results/20434.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D41+RecoFullGlobal_2026D41+HARVESTFullGlobal_2026D41/step3_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D41+RecoFullGlobal_2026D41+HARVESTFullGlobal_2026D41.log

21234.0 step3

runTheMatrix-results/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D44_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D44+RecoFullGlobal_2026D44+HARVESTFullGlobal_2026D44/step3_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D44_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D44+RecoFullGlobal_2026D44+HARVESTFullGlobal_2026D44.log

23234.0 step3

runTheMatrix-results/23234.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D49_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D49+RecoFullGlobal_2026D49+HARVESTFullGlobal_2026D49/step3_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D49_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D49+RecoFullGlobal_2026D49+HARVESTFullGlobal_2026D49.log

250202.181 step4

runTheMatrix-results/250202.181_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25/step4_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25.log

AddOn:

I found errors in the following addon tests:

cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - time: date Tue Mar 10 23:26:50 2020-date Tue Mar 10 23:24:22 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake2.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:27:36 2020-date Tue Mar 10 23:24:27 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:Fake2,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake2 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --processName=HLTRECO --filein file:RelVal_Raw_Fake2_DATA.root --fileout file:RelVal_Raw_Fake2_DATA_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:27:36 2020-date Tue Mar 10 23:24:27 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake1.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:31:19 2020-date Tue Mar 10 23:24:28 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_MC.root --fileout file:RelVal_Raw_Fake1_MC_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:31:19 2020-date Tue Mar 10 23:24:28 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_2018.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:36:28 2020-date Tue Mar 10 23:24:31 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:2018,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_2018 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_2018_MC.root --fileout file:RelVal_Raw_2018_MC_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:36:28 2020-date Tue Mar 10 23:24:31 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_HIon.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:30:07 2020-date Tue Mar 10 23:24:36 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018_pp_on_AA --processName=HLTRECO --filein file:RelVal_Raw_HIon_DATA.root --fileout file:RelVal_Raw_HIon_DATA_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:30:07 2020-date Tue Mar 10 23:24:36 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_GRun.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:35:25 2020-date Tue Mar 10 23:24:39 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run3 --processName=HLTRECO --filein file:RelVal_Raw_GRun_MC.root --fileout file:RelVal_Raw_GRun_MC_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:35:25 2020-date Tue Mar 10 23:24:39 2020 s - exit: 16896
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc_l1stage1 --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns : FAILED - time: date Tue Mar 10 23:27:33 2020-date Tue Mar 10 23:24:44 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_2018.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:33:47 2020-date Tue Mar 10 23:24:49 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:2018,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_2018 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_2018_DATA.root --fileout file:RelVal_Raw_2018_DATA_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:33:47 2020-date Tue Mar 10 23:24:49 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PRef.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:31:21 2020-date Tue Mar 10 23:24:50 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run3 --processName=HLTRECO --filein file:RelVal_Raw_PRef_MC.root --fileout file:RelVal_Raw_PRef_MC_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:31:21 2020-date Tue Mar 10 23:24:50 2020 s - exit: 16896
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 : FAILED - time: date Tue Mar 10 23:27:26 2020-date Tue Mar 10 23:24:53 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PIon.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:27:40 2020-date Tue Mar 10 23:24:57 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:27:40 2020-date Tue Mar 10 23:24:57 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:31:13 2020-date Tue Mar 10 23:24:59 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_MC.root --fileout file:RelVal_Raw_Fake_MC_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:31:13 2020-date Tue Mar 10 23:24:59 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_HIon.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:32:53 2020-date Tue Mar 10 23:25:03 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018_pp_on_AA --processName=HLTRECO --filein file:RelVal_Raw_HIon_MC.root --fileout file:RelVal_Raw_HIon_MC_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:32:53 2020-date Tue Mar 10 23:25:03 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake2.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:31:20 2020-date Tue Mar 10 23:25:04 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:Fake2,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_Fake2 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --processName=HLTRECO --filein file:RelVal_Raw_Fake2_MC.root --fileout file:RelVal_Raw_Fake2_MC_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:31:20 2020-date Tue Mar 10 23:25:04 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PRef.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:28:31 2020-date Tue Mar 10 23:25:07 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:28:31 2020-date Tue Mar 10 23:25:07 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:27:59 2020-date Tue Mar 10 23:25:12 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:Fake,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run1_data_Fake --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --processName=HLTRECO --filein file:RelVal_Raw_Fake_DATA.root --fileout file:RelVal_Raw_Fake_DATA_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:27:59 2020-date Tue Mar 10 23:25:12 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_GRun.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:34:05 2020-date Tue Mar 10 23:25:14 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2018 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:34:05 2020-date Tue Mar 10 23:25:14 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake1.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:27:41 2020-date Tue Mar 10 23:25:16 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:Fake1,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake1 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --processName=HLTRECO --filein file:RelVal_Raw_Fake1_DATA.root --fileout file:RelVal_Raw_Fake1_DATA_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:27:41 2020-date Tue Mar 10 23:25:16 2020 s - exit: 16896
cmsRun /cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-09-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PIon.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Tue Mar 10 23:31:59 2020-date Tue Mar 10 23:25:20 2020 s - exit: 16896
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run3 --processName=HLTRECO --filein file:RelVal_Raw_PIon_MC.root --fileout file:RelVal_Raw_PIon_MC_HLT_RECO.root : FAILED - time: date Tue Mar 10 23:31:59 2020-date Tue Mar 10 23:25:20 2020 s - exit: 16896

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

#28902 @hqucms: MXNet-based implementation of the ParticleNet tagger

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-323b31/5117/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-323b31/5117/git-merge-result

cmsbuild · 2020-03-10T22:40:08Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

cmsbuild · 2020-03-10T22:53:08Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmssw#29172
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5121/console Started: 2020/03/10 23:54

cmsbuild · 2020-03-11T00:03:29Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b9ddee/5115/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2680577
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2680257
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

cmsbuild · 2020-03-19T17:16:20Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1384f4/5272/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 10 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2692437
DQMHistoTests: Total failures: 37
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2692081
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

cmsbuild · 2020-03-19T19:27:24Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmssw#29172
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5279/console Started: 2020/03/19 20:28

cmsbuild · 2020-03-19T21:16:11Z

+1
Tested at: 9f5e916
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-323b31/5279/summary.html
CMSSW: CMSSW_11_1_X_2020-03-19-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-03-19T21:16:15Z

Comparison job queued.

cmsbuild · 2020-03-19T23:18:41Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-323b31/5279/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2692493
DQMHistoTests: Total failures: 29
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2692145
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

cmsbuild · 2020-04-07T21:13:18Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmssw#29172
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5578/console Started: 2020/04/07 23:15

cmsbuild · 2020-04-07T23:03:47Z

+1
Tested at: 9f5e916
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-323b31/5578/summary.html
CMSSW: CMSSW_11_1_X_2020-04-07-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-04-07T23:03:51Z

Comparison job queued.

cmsbuild · 2020-04-08T00:32:46Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-323b31/5578/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 41 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2692110
DQMHistoTests: Total failures: 46
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2691745
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

smuzaffar · 2020-06-19T09:08:24Z

@hqucms , should we close this PR ( cms-sw/cmssw#29172 was closed too).

hqucms · 2020-06-19T14:27:13Z

@smuzaffar Yes this is no longer needed -- let me close it.

Convert TF models to remove training nodes.

9f5e916

https://github.com/hqucms/keras_to_tensorflow/blob/master/keras_to_tensorflow.py

cmsbuild added comparison-pending externals-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Mar 10, 2020

hqucms mentioned this pull request Mar 10, 2020

Restore TF and MXNet-based inference for DeepJet, DeepDoubleX and DeepAK8 cms-sw/cmssw#29172

Closed

cmsbuild added tests-started and removed tests-pending labels Mar 10, 2020

cmsbuild added the requires-external label Mar 10, 2020

cmsbuild added tests-approved and removed tests-started labels Mar 10, 2020

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-approved labels Mar 10, 2020

cmsbuild added tests-started and removed tests-rejected labels Mar 10, 2020

cmsbuild removed the comparison-notrun label Mar 11, 2020

cmsbuild added comparison-available and removed comparison-pending labels Mar 19, 2020

cmsbuild added tests-started and removed tests-approved labels Mar 19, 2020

cmsbuild added comparison-pending tests-approved and removed comparison-available tests-started labels Mar 19, 2020

cmsbuild added comparison-available and removed comparison-pending labels Mar 19, 2020

cmsbuild added tests-started and removed tests-approved labels Apr 7, 2020

cmsbuild added comparison-pending tests-approved and removed comparison-available tests-started labels Apr 7, 2020

cmsbuild added comparison-available and removed comparison-pending labels Apr 8, 2020

hqucms closed this Jun 19, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Technical update to the DeepJet and DeepDoubleX TF models #27

Technical update to the DeepJet and DeepDoubleX TF models #27

hqucms commented Mar 10, 2020 •

edited

Loading

cmsbuild commented Mar 10, 2020

slava77 commented Mar 10, 2020

cmsbuild commented Mar 10, 2020 •

edited

Loading

cmsbuild commented Mar 10, 2020 •

edited

Loading

cmsbuild commented Mar 10, 2020

cmsbuild commented Mar 10, 2020

cmsbuild commented Mar 10, 2020

cmsbuild commented Mar 10, 2020

cmsbuild commented Mar 10, 2020 •

edited

Loading

cmsbuild commented Mar 11, 2020

cmsbuild commented Mar 19, 2020

cmsbuild commented Mar 19, 2020 •

edited

Loading

cmsbuild commented Mar 19, 2020

cmsbuild commented Mar 19, 2020

cmsbuild commented Mar 19, 2020

cmsbuild commented Apr 7, 2020 •

edited

Loading

cmsbuild commented Apr 7, 2020

cmsbuild commented Apr 7, 2020

cmsbuild commented Apr 8, 2020

smuzaffar commented Jun 19, 2020

hqucms commented Jun 19, 2020

Technical update to the DeepJet and DeepDoubleX TF models #27

Technical update to the DeepJet and DeepDoubleX TF models #27

Conversation

hqucms commented Mar 10, 2020 • edited Loading

cmsbuild commented Mar 10, 2020

slava77 commented Mar 10, 2020

cmsbuild commented Mar 10, 2020 • edited Loading

cmsbuild commented Mar 10, 2020 • edited Loading

cmsbuild commented Mar 10, 2020

cmsbuild commented Mar 10, 2020

cmsbuild commented Mar 10, 2020

cmsbuild commented Mar 10, 2020

cmsbuild commented Mar 10, 2020 • edited Loading

cmsbuild commented Mar 11, 2020

cmsbuild commented Mar 19, 2020

cmsbuild commented Mar 19, 2020 • edited Loading

cmsbuild commented Mar 19, 2020

cmsbuild commented Mar 19, 2020

cmsbuild commented Mar 19, 2020

cmsbuild commented Apr 7, 2020 • edited Loading

cmsbuild commented Apr 7, 2020

cmsbuild commented Apr 7, 2020

cmsbuild commented Apr 8, 2020

smuzaffar commented Jun 19, 2020

hqucms commented Jun 19, 2020

hqucms commented Mar 10, 2020 •

edited

Loading

cmsbuild commented Mar 10, 2020 •

edited

Loading

cmsbuild commented Mar 10, 2020 •

edited

Loading

cmsbuild commented Mar 10, 2020 •

edited

Loading

cmsbuild commented Mar 19, 2020 •

edited

Loading

cmsbuild commented Apr 7, 2020 •

edited

Loading