TensorFlow-based DeepDoubleB tagger integration

[jmduarte]

This adds the DeepDoubleB TensorFlow based model which is based on a lot of the changes brought in from the DeepFlavour pull request #19893. The latest BTV meeting presentation provides more information about the tagger.

We've checked the compatibility between CMSSW-based discriminators computed using the producers in this PR with the output from the training framework (DeepJet) on a ttbar RelVal sample:

MINIAOD vs reference training framework:
https://cernbox.cern.ch/index.php/s/f8bJcJcS8hfU3WF

RECO vs reference training framework:
https://cernbox.cern.ch/index.php/s/hEqbrrMkz5xBSHs

The level of discrepancy is similar to what was observed for the DeepFlavour model, as we also train using MINIAOD inputs.

We also checked that our changes don't affect the existing DeepFlavour model:
https://cernbox.cern.ch/index.php/s/KHs8RKuE0qzREq4

@cvernier @ferencek

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-23033/4432

[cmsbuild]

A new Pull Request was created by @jmduarte (Javier Duarte) for master.

It involves the following packages:

DataFormats/BTauReco
PhysicsTools/PatAlgos
RecoBTag/DeepFlavour

@perrotta, @monttj, @cmsbuild, @slava77, @gpetruc, @arizzi can you please review it and eventually sign? Thanks.
@TaiSakuma, @gouskos, @imarches, @ahinzmann, @rappoccio, @gpetruc, @mmarionncern, @rovere, @jdamgov, @jdolen, @nhanvtran, @JyothsnaKomaragiri, @gkasieczka, @schoef, @ferencek, @mverzett, @mariadalfonso, @pvmulder, @acaudron, @seemasharmafnal this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[perrotta]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/27602/console Started: 2018/04/22 22:51

[cmsbuild]

+1
Tested at: 03479b2
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23033/27602/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23033/27602/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 29
• DQMHistoTests: Total histograms compared: 2492830
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2492653
• DQMHistoTests: Total skipped: 176
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 28 files compared)
• Checked 119 log files, 9 edm output root files, 29 DQM output files

[slava77]

@jmduarte
is the new tagger running already with this PR?
I think that it should be enabled if it is planned to be a part of miniAOD (or RECO)

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-23033/4959

[cmsbuild]

Pull request #23033 was updated. @perrotta, @monttj, @cmsbuild, @slava77, @gpetruc, @arizzi can you please check and sign again.

[slava77]

@cmsbuild please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28372/console Started: 2018/06/01 17:52

[cmsbuild]

+1
Tested at: a3fceea
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23033/28372/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23033/28372/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 61 differences found in the comparisons
• DQMHistoTests: Total files compared: 31
• DQMHistoTests: Total histograms compared: 2902471
• DQMHistoTests: Total failures: 17
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2902264
• DQMHistoTests: Total skipped: 190
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 30 files compared)
• Checked 128 log files, 14 edm output root files, 31 DQM output files

[slava77]

+1

for #23033 a3fceea

• technical update since the last reco signoff: dropping a superfluous implementation of a function in RecoBTag/TensorFlow/src/JetConverter.cc.
• jenkins tests pass:
  • the compilation warning about unused function is gone
  • all observations from the previous signoff TensorFlow-based DeepDoubleB tagger integration #23033 (comment) should apply

[fabiocos]

+1

[fabiocos]

merge

[jmduarte]

Hi @slava77, for submitting a backport to 94X as discussed previously, do we want to keep all the changes (including renameing RecoBTag/DeepFlavour -> RecoBTag/TensorFlow, etc.)?

One key difference we will implement is we will disable this in MINIAOD by default (so the code will be there for users to recompute in 94X, but it will not be produced in MINIAOD by default).

Thanks,
Javier

[slava77]

On 6/6/18 9:44 AM, Javier Duarte wrote: Hi @slava77 <https://github.com/slava77>, for submitting a backport to 94X as discussed previously, do we want to keep all the changes (including renameing RecoBTag/DeepFlavour -> RecoBTag/TensorFlow, etc.)? One key difference we will implement is we will disable this in MINIAOD by default (so the code will be there for users to recompute in 94X, but it will not be produced in MINIAOD by default). Thanks, Javier

Hi Javier, I find it most practical to have a full backport, including the file renaming. Will the double-B expected to be used in 94X only for analysis?