Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MTD TDR Reconstruction + MTD Track Association BDT #28815

Merged
merged 28 commits into from
Feb 12, 2020

Conversation

pmeridian
Copy link
Contributor

@pmeridian pmeridian commented Jan 28, 2020

PR description:

Reopening of PR #28279: adds the tuning of the MTD track extender done for MTD TDR (using also a beam spot/vertex time constrain when finding new hits) and add a track quality MTD associator score based on a BDT classifier.

Most of the comments raised for PR #28279 are addressed (not addressing all the style comments made for TrackPIDinfo, as this will need to completely evolve once MTD reconstruction is fully integrated into the tracker)

The TrackPUIDMVA Producer and TOFPIDProducer are now stored into a new package in RecoMTD/TimingIDTools as suggested by @slava77 .

Also the BDT evaluation now used TMVAEvaluator with the GBRForest evaluation.

Future backport into 11_0_X can be imagined once this is fully integrated.

PR validation:

Validated this technically rerecoing 1000 events from RelValZMM_14TeV. To run a real test is required to make a repository to store the BDT xml file (now residing under RecoMTD/TimingIDTools/data). Please create a cms-data repository for this package.

@parbol @hatakeyamak @bendavid @simonepigazzini

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

Copy link
Contributor

@slava77 slava77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are mostly for posterity. No need to update in this PR.

#define VAR_ENUM(ENUM) ENUM,
#define VAR_STRING(STRING) #STRING,
#define MTDTRACKQUALITYMVA_VARS(MTDBDTVAR) \
MTDBDTVAR(pt) \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the arguments didn't need to have MTD in the prefix, but it's fine.

t_vtx,
t_vtx_err, //put vtx error by hand for the moment
false);
TrackTofPidInfo tof = computeTrackTofPidInfo(p.mag2(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const TrackTofPidInfo tof = or event auto tof =

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-487dad/4596/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 331 differences found in the comparisons
  • DQMHistoTests: Total files compared: 34
  • DQMHistoTests: Total histograms compared: 2694005
  • DQMHistoTests: Total failures: 725
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2692934
  • DQMHistoTests: Total skipped: 346
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
  • Checked 147 log files, 16 edm output root files, 34 DQM output files

@slava77
Copy link
Contributor

slava77 commented Feb 11, 2020

it looks like this PR is adding about 5% to the total RECO time in PU200 (in a production setup, writing AODSIM), which is coming from about 5-fold increase in CPU use by the trackExtenderWithMTD (from 5.7 s to ~28 s/ev). [this is still based on a partial set of events in the test that's still running]

@pmeridian
from the context or earlier responses I understood that you did not rerun this code on high PU and did not study the technical performance.
Please correct me if I'm wrong.

@kpedro88 please take a note for something to follow up.

@slava77
Copy link
Contributor

slava77 commented Feb 11, 2020

+1

for #28815 d4a01be

  • code changes are in line with the PR description and the follow up review
  • jenkins tests pass and comparisons with the baseline show differences in variables related to MTD variables
  • local tests show that this PR adds almost 6% to the total RECO time in the ttbar sample with PU200 in D49 setup in 11_1_0_pre2 (all MTD-related reco is back to ~30% of the total RECO time).
    • there is no significant change in the total AODSIM size: newly added floats ValueMaps are balanced with about 0.5% decrease in size of Tracks and TrackExtras from the trackExtenderWithMTD [BTW, trackExtra and hits from trackExtenderWithMTD are about 20% of AODSIM; something to be dropped in some not too distant future].
    • there is no visible change in the peak RSS use

@silviodonato
Copy link
Contributor

Kind reminder to @kpedro88 @santocch

@kpedro88
Copy link
Contributor

+upgrade

@slava77
Copy link
Contributor

slava77 commented Feb 12, 2020

some inclusive plots from the root files for the ttbar PU200 D49 setup, 200 events

At the track level, it looks like most of the population changes are in the ETL
all_sign1096VSorig_TTbar14TeV2026D49PU200localwf23434p21c_floatedmValueMap_trackExtenderWithMTD_pathLength_RECO_obj_values_

pf CH times are closer to 0
all_sign1096VSorig_TTbar14TeV2026D49PU200localwf23434p21c_max-250,min150,recoPFCandidates_particleFlow__RECO_obj_time_18

pf electrons show the most significant effect
all_sign1096VSorig_TTbar14TeV2026D49PU200localwf23434p21c_max-250,min150,recoPFCandidates_particleFlow__RECO_obj_time_25

the time error of the 4D PVs is better (with some migration to the default no-time-set value [displayed at 0])
all_sign1096VSorig_TTbar14TeV2026D49PU200PATlocalwf23434p21c_log10recoVertexs_offlineSlimmedPrimaryVertices4D__PAT_obj_tError

The time is closer to zero as well with significantly reduced tail on the positive side
all_sign1096VSorig_TTbar14TeV2026D49PU200PATlocalwf23434p21c_recoVertexs_offlineSlimmedPrimaryVertices4D__PAT_obj_t

x (and y) errors are a bit better, but there is also a small component that gets worse
all_sign1096VSorig_TTbar14TeV2026D49PU200PATlocalwf23434p21c_log10recoVertexs_offlineSlimmedPrimaryVertices4D__PAT_obj_xError

At a glance, this all looks like going roughly in the direction of improving the performance.

@cmsbuild cmsbuild mentioned this pull request Feb 12, 2020
@silviodonato
Copy link
Contributor

merge
@santocch

@cmsbuild cmsbuild merged commit 3453411 into cms-sw:master Feb 12, 2020
@smuzaffar
Copy link
Contributor

@silviodonato , this also needs cms-sw/cmsdist#5538 which is not merged yet. I am going to merge it now. You might want to restart 11.1.0.pre3 release to get latest cmsdist

@smuzaffar
Copy link
Contributor

@silviodonato , Ok, I have restarted the 11.1.0.pre3 release jobs , so every thing should be fine now

@santocch
Copy link

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.