Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Layer processing v2 #129

Closed
wants to merge 13 commits into from
Closed

Layer processing v2 #129

wants to merge 13 commits into from

Conversation

osschar
Copy link
Collaborator

@osschar osschar commented Jun 17, 2023

Changes in ModuleInfo / Hit / LayerOfHits

  1. Add half_length member to ModuleInfo, fill it in MkFitGeometryESProducer.cc
  2. Extend HitInfo with half-strip-length, add qbar (the other of r/z variables). Store fast-access arrays as std::vector.

Initially, the idea was that strip half-length from ModuleInfo would be filled into the fast-access structure HitInfo. Then I realized that for strips the same thing can be obtained as sqrt(12 * (exx + eyy)/2 (for endcaps) and `sqrt(12 * ezz)/2 (for barrel). This is now used while filling up the hits.

We still need to think which is better and what this means for pixel detectors (and, if the calculation of clusters "long" direction should be different there).

Changes in MkFinder::selectHitIndicies()

  1. When local bool NEW_SELECTION is set, propagate parameters (not errors) of the candidate to the hit position and re-calculate dphi and dq values at the hit position. Select int NEW_MAX_HIT (tried 4 and 6) hits with best dphi for further processing.

  2. Add additional ROOT-text-tree ntuples dumps (like used for the hit-window tuning):

    • per considered hit information;
    • per candidate in a layer information.
  3. This is to provide us with data for understanding of possible performance and tuning of the above algorithm.

Further explorations to be done

  1. Try simpler propagate-to-hit schemas (linear, quadratic/cubic Taylor).

  2. Study how to replace current windows. As a fist approximation, propagate to the beginning and the end of the current layer and use these points (+ some simple delta) to calculate phi and q variable ranges for iteration over bins in LayerOfHits.
    These beg/end points / momentum vectors could also be used as a basis for simpler propagation above.

  3. Study dq and dphi windows to apply to individual hits. For now the original (old) windows are still used. Especially relevant for q as we are going to limit the number of hits that pass the phi cuts anyway.
    Make N_max_hits iteration dependent (or use a number calculated from N_max_cands).

  4. Investigate stereo layers and usage of ModuleInfo position, direction vectors for calculating the track-to-hit distance (this could even replace phi). We might need additional fan-out angle parameter for endcap modules.

Plans for later / follow-up

  1. Multi-step layer (+ optionally the matching stereo layer) processing to properly pick-up overlap hits and to properly score candidates. Name 'thin-layers' was used for this.
    Requires some intermediate (simplified?) Candidate storage (as we might have to go beyon N-max-cands not to penalize tracks that only pick up hits at the back of the layer) and a local scoring function.

Side issues

  1. In initialStep low-pt cands (< 0.5 GeV) seem to be picking up way more bad hits, especially in the barrel -- i.e. true hit dphi does not have rank 1. or 2. as is the case almost everywhere else.
    There is some indication that pT of seeds is lower than pT of sim tracks. Slava warns that this might just be due to the energy loss in the 4 seed layers.
    Find the source, can it be offset somehow? Consider separate low-pT processing / finding parameters?

…iables), Store fast-access arrays as std::vector<HitInfo>.
* MkFitCMS/standalone/buildtestMPlex.cc
  When validation is on make sure seeds are restored after BH, STD, CE runs.

* MkFitCMS/standalone/mkFit.cc
  Rename run-all to run default -- make it run std, and ce.

* MkFitCore/src/MkBuilder.cc
  Missing MkFinder setup in backwardFitBH().

* MkFitCore/standalone/Geoms/CylCowWLids.cc
  Array of eta extents too short.

* MkFitCore/standalone/Makefile.config
  Consolidate CMSSW and other occurences of Ofast, msse2, std=c++17/1z.
  Cleanup unused -D defines / settings.

* MkFitCore/standalone/ConfigStandalone.h
  Remove unused simulation multiple scattering parameters.
Call this from both methods (beginHitRegistration() and suckInHits()).
* RecoTracker/MkFitCore/interface/Track.h
  Add function mcHitIDofFirstHit() to simplify dump-hit-window code.

* RecoTracker/MkFitCore/src/MkBuilder.cc
  Add missing MkFinder setup in dump-hit-window case.

* RecoTracker/MkFitCore/standalone/Makefile.config
  Clarify usage of Ofast (by Steve).

* RecoTracker/MkFitCMS/standalone/Makefile
  Use GNU make &: syntax (grouped targets) for dictionary targets.
…itIndices.

Includes a dedicated ROOT tree text dumpers to evaluate possible performance.

MiniPropagators.h/cc are to hold implementations of propagators of track
parameters only (no errors/covariances), with various levels of approximation
and eventually also supporting vectorization.
@trackreco trackreco deleted a comment from slava77 Jul 21, 2023
@leonardogiannini
Copy link

Comparison for default configuration (4+1 mkFit iterations)
http://uaf-10.t2.ucsd.edu/~legianni/plots_PR129

to add low pT and pixelLess iterations comparisons

@slava77
Copy link

slava77 commented Aug 4, 2023

Comparison for default configuration (4+1 mkFit iterations)
http://uaf-10.t2.ucsd.edu/~legianni/plots_PR129

detached quad and triplet have some inefficiency at low pt

image image

@leonardogiannini
Copy link

leonardogiannini commented Aug 7, 2023

I'm adding the plots where I compare pure CKF (blue) to mkFit for one iteration (red) to mkFIt + PR129 for one iteration

PixelLess-PR129-plots/
LowPtT-PR129-plots/
LowPtQ-PR129-plots/

I updated the links. there is still some discrepancy in the post selection tracks (due to cross cleaning?),
but now the seeds are identical up to the iteration of interest.

@slava77
Copy link

slava77 commented Aug 8, 2023

I'm adding the plots where I compare pure CKF (blue) to mkFit for one iteration (red) to mkFIt + PR129 for one iteration

PixelLess-PR129-plots/ LowPtT-PR129-plots/ LowPtQ-PR129-plots/

I updated the links. there is still some discrepancy in the post selection tracks (due to cross cleaning?), but now the seeds are identical up to the iteration of interest.

pixelLess is improving (left) while lowPtQuad (right) gets slightly worse
image image

The improvement for the pixelLess is expected. From debugging in the past maxhit of 16 was inefficient and this update narrows down to hit selection more appropriately.

The number of strip hits in lowPtQuad is up, perhaps the efficiency loss is actually a classification migration (the track is there but we pick up wrong hits).

- use VDT for fast transcendentals in MiniPropagators

- add scalar and element-wise arithemtic operators and transcendentals to Matriplex

- add preliminary bin-range detection to MkFinder::selectHitIndices()
…(and elsewhere).

Old dumpers in selectHitIndices still need to be removed - as functionality is moved over.

* MkFitCore/interface/Config.h
  Fix typo.

* MkFitCore/interface/MkBuilder.h
* MkFitCore/interface/Track.h
* MkFitCore/interface/TrackStructures.h
* MkFitCore/src/TrackStructures.cc
* MkFitCore/src/MkBuilder.cc
* MkFitCore/standalone/Event.cc
* MkFitCore/standalone/Event.h
  Pass seed-index in the current seed-vector along.
    In Event, provide a means of determining sim-track from hits of each
      seed track.

* MkFitCore/src/MkBuilder.cc
  Add handling of WSR_Failed.

* MkFitCore/src/Matrix.h
  Add short-int Matriplex typedefs

* MkFitCore/src/MiniPropagators.cc
* MkFitCore/src/MiniPropagators.h
  Fail-flag consistency, rename State dphi to dalpha.

* MkFitCore/src/MkBase.h
  Add radius() function.

* MkFitCore/src/MkFinder.cc
* MkFitCore/src/MkFinder.h
  Modularization of new/old hit-selection.
    Reimplemented bin-search for edge positions and determined global scaling
      of bin-search windows to reproduce current behavior.

* MkFitCore/standalone/Makefile
* MkFitCore/standalone/Makefile.config
  Add rules for building RntDumper stuff.

* MkFitCore/standalone/RntDumper/
  New sub-directory: Implementation of classes and structures for dumping ROOT
    TTrees or RNTuples.

* MkFitCMS/standalone/Makefile
  Add linking agains libMicRntDump.so

* MkFitCMS/standalone/MkStandaloneSeqs.cc
  Make it easier to find out why standalone tracks are not matched to sim
    tracks.

* MkFitCMS/standalone/Shell.cc
* MkFitCMS/standalone/buildtestMPlex.cc
  Setup Event current-seed-vector pointers as needed during event processing.

* MkFitCMS/standalone/mkFit.cc
  Finalize all RntDumpers on exit.
@leonardogiannini
Copy link

I'm adding here the MTV plots with the latest commit (red is the former version as of Sept 21 and black is the most recent), comparing the to the standard mkFit (blue) for the usual 4+1 iterations

http://uaf-10.t2.ucsd.edu/~legianni/plots_compare_layerProcv2_octoberVersion/

@leonardogiannini
Copy link

this is the MTV comparison after switching off the new layer processing. (code-format commit not included)
http://uaf-10.t2.ucsd.edu/~legianni/plots_compare_layerProcv2_PRVersion/

it looks like the PR is good for CMSSW

@srlantz
Copy link

srlantz commented Oct 30, 2023

If I'm not mistaken, the commits in this PR's layer-processing-v2 branch were squashed and rebased in a new branch, layer-processing-v2-rb1, which became the source for a PR to cmssw itself, cms-sw#43145 ...yes, this is confirmed by @osschar

template <typename T, idx_t D1, idx_t D2, idx_t N>
MPlex<T, D1, D2, N> sqr(const MPlex<T, D1, D2, N>& a) {
MPlex<T, D1, D2, N> t;
return t.sqrt(a);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty sure there's a typo here? Seems like it should be return t.sqr(a);

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you Steve, I added this change into prop-to-plane PR, cms-sw#43146, commit ab89b2f

@osschar
Copy link
Collaborator Author

osschar commented Nov 14, 2023

The CMSSW PR cms-sw#43145 has been merged and should be picked up by master resync ... this one is out of sync -- closing.

@osschar osschar closed this Nov 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants