Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement ECAL rechits on GPU #462

Conversation

amassiro
Copy link

From uncalibrated rechit to rechit on GPU, in CMSSW_11_1_X

The PR includes validation code for rechit comparison, similar to what shown at ECAL DPG meeting.

Some to the files changed are just "c++ stylistic", it can be reversed (I needed to debug).

Similar to #457 but mergeable.

@fwyzard fwyzard changed the base branch from CMSSW_11_1_X_Patatrack to master May 14, 2020 16:09
@fwyzard fwyzard changed the base branch from master to CMSSW_11_1_X_Patatrack May 14, 2020 16:09
// Then only if the LS is different, update the laser correction
// The variation within a LS is not worth pursuing (<< 0.1% !!)
// and below the precision we can claim on the laser corrections (right?).
// This will save quite some time (also for the CPU version?)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do the corrections actually change within a lumisection ?
or rather, does the interpolation has a granularity smaller than the lumisection ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current correction is based on "time". Namely, we have two measurements, one every 40 minutes, and we perform a linear interpolation among them.
Based on the code I would say that the correction is really time dependent, time of the event.

float interpolatedLaserResponse = p_i / apdpnref + float(tt - t_i) * (p_f - p_i) / (apdpnref * float(t_f - t_i));

One possibility (just an idea) to speed up is to calculate the correction once every LS. From physics point of view, this should have ~ no effect (24 seconds step function vs linear interpolation that extend 40 minutes), but it can be tested with the current available data.
Computationally wise, this could save some time.

To be discussed with who wrote in first place the interpolation code to see if there is something subtle I am missing (special cases).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like a very good idea !
(I always assumed the corrections were LS-based and handled by the EventSetup... this shows how far one can be wrong with reasonable assumptions)

@amassiro
Copy link
Author

Validation plots attached.

In the following energy and chi2 for the rechits reconstructed in both cpu and gpu.

NB: there are some cases where gpu is not reconstructed, coming from recovered channels.
NB2: differences to chi2 are due to encoding of chi2 in cpu rechit format (float -> int encoding), while in gpu I left float for the time being.

I am investigating the few outliers.

ecal-energies
ecal-chi2

@fwyzard fwyzard changed the base branch from CMSSW_11_1_X_Patatrack to master May 27, 2020 10:19
@fwyzard fwyzard changed the base branch from master to CMSSW_11_1_X_Patatrack May 27, 2020 10:19
@fwyzard fwyzard changed the title Ecal rechit on GPU, 11_1_X (v2) Implement ECAL rechits on GPU May 31, 2020
Copy link

@makortel makortel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some remarks for future work, none of these are strictly required.

Copy link

@makortel makortel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two additional comments that need to be addressed before the GPU producer can be used with a SwitchProducer on a non-GPU machine.

@fwyzard

This comment has been minimized.

@fwyzard
Copy link

fwyzard commented Jun 4, 2020

Validation summary

Reference release CMSSW_11_1_0_pre8 at 442ae07
Development branch cms-patatrack/CMSSW_11_1_X_Patatrack at d58485e
Testing PRs:

Validation plots

/RelValTTbar_14TeV/CMSSW_11_0_0_patch1-PU_110X_mcRun3_2021_realistic_v6-v1/GEN-SIM-DIGI-RAW

  • tracking validation plots and summary for workflow 10824.5
  • tracking validation plots and summary for workflow 10824.501
  • tracking validation plots and summary for workflow 10824.502
  • tracking validation plots and summary for workflow 10824.511
  • tracking validation plots and summary for workflow 10824.512

/RelValZMM_14/CMSSW_11_0_0_patch1-110X_mcRun3_2021_realistic_v6-v1/GEN-SIM-DIGI-RAW

  • tracking validation plots and summary for workflow 10824.5
  • tracking validation plots and summary for workflow 10824.501
  • tracking validation plots and summary for workflow 10824.502
  • tracking validation plots and summary for workflow 10824.511
  • tracking validation plots and summary for workflow 10824.512

/RelValZEE_14/CMSSW_11_0_0_patch1-110X_mcRun3_2021_realistic_v6-v1/GEN-SIM-DIGI-RAW

  • tracking validation plots and summary for workflow 10824.5
  • tracking validation plots and summary for workflow 10824.501
  • tracking validation plots and summary for workflow 10824.502
  • tracking validation plots and summary for workflow 10824.511
  • tracking validation plots and summary for workflow 10824.512

Throughput plots

/EphemeralHLTPhysics1/Run2018D-v1/RAW run=323775 lumi=53

scan-136.885502.png
zoom-136.885502.png
scan-136.885512.png
zoom-136.885512.png

logs and nvprof/nvvp profiles

/RelValTTbar_14TeV/CMSSW_11_0_0_patch1-PU_110X_mcRun3_2021_realistic_v6-v1/GEN-SIM-DIGI-RAW

  • reference release, workflow 10824.5
  • development release, workflow 10824.5
  • development release, workflow 10824.501
  • development release, workflow 10824.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 10824.511
  • development release, workflow 10824.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 136.885502
  • development release, workflow 136.885512
  • testing release, workflow 10824.5
  • testing release, workflow 10824.501
  • testing release, workflow 10824.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 10824.511
  • testing release, workflow 10824.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 136.885502
  • testing release, workflow 136.885512

/RelValZMM_14/CMSSW_11_0_0_patch1-110X_mcRun3_2021_realistic_v6-v1/GEN-SIM-DIGI-RAW

  • reference release, workflow 10824.5
  • development release, workflow 10824.5
  • development release, workflow 10824.501
  • development release, workflow 10824.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 10824.511
  • development release, workflow 10824.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 136.885502
  • development release, workflow 136.885512
  • testing release, workflow 10824.5
  • testing release, workflow 10824.501
  • testing release, workflow 10824.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 10824.511
  • testing release, workflow 10824.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 136.885502
  • testing release, workflow 136.885512

/RelValZEE_14/CMSSW_11_0_0_patch1-110X_mcRun3_2021_realistic_v6-v1/GEN-SIM-DIGI-RAW

  • reference release, workflow 10824.5
  • development release, workflow 10824.5
  • development release, workflow 10824.501
  • development release, workflow 10824.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 10824.511
  • development release, workflow 10824.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 136.885502
  • development release, workflow 136.885512
  • testing release, workflow 10824.5
  • testing release, workflow 10824.501
  • testing release, workflow 10824.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 10824.511
  • testing release, workflow 10824.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 136.885502
  • testing release, workflow 136.885512

Logs

The full log is available at https://patatrack.web.cern.ch/patatrack/validation/pulls/5a1f5ea979963159dec080ee56225c8fd1f4d765/log .

@fwyzard fwyzard merged commit a41e061 into cms-patatrack:CMSSW_11_1_X_Patatrack Jun 4, 2020
fwyzard pushed a commit that referenced this pull request Oct 7, 2020
Include standalone executables for the validation of ECAL uncalibrated
and calibrated rechits.
fwyzard pushed a commit that referenced this pull request Oct 8, 2020
Include standalone executables for the validation of ECAL uncalibrated
and calibrated rechits.
fwyzard pushed a commit that referenced this pull request Oct 8, 2020
Include standalone executables for the validation of ECAL uncalibrated
and calibrated rechits.
fwyzard pushed a commit that referenced this pull request Nov 9, 2020
Include standalone executables for the validation of ECAL uncalibrated
and calibrated rechits.
fwyzard pushed a commit that referenced this pull request Nov 12, 2020
Include standalone executables for the validation of ECAL uncalibrated
and calibrated rechits.
fwyzard pushed a commit that referenced this pull request Nov 16, 2020
Include standalone executables for the validation of ECAL uncalibrated
and calibrated rechits.
fwyzard pushed a commit that referenced this pull request Dec 25, 2020
Include standalone executables for the validation of ECAL uncalibrated
and calibrated rechits.
fwyzard pushed a commit that referenced this pull request Dec 29, 2020
Include standalone executables for the validation of ECAL uncalibrated
and calibrated rechits.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ECAL ECAL-related developments
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants