Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Backport: fix Service access from ES modules (#30688) #514

Merged
merged 2 commits into from
Jul 17, 2020
Merged

Backport: fix Service access from ES modules (#30688) #514

merged 2 commits into from
Jul 17, 2020

Conversation

fwyzard
Copy link

@fwyzard fwyzard commented Jul 15, 2020

Backport cms-sw#30688.

It was found that when using multiple threads an ES module might not be able to access a Service.
This change properly propagates the Service access to any task that will run an ES module.

@Dr15Jones @fwyzard
Fix: Make Services available to ES Modules
fc92bcd 
Now the ServiceToken is used before calling the methods of an ES module.
@Dr15Jones @fwyzard
Check that Services are accessible from ESProducer
9197def 
If the Service system is not available, tests using this module will throw an exception.
@fwyzard
Copy link
Author

fwyzard commented Jul 15, 2020

Validation summary

Reference release CMSSW_11_2_0_pre2 at 92997c6
Development branch cms-patatrack/CMSSW_11_2_X_Patatrack at 11c3c16
Testing PRs:

Validation plots

/RelValTTbar_14TeV/CMSSW_11_1_0_pre8-PU_111X_mcRun3_2021_realistic_v4-v1/GEN-SIM-DIGI-RAW

  • tracking validation plots and summary for workflow 11634.5
  • tracking validation plots and summary for workflow 11634.501
  • tracking validation plots and summary for workflow 11634.502

/RelValZMM_14/CMSSW_11_1_0_pre8-111X_mcRun3_2021_realistic_v4-v1/GEN-SIM-DIGI-RAW

  • tracking validation plots and summary for workflow 11634.5
  • tracking validation plots and summary for workflow 11634.501
  • tracking validation plots and summary for workflow 11634.502

/RelValZEE_14/CMSSW_11_1_0_pre8-111X_mcRun3_2021_realistic_v4-v1/GEN-SIM-DIGI-RAW

  • tracking validation plots and summary for workflow 11634.5
  • tracking validation plots and summary for workflow 11634.501
  • tracking validation plots and summary for workflow 11634.502

Throughput plots

/EphemeralHLTPhysics1/Run2018D-v1/RAW run=323775 lumi=53

scan-136.885502.png
zoom-136.885502.png
scan-136.885522.png
zoom-136.885522.png

logs and nvprof/nvvp profiles

/RelValTTbar_14TeV/CMSSW_11_1_0_pre8-PU_111X_mcRun3_2021_realistic_v4-v1/GEN-SIM-DIGI-RAW

  • reference release, workflow 11634.5
  • development release, workflow 11634.5
  • development release, workflow 11634.501
  • development release, workflow 11634.502
    • ❌ step3.py: log
    • ❌ profile.py: log, profile and summary are missing, see the full log for more information
    • ⚠️ cuda-memcheck --tool initcheck did not run
    • ⚠️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all did not run
    • ⚠️ cuda-memcheck --tool synccheck did not run
  • development release, workflow 11634.511
  • development release, workflow 11634.512
    • ❌ step3.py: log
    • ❌ profile.py: log, profile and summary are missing, see the full log for more information
    • ⚠️ cuda-memcheck --tool initcheck did not run
    • ⚠️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all did not run
    • ⚠️ cuda-memcheck --tool synccheck did not run
  • development release, workflow 11634.521
  • development release, workflow 11634.522
    • ❌ step3.py: log
    • ❌ profile.py: log, profile and summary are missing, see the full log for more information
    • ⚠️ cuda-memcheck --tool initcheck did not run
    • ⚠️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all did not run
    • ⚠️ cuda-memcheck --tool synccheck did not run
  • development release, workflow 136.885502
  • development release, workflow 136.885512
  • development release, workflow 136.885522
  • testing release, workflow 11634.5
  • testing release, workflow 11634.501
  • testing release, workflow 11634.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 11634.511
  • testing release, workflow 11634.512
    • ❌ step3.py: log
    • ❌ profile.py: log, profile and summary are missing, see the full log for more information
    • ⚠️ cuda-memcheck --tool initcheck did not run
    • ⚠️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all did not run
    • ⚠️ cuda-memcheck --tool synccheck did not run
  • testing release, workflow 11634.521
  • testing release, workflow 11634.522
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ❌ cuda-memcheck --tool initcheck (report, log) found 955168 errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ❌ cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
  • testing release, workflow 136.885502
  • testing release, workflow 136.885512
  • testing release, workflow 136.885522

/RelValZMM_14/CMSSW_11_1_0_pre8-111X_mcRun3_2021_realistic_v4-v1/GEN-SIM-DIGI-RAW

  • reference release, workflow 11634.5
  • development release, workflow 11634.5
  • development release, workflow 11634.501
  • development release, workflow 11634.502
    • ❌ step3.py: log
    • ❌ profile.py: log, profile and summary are missing, see the full log for more information
    • ⚠️ cuda-memcheck --tool initcheck did not run
    • ⚠️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all did not run
    • ⚠️ cuda-memcheck --tool synccheck did not run
  • development release, workflow 11634.511
  • development release, workflow 11634.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ❌ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) found 0 errors
    • ❌ cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
  • development release, workflow 11634.521
  • development release, workflow 11634.522
    • ❌ step3.py: log
    • ❌ profile.py: log, profile and summary are missing, see the full log for more information
    • ⚠️ cuda-memcheck --tool initcheck did not run
    • ⚠️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all did not run
    • ⚠️ cuda-memcheck --tool synccheck did not run
  • development release, workflow 136.885502
  • development release, workflow 136.885512
  • development release, workflow 136.885522
  • testing release, workflow 11634.5
  • testing release, workflow 11634.501
  • testing release, workflow 11634.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 11634.511
  • testing release, workflow 11634.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ❌ cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
  • testing release, workflow 11634.521
  • testing release, workflow 11634.522
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ❌ cuda-memcheck --tool initcheck (report, log) found 47840 errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 136.885502
  • testing release, workflow 136.885512
  • testing release, workflow 136.885522

/RelValZEE_14/CMSSW_11_1_0_pre8-111X_mcRun3_2021_realistic_v4-v1/GEN-SIM-DIGI-RAW

  • reference release, workflow 11634.5
  • development release, workflow 11634.5
  • development release, workflow 11634.501
  • development release, workflow 11634.502
    • ❌ step3.py: log
    • ❌ profile.py: log, profile and summary are missing, see the full log for more information
    • ⚠️ cuda-memcheck --tool initcheck did not run
    • ⚠️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all did not run
    • ⚠️ cuda-memcheck --tool synccheck did not run
  • development release, workflow 11634.511
  • development release, workflow 11634.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ❌ cuda-memcheck --tool initcheck (report, log) found 0 errors
    • ❌ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) found 0 errors
    • ❌ cuda-memcheck --tool synccheck (report, log) found 0 errors
  • development release, workflow 11634.521
  • development release, workflow 11634.522
    • ✔️ step3.py: log
    • ❌ profile.py: log
    • ❌ cuda-memcheck --tool initcheck (report, log) found 26456 errors
    • ❌ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) found 0 errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • development release, workflow 136.885502
  • development release, workflow 136.885512
  • development release, workflow 136.885522
  • testing release, workflow 11634.5
  • testing release, workflow 11634.501
  • testing release, workflow 11634.502
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 11634.511
  • testing release, workflow 11634.512
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ❌ cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
  • testing release, workflow 11634.521
  • testing release, workflow 11634.522
    • ✔️ step3.py: log
    • ✔️ profile.py: log
    • ❌ cuda-memcheck --tool initcheck (report, log) found 26856 errors
    • ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
    • ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
  • testing release, workflow 136.885502
  • testing release, workflow 136.885512
  • testing release, workflow 136.885522

Logs

The full log is available at https://patatrack.web.cern.ch/patatrack/validation/pulls/ae7275e8d6eddce48219c576f1fba632a0ae8de5/log .

@fwyzard
Copy link
Author

fwyzard commented Jul 15, 2020

This does fix many errors - the remaining ones are hopefully fixed by #515.

@fwyzard fwyzard merged commit e2840ed into CMSSW_11_2_X_Patatrack Jul 17, 2020
@fwyzard fwyzard deleted the backport_30688 branch July 17, 2020 00:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@fwyzard @Dr15Jones