Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Phase 2 Integration GTTFileReader expansion and refactor #43968

Merged
merged 24 commits into from
Mar 4, 2024

Conversation

NJManganelli
Copy link
Contributor

PR description:

The aim of this PR is to extend the capability of GTT workflows by validating and fixing a minor discrepancy in the loading of vertices from APx/EMPv2 formatted buffer files (APx/Serenity GTT boards), and add a partial implementation for doing the same for tracks written to buffer files (at the stage of "GTT Input"). Along the way, the code is refactored to move common elements from GTTFile{Reader,Writer}.cc plugins to a GTTInterface.h header, as a code/maintainability improvement. Some compile-time constants are made python-configurable (as advised by Tom Williams).

The changes and capabilities added here are not invoked in a typical Phase 2 L1T workflow, and are currently only called for the L1Trigger/DemonstratorTools/test/gtt/createFirmwareInputFiles_cfg.py tool (which is used for creating - and now consuming - buffer files in APx/EMPv2 format for integration tests, and as stepping stones towards eventual Phase 2 GTT packer/unpacker)

Presented in https://indico.cern.ch/event/1352629/contributions/5695626/attachments/2763970/4814068/NickManganelli_GTT_01_12_2023.pdf
PR validation:

This PR passes
scram b code-format
scram b code-checks
scram b
runTheMatrix.py -l limited -i all --ibeos
GTT Firmware workflows: cmsRun createFirmwareInputFiles_cfg.py maxEvents=1008 format=W inputFiles=L1Trigger/DemonstratorTools/python/TT_TuneCP5_14TeV-powheg-pythia8_Phase2Spring23DIGIRECOMiniAOD-PU200_131X_mcRun4_realistic_v5-v1_GEN-SIM-DIGI-RAW-MINIAOD_APxBuffers_cff.py tracks=X vertices=Y readerformat=Z (X and Y in 'donotload' , 'load' , 'overwrite' ; W and Z in 'APx' or 'EMPv2'

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

This is the port from the cms-l1t-offline integration branch to the master branch
cms-l1t-offline#1188

cgsavard and others added 24 commits February 14, 2024 19:09
…hod in codecs and L1TrackVertexAssociationProducer, minus conflicts from rebase
…set to False for tracks from l1tGTTFileReader
…ional input capability and new parameters to support it.
…options to createFirmwareInputFiles_cfg.py
@epalencia
Copy link
Contributor

Please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-aac48a/37468/summary.html
COMMIT: 4c0b06b
CMSSW: CMSSW_14_1_X_2024-02-14-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43968/37468/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-aac48a/37468/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-aac48a/37468/git-merge-result

Comparison Summary

Summary:

@aloeliger
Copy link
Contributor

@NJManganelli Is this ready from your side? You mentioned an issue in your last message in cms-l1t-offline#1188. I think our review was done.

@NJManganelli
Copy link
Contributor Author

@NJManganelli Is this ready from your side? You mentioned an issue in your last message in cms-l1t-offline#1188. I think our review was done.

All good on my side. The issue mentioned prevents switching to the alternative BoardDataReader constructor as Tom suggested, but I don't have time to try debugging that issue for this PR. I will try to reproduce and submit an issue later on.

@subirsarkar
Copy link

@cmsbuild Please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-aac48a/37562/summary.html
COMMIT: 4c0b06b
CMSSW: CMSSW_14_1_X_2024-02-19-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43968/37562/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 4 lines from the logs
  • Reco comparison results: 40 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3248389
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3248367
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 200 log files, 161 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

@aloeliger
Copy link
Contributor

+l1

@subirsarkar
Copy link

+Upgrade

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 2, 2024

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @rappoccio, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

@antoniovilela
Copy link
Contributor

Number of commits grew quite a bit. If squashing some of them is strongly desired, please let me know

We are currently requiring squashing in the more obvious cases.

In general however, grouping commits is encouraged in order to simplify the history in cmssw.

@antoniovilela
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 9fd050f into cms-sw:master Mar 4, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants