Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial data files #1

Merged
merged 1 commit into from
Jun 4, 2023
Merged

Conversation

wddgit
Copy link
Contributor

@wddgit wddgit commented May 31, 2023

Add initial data files that will be used in a unit test of the Run 3 Scouting data formats. These will be read by future releases to verify backward compatibility.

Note the real test for this is in the unit test added to CMSSW. I will reference this PR from that PR (it will be submitted soon). After they are both approved, it would probably be better for this data PR to be merged first to avoid potential problems with the data files not being included in the CMSSW IB used for IB or PR tests. Then maybe wait a couple days to merge the CMSSW PR.

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @wddgit (W. David Dagenhart) for branch main.

@smuzaffar, @Dr15Jones, @makortel, @aandvalenzuela, @iarspider, @cmsbuild can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here

@wddgit
Copy link
Contributor Author

wddgit commented May 31, 2023

please test

(The PR test for cms-sw/cmssw#41834 is really the one that tests this PR, the tests started by this comment don't mean much. I'm just running them to get the box checked...)

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 1, 2023

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cdcf12/32911/summary.html
COMMIT: 831a709
CMSSW: CMSSW_13_2_X_2023-05-31-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-data/DataFormats-Scouting/1/32911/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 10 lines from the logs
  • Reco comparison results: 51 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3221457
  • DQMHistoTests: Total failures: 2235
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3219200
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 207 log files, 159 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea in cms-sw/framework-team#530 was to have the class versions encoded in the file name. That would help in the case a scouting class gets updated twice (with different PRs) between two adjacent (pre-)releases to make sure the correct file is kept and used in the tests (if the "latest file" has not been used in production, it would be deleted). In this case the file name would be then test3Run3Scouting_v3_v5_v3_v4_v5_v3_v5_v3_v3_CMSSW_12_4_0.root (assuming I extracted the class versions in their alphabetical order correctly). I admit managing the file versions is a bit cumbersome, but I hope it would still be beneficial in the long run. What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll change the filenames.

It seemed kind of long when I was first looking at including 8 version numbers. It helps if we assume alphabetical order. Originally, I was thinking of how to identify the class associated with each version also and it was even longer. Currently I am making the data files after the release is already finalized. At least for these, the release number is associated with an exact set of versions. But if we want consistency going forward and there might be multiple versions in master while preparing a specific release, then it makes sense to include the version numbers.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seemed kind of long when I was first looking at including 8 version numbers. It helps if we assume alphabetical order. Originally, I was thinking of how to identify the class associated with each version also and it was even longer.

The alphabetical order was the least bad compromise I could come up with (between clarity and compactness). Could you include the convention in the README.md of the CMSSW PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed this comment, but I think I explained the convention in the README already (the one I just pushed).

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 1, 2023

Pull request #1 was updated.

@wddgit
Copy link
Contributor Author

wddgit commented Jun 1, 2023

please test

files renamed, content unchanged

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 2, 2023

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cdcf12/32942/summary.html
COMMIT: 1d52c59
CMSSW: CMSSW_13_2_X_2023-06-01-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-data/DataFormats-Scouting/1/32942/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 11 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 224 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3186957
  • DQMHistoTests: Total failures: 2228
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3184707
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 168.456 KiB( 47 files compared)
  • DQMHistoSizes: changed ( 23234.0,... ): 28.076 KiB HGCAL/HGCalValidator
  • Checked 207 log files, 159 edm output root files, 48 DQM output files
  • TriggerResults: found differences in 1 / 46 workflows

@perrotta
Copy link

perrotta commented Jun 4, 2023

+1

@perrotta
Copy link

perrotta commented Jun 4, 2023

merge

@cmsbuild cmsbuild merged commit 7efb1b5 into cms-data:main Jun 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants