Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make WriteBuffer delete all its temp files in /dev/shm #40652

Merged
merged 1 commit into from
Feb 1, 2023

Conversation

wddgit
Copy link
Contributor

@wddgit wddgit commented Jan 31, 2023

PR description:

Make WriteBuffer delete all its temp files in /dev/shm.

Each time the unit tests run the bug fixed here leaves behind 4 files with names similar to the following.

-rw-r--r--. 1 wdd    zh 1842 Jan 31 20:44 testProd23885_2buffer1
-rw-r--r--. 1 wdd    zh 1842 Jan 31 20:44 testProd23885_3buffer1
-rw-r--r--. 1 wdd    zh 1842 Jan 31 20:44 testProd23885_1buffer1
-rw-r--r--. 1 wdd    zh 5842 Jan 31 20:44 testProd23885_0buffer1

The 5 digit number is the PID. There is a possibility of a name clash if a later unit test both has the same PID AND is run by a different user. I noticed this while testing an unrelated PR and probably unluckily hit such a name clash.

This is related to #40484. Note that on cmsdev32 there are a few thousand old files in /dev/shm related to this problem cluttering up that directory. Some are there from problems fixed in #40484 and some from problems fixed by this PR. There could be some on other machines as well. I'm not sure how the machines are configured, but I would guess these will cleaned up next time each machine is rebooted. Until then, there is a possibility of name clashes in these unit tests (probably rarely). We believe the IBs avoid the problem because the code does try to delete the clashing file if it can. That does not work if a different user owns the file though, so for users running the unit tests outside the IBs there is a potential for problems.

PR validation:

Existing unit tests pass. I manually ran them and manually examined the contents of /dev/shm and new files are not left there anymore when the unit tests run. They appear to be getting cleaned up when the test ends as intended.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40652/33974

  • This PR adds an extra 12KB to repository

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @wddgit (W. David Dagenhart) for master.

It involves the following packages:

  • FWCore/SharedMemory (core)

@cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please review it and eventually sign? Thanks.
@makortel, @missirol this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@wddgit
Copy link
Contributor Author

wddgit commented Jan 31, 2023

please test

@Dr15Jones
Copy link
Contributor

Looks good. Thanks!

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 1, 2023

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-540120/30289/summary.html
COMMIT: c30c771
CMSSW: CMSSW_13_0_X_2023-01-31-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40652/30289/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 6 lines to the logs
  • Reco comparison results: 1 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3555495
  • DQMHistoTests: Total failures: 3
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3555470
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 211 log files, 162 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

@makortel
Copy link
Contributor

makortel commented Feb 1, 2023

+1

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 1, 2023

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

perrotta commented Feb 1, 2023

+1

@cmsbuild cmsbuild merged commit e4e860f into cms-sw:master Feb 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants