Calibration update 80 x #14198

dimattia · 2016-04-21T19:51:12Z

Greetings,

this PR contains the fix for the memory issue reported here:
https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/2291/2/1/1/1.html

Functional tests are ok. Profile plots will come tomorrow.

Cheers,
Alessandro

cmsbuild · 2016-04-21T19:51:58Z

A new Pull Request was created by @dimattia for CMSSW_8_1_X.

It involves the following packages:

CalibTracker/Configuration
CalibTracker/SiStripChannelGain
CalibTracker/SiStripCommon

@cmsbuild, @mmusich, @franzoni, @cerminar, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @mverzett, @gbenelli, @tocheng, @jlagram this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

mmusich · 2016-04-21T19:52:40Z

please test

cmsbuild · 2016-04-21T19:52:57Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/12567/console

cmsbuild · 2016-04-21T20:42:49Z

+1
Tested at: c203b05
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14198/12567/summary.html

mmusich · 2016-04-21T20:47:29Z

Hi @dimattia, running wf 1001.0 I get now:

------------------------------------------------------------------------------------------
Detailed ressource usage information FOR ALL ME
------------------------------------------------------------------------------------------
subsystem/folder                          histograms       bins        Empty bins     Empty/Total      bins per       MB         kB per
                                           (total)        (total)        (total)                      histogram     (total)    histogram  
------------------------------------------------------------------------------------------
AlCaReco
 -> DtCalibSynch                            2060        566490        303210         0.535           275          2.16          1.07
 -> SiStrip                                14878       9522460       9153064         0.961           640          36.4          2.51
 -> SiStripGains                               9     177290000         92561      0.000522      1.97e+07           676      7.69e+04
 -> SiStripGainsAfterAbortGap                  9     177290000         92561      0.000522      1.97e+07           676      7.69e+04
 -> TkAlMinBias                              268        124031         21397         0.173           463         0.532          2.03
    SUBSYSTEM TOTAL                        17224     364792981       9662793        0.0265      2.12e+04      1.43e+03          82.7
..........................................................................................

when it used to be:

------------------------------------------------------------------------------------------
Detailed ressource usage information FOR ALL ME
------------------------------------------------------------------------------------------
subsystem/folder             histograms       bins        Empty bins     Empty/Total      bins per       MB         kB per
                                 (total)     (total)        (total)                       histogram     (total)    histogram  
------------------------------------------------------------------------------------------
AlCaReco
 -> DtCalibSynch                   2060        566490        303210         0.535           275          2.16          1.07
 -> SiStrip                       14878       9522460       9153064         0.961           640          36.4          2.51
 -> SiStripGains                     18     354580000        185170      0.000522      1.97e+07      1.35e+03      7.69e+04
 -> SiStripGainsAfterAbortGap        18     354580000        185170      0.000522      1.97e+07      1.35e+03      7.69e+04
 -> TkAlMinBias                     265        123382         21308         0.173           466          0.53          2.05
    SUBSYSTEM TOTAL               17239     719372332       9847922        0.0137      4.17e+04      2.81e+03           163
..........................................................................................

cmsbuild · 2016-04-21T21:37:50Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14198/12567/summary.html

mmusich · 2016-04-22T08:27:35Z

@dimattia we need also a backport in 80X once finished with the checks

mmusich · 2016-04-22T10:17:41Z

some crude profiling of RSS vs time in WF 1001.0 shows we should be almost back to 803.

mmusich · 2016-04-22T11:12:59Z

+1

cmsbuild · 2016-04-22T11:13:25Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_1_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @Degano, @smuzaffar

davidlange6 · 2016-04-22T11:37:38Z

A ~1.5G increase isn't really "close" to 803... but definitely better than 804. will have a look in the next days

davidlange6 · 2016-04-25T13:33:20Z

@dimattia @mmusich
so I begin to realize that this is memory in a single threaded job...as its all an end-of-job blow up, this is a reminder as to why the dqm does not use something as simple as TFileService in production to store big histograms. We'll need to resolve that.

However, as a shorter term way to improving things, I have two questions regarding the big histograms:

why not TH1S? Are there either 30k entries in a bin?
was 2000 bins determined? Eg, Is the rms of the Landau you are fitting something like 3-5 bins?

davidlange6 · 2016-04-27T10:01:25Z

Hi @dimattia - any news on the questions from yesterday's ORP?

dimattia · 2016-04-27T14:26:08Z

Hi,

@davidlange6 I checked the maximum number of entries using histograms that contains the “gold plated” statistics (e.g. 2x10^9 clusters) for a calibration run. Indeed there are no more than 3000 entries per bin for the maximum split of statistics (e.g. split per APV readout chip). We may have troubles if we merge the statistics per readout link or per module (e.g. going coarser than a single APV chip). Anyway this usually happens when the statistics is not enough to calibrate a single APV, hence I believe the statistics per single bin will never exceed 32000.

I will implement the switch TH2F -> TH2S in the next update of this pull request. Likely within the next hour.

Cheers,
Alessandro.

On 27 Apr 2016, at 12:01, David Lange notifications@github.com wrote:

Hi @dimattia https://github.com/dimattia - any news on the questions from yesterday's ORP?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub #14198 (comment)

cmsbuild · 2016-04-27T16:46:53Z

Pull request #14198 was updated. @cmsbuild, @mmusich, @franzoni, @cerminar, @davidlange6 can you please check and sign again.

mmusich · 2016-04-27T16:47:56Z

please test

cmsbuild · 2016-04-27T16:48:28Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/12675/console

dimattia · 2016-04-27T17:16:20Z

Hi,

@davidlange6 the change TH2F - > TH2F has been implemented in the PR update. Sorry it took a bit more because I need to perform also the test for the gain validation.

Cheers,
Alessandro.

On 27 Apr 2016, at 18:48, cmsbuild notifications@github.com wrote:

The tests are being triggered in jenkins.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub #14198 (comment)

cmsbuild · 2016-04-27T18:21:28Z

+1
Tested at: 9e47c92
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14198/12675/summary.html

cmsbuild · 2016-04-27T19:26:05Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14198/12675/summary.html

mmusich · 2016-04-27T20:15:16Z

peak RSS measured in wf 1001.0 merging 9e47c92 is somewhere in 3.5 - 4 GB

mmusich · 2016-04-27T20:15:31Z

+1

cmsbuild · 2016-04-27T20:15:44Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_1_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @Degano, @smuzaffar

dimattia added 2 commits April 8, 2016 20:18

update of python configuration

54fa9b1

automatic booking of 0T histos removed to reduce the memory consumption

c203b05

cmsbuild added this to the Next CMSSW_8_1_X milestone Apr 21, 2016

cmsbuild added alca-pending pending-signatures tests-pending orp-pending comparison-pending labels Apr 21, 2016

cmsbuild added tests-started and removed tests-pending labels Apr 21, 2016

cmsbuild added tests-approved and removed tests-started labels Apr 21, 2016

cmsbuild added comparison-available and removed comparison-pending labels Apr 21, 2016

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Apr 22, 2016

cmsbuild added comparison-available and removed comparison-pending labels Apr 26, 2016

use TH2S instead of TH2F

9e47c92

cmsbuild added tests-pending comparison-pending and removed comparison-available tests-approved labels Apr 27, 2016

cmsbuild added tests-started and removed tests-pending labels Apr 27, 2016

cmsbuild added tests-approved and removed tests-started labels Apr 27, 2016

cmsbuild added comparison-available and removed comparison-pending labels Apr 27, 2016

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Apr 27, 2016

davidlange6 merged commit 8de78fe into cms-sw:CMSSW_8_1_X Apr 28, 2016

mmusich mentioned this pull request Nov 8, 2019

DQM: Remove more DQMStore features #28247

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Calibration update 80 x #14198

Calibration update 80 x #14198

dimattia commented Apr 21, 2016

cmsbuild commented Apr 21, 2016

mmusich commented Apr 21, 2016

cmsbuild commented Apr 21, 2016

cmsbuild commented Apr 21, 2016

mmusich commented Apr 21, 2016 •

edited

Loading

cmsbuild commented Apr 21, 2016

mmusich commented Apr 22, 2016

mmusich commented Apr 22, 2016

mmusich commented Apr 22, 2016

cmsbuild commented Apr 22, 2016

davidlange6 commented Apr 22, 2016

davidlange6 commented Apr 25, 2016

davidlange6 commented Apr 27, 2016

dimattia commented Apr 27, 2016

cmsbuild commented Apr 27, 2016

mmusich commented Apr 27, 2016

cmsbuild commented Apr 27, 2016 •

edited

Loading

dimattia commented Apr 27, 2016

cmsbuild commented Apr 27, 2016

cmsbuild commented Apr 27, 2016

mmusich commented Apr 27, 2016

mmusich commented Apr 27, 2016

cmsbuild commented Apr 27, 2016

Calibration update 80 x #14198

Calibration update 80 x #14198

Conversation

dimattia commented Apr 21, 2016

cmsbuild commented Apr 21, 2016

mmusich commented Apr 21, 2016

cmsbuild commented Apr 21, 2016

cmsbuild commented Apr 21, 2016

mmusich commented Apr 21, 2016 • edited Loading

cmsbuild commented Apr 21, 2016

mmusich commented Apr 22, 2016

mmusich commented Apr 22, 2016

mmusich commented Apr 22, 2016

cmsbuild commented Apr 22, 2016

davidlange6 commented Apr 22, 2016

davidlange6 commented Apr 25, 2016

davidlange6 commented Apr 27, 2016

dimattia commented Apr 27, 2016

cmsbuild commented Apr 27, 2016

mmusich commented Apr 27, 2016

cmsbuild commented Apr 27, 2016 • edited Loading

dimattia commented Apr 27, 2016

cmsbuild commented Apr 27, 2016

cmsbuild commented Apr 27, 2016

mmusich commented Apr 27, 2016

mmusich commented Apr 27, 2016

cmsbuild commented Apr 27, 2016

mmusich commented Apr 21, 2016 •

edited

Loading

cmsbuild commented Apr 27, 2016 •

edited

Loading