SiStrip Payload Inspector: refinements to `SiStripCondObjectRepresent` #37492

mmusich · 2022-04-07T15:11:15Z

PR description:

The purpose of this PR is to introduce several improvements and clean-ups into the SiStrip payload inspector classes and builds on top of #37265 and #37333:

d29103b adds a THStack plot for the gains broken down by subdetector.
b07b401 and 86d19b1 streamline the code in order to adhere to more modern pattern employed in other classes in the same package.
4be848b rationalizes the usage of TrackerMaps for the SiStripDetVOff object by using class templates
66339a9 and ab32dc4 perform the infrastructural changes to percolate the tag names into the SiStripCondObjectRepresent classes.
finally I profit of this PR to propose d21c579 as well, which instead is related to the Tracker Alignment inspectors (graphical improvements of the legends)

PR validation:

The touched classes were tested privately with e.g. the following commands:

getPayloadData.py --plugin pluginSiStripNoises_PayloadInspector --plot plot_SiStripNoiseCompareByPartition --tag SiStripNoise_v2_prompt --tagtwo SiStripNoise_v2_prompt --time_type Run --iovs '{"start_iov": "348878", "end_iov": "348878"}' --iovstwo '{"start_iov": "349927", "end_iov": "349927"}' --db Prod --test ;

etPayloadData.py --plugin pluginSiStripNoises_PayloadInspector --plot plot_SiStripNoiseDiffByPartition --tag SiStripNoise_v2_prompt --tagtwo SiStripNoise_v2_prompt --time_type Run --iovs '{"start_iov": "348878", "end_iov": "348878"}' --iovstwo '{"start_iov": "349927", "end_iov": "349927"}' --db Prod --test ;

getPayloadData.py --plugin pluginSiStripDetVOff_PayloadInspector --plot plot_SiStripDetVOff_IsModuleVOff_TrackerMap --tag SiStripDetVOff_v6_prompt --time_type Time --iovs '{"start_iov": "7083471724997514240", "end_iov": "7083471724997514240"}' --db Prod --test ;

if this PR is a backport please specify the original PR and why you need to backport that PR:

N/A

cmsbuild · 2022-04-07T15:20:03Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37492/29194

This PR adds an extra 72KB to repository

cmsbuild · 2022-04-07T15:20:29Z

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

CondCore/AlignmentPlugins (db)
CondCore/SiStripPlugins (db)

@cmsbuild, @ggovi, @tvami, @malbouis, @francescobrivio can you please review it and eventually sign? Thanks.
@mmusich, @VinInn this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

tvami · 2022-04-07T17:17:31Z

@cmsbuild , please test

cmsbuild · 2022-04-07T17:28:00Z

-1

Failed Tests: Build ClangBuild
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e5910/23742/summary.html
COMMIT: d21c579
CMSSW: CMSSW_12_4_X_2022-04-07-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37492/23742/install.sh to create a dev area with all the needed externals and cmssw changes.

Build

I found compilation error when building:

>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_4_X_2022-04-07-1100/src/CondTools/SiStrip/plugins/SiStripMiscalibrateHelper.cc
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_4_X_2022-04-07-1100/src/CondTools/SiStrip/plugins/SiStripNoisesBuilder.cc
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_4_X_2022-04-07-1100/src/CondTools/SiStrip/plugins/SiStripNoisesFromDBMiscalibrator.cc
In file included from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_4_X_2022-04-07-1100/src/CondCore/SiStripPlugins/interface/SiStripPayloadInspectorHelper.h:13,
                 from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_4_X_2022-04-07-1100/src/CondTools/SiStrip/plugins/SiStripMiscalibrateHelper.cc:5:
/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_X_2022-04-07-1100/src/CondCore/Utilities/interface/PayloadInspector.h:18:10: fatal error: pybind11/pybind11.h: No such file or directory
   18 | #include 
      |          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_4_X_2022-04-07-1100/src/CondTools/SiStrip/plugins/SiStripNoisesReader.cc
gmake: *** [tmp/slc7_amd64_gcc10/src/CondTools/SiStrip/plugins/CondToolsSiStripPlugins/SiStripMiscalibrateHelper.cc.o] Error 1

Clang Build

I found compilation warning while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' scram build -k -j 32 COMPILER='llvm compile'

See details on the summary page.

…ndObjectRepresent class

…ipCondObjectRepresent and update scripts

mmusich · 2022-04-07T18:43:47Z

-1

Failed Tests: Build ClangBuild

So, I removed the clang warnings reported here in the last push, but I am not able to reproduce the compilation failure in my checkout using:

scram b vclean && BUILD_LOG=yes USER_CXXFLAGS=-DUSE_CMS_DEPRECATED scram b -k -j 16

so I am not sure if triggering the test would result again in a failure.

tvami · 2022-04-07T18:47:03Z

@cmsbuild , please test

mmusich · 2022-04-08T06:16:43Z

@smuzaffar

I think it is better to update CondCore/SiStripPlugins/BuildFile.xml as the CondCore/Utilities deps is in CondCore/SiStripPlugins/interface/SiStripPayloadInspectorHelper.h

There is no such file in release (see: https://github.com/cms-sw/cmssw/tree/master/CondCore/SiStripPlugins), shall I add it?

smuzaffar · 2022-04-08T06:39:17Z

ah Ok, sorry I did not check that @mmusich . In that case what is the purpose of CondCore/SiStripPlugins/interface ? My worry was that as there are header files in CondCore/SiStripPlugins/interface which can be included by other packages (and then those packages also have to add the missing CondCore/Utilities deps in their BuildFiles) . Looks like CondTools/SiStrip is the only package using one of the headers file of CondCore/SiStripPlugins ... why not merge these 2 packages? otherwise

CondCore/SiStripPlugins/interface/SiStripCondObjectRepresent.h is only used by CondCore/SiStripPlugins/plugins so move it to CondCore/SiStripPlugins/plugins directory so that no one can include it
CondCore/SiStripPlugins/interface/SiStripPayloadInspectorHelper.h is used by CondTools/SiStrip so we need a CondCore/SiStripPlugins/BuildFile.xml in which please add deps on all packages included in CondCore/SiStripPlugins/interface/SiStripPayloadInspectorHelper.h (as CondCore/SiStripPlugins does not generate any share lib, so please do not include<export><lib name="1"/></export> section in BuildFile) and update CondTools/SiStrip/plugins/BuildFile.xml to dep on CondCore/SiStripPlugins.

I personally would prefer to merge the packages

…SiStrip package

cmsbuild · 2022-04-08T10:11:31Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37492/29208

This PR adds an extra 76KB to repository

cmsbuild · 2022-04-08T10:11:57Z

Pull request #37492 was updated. @cmsbuild, @ggovi, @tvami, @malbouis, @francescobrivio can you please check and sign again.

mmusich · 2022-04-08T10:13:25Z

I personally would prefer to merge the packages

as they are supposed to carry out separate tasks, I would rather not merge them, I went for the recipe you recommended at #37492 (comment)

mmusich · 2022-04-08T10:13:37Z

@cmsbuild, please test

cmsbuild · 2022-04-08T15:43:47Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9e5910/23765/summary.html
COMMIT: 3d2701d
CMSSW: CMSSW_12_4_X_2022-04-08-0600/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37492/23765/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3593039
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3593009
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 47 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 200 log files, 45 edm output root files, 48 DQM output files
TriggerResults: no differences found

tvami · 2022-04-08T17:59:40Z

+db

tests pass, diffs seen are known

cmsbuild · 2022-04-08T18:00:07Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-04-11T09:26:45Z

+1

mmusich added 4 commits April 5, 2022 16:55

add gain break-down for partition plot

d29103b

get rid of redundant cond::payloadInspector namespace

b07b401

use new fill method and class templates

86d19b1

rationalize DetVOff TrackerMaps with class templates

4be848b

cmsbuild added this to the CMSSW_12_4_X milestone Apr 7, 2022

cmsbuild added code-checks-pending db-pending orp-pending pending-signatures tests-pending labels Apr 7, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 7, 2022

cmsbuild added tests-started and removed tests-pending labels Apr 7, 2022

cmsbuild added tests-rejected and removed tests-started labels Apr 7, 2022

mmusich added 3 commits April 7, 2022 20:34

infrastructural changes to percolate the tag name in to the SiStripCo…

028cbc2

…ndObjectRepresent class

introduce tag names on comparison / difference / ratio plots in SiStr…

483d578

…ipCondObjectRepresent and update scripts

improve Tracker Alignment Payload Inspector legends

82449e4

mmusich force-pushed the gainTHstack_102X branch from d21c579 to 82449e4 Compare April 7, 2022 18:41

cmsbuild added code-checks-pending tests-pending and removed tests-rejected code-checks-approved labels Apr 7, 2022

cmsbuild added the tests-approved label Apr 7, 2022

mmusich added 2 commits April 8, 2022 11:54

move SiStripCondObjectRepresent from interface to plugins directory

9110628

add BuildFile to CondCore/SiStripPlugins and use it in the CondTools/…

3d2701d

…SiStrip package

cmsbuild added code-checks-pending tests-pending and removed tests-approved code-checks-approved labels Apr 8, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 8, 2022

cmsbuild added tests-started and removed tests-pending labels Apr 8, 2022

cmsbuild added tests-approved and removed tests-started labels Apr 8, 2022

cmsbuild added db-approved fully-signed and removed db-pending pending-signatures labels Apr 8, 2022

cmsbuild added orp-approved and removed orp-pending labels Apr 11, 2022

cmsbuild merged commit 4632d5c into cms-sw:master Apr 11, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SiStrip Payload Inspector: refinements to `SiStripCondObjectRepresent` #37492

SiStrip Payload Inspector: refinements to `SiStripCondObjectRepresent` #37492

mmusich commented Apr 7, 2022

cmsbuild commented Apr 7, 2022

cmsbuild commented Apr 7, 2022

tvami commented Apr 7, 2022

cmsbuild commented Apr 7, 2022

mmusich commented Apr 7, 2022

tvami commented Apr 7, 2022

mmusich commented Apr 8, 2022

smuzaffar commented Apr 8, 2022 •

edited

Loading

cmsbuild commented Apr 8, 2022

cmsbuild commented Apr 8, 2022

mmusich commented Apr 8, 2022

mmusich commented Apr 8, 2022

cmsbuild commented Apr 8, 2022

tvami commented Apr 8, 2022

cmsbuild commented Apr 8, 2022

perrotta commented Apr 11, 2022

SiStrip Payload Inspector: refinements to SiStripCondObjectRepresent #37492

SiStrip Payload Inspector: refinements to SiStripCondObjectRepresent #37492

Conversation

mmusich commented Apr 7, 2022

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Apr 7, 2022

cmsbuild commented Apr 7, 2022

tvami commented Apr 7, 2022

cmsbuild commented Apr 7, 2022

Build

Clang Build

mmusich commented Apr 7, 2022

tvami commented Apr 7, 2022

mmusich commented Apr 8, 2022

smuzaffar commented Apr 8, 2022 • edited Loading

cmsbuild commented Apr 8, 2022

cmsbuild commented Apr 8, 2022

mmusich commented Apr 8, 2022

mmusich commented Apr 8, 2022

cmsbuild commented Apr 8, 2022

Comparison Summary

tvami commented Apr 8, 2022

cmsbuild commented Apr 8, 2022

perrotta commented Apr 11, 2022

SiStrip Payload Inspector: refinements to `SiStripCondObjectRepresent` #37492

SiStrip Payload Inspector: refinements to `SiStripCondObjectRepresent` #37492

smuzaffar commented Apr 8, 2022 •

edited

Loading