-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement suggestions for CMSSW_14_1_X PR #45665 #137
Implement suggestions for CMSSW_14_1_X PR #45665 #137
Conversation
… HGCalROCConfig_t, HGCalECONDConfig_t, HGCalFedConfig_t structs
…ebugging cout or use MessageLogger; move includes; remove trailing '_' from function arguments
…(TODO: throw exception if not found?)
@yulunmiao & @hqucms, can you have a look at specifically the changes to the enumeration/constants, please? Following Matti's comments, I changed the enumerators enum ECONDFlag {
BITT_POS = 0,
BITM_POS = 1,
EBO_POS = 2,
EBO_MASK = 0b11,
HT_POS = 4,
HT_MASK = 0b11,
BITE_POS = 6,
BITS_POS = 7
}; to namespace ECONDFlag {
constexpr uint8_t BITT_POS = 0, BITM_POS = 1, EBO_POS = 2, EBO_MASK = 0b11, HT_POS = 4, HT_MASK = 0b11,
BITE_POS = 6, BITS_POS = 7;
} (Unfortunately, the CMSSW code formatting converts the line breakes after commas to a single wrapped line as above, which is a bit ugly. Alternatively, we can write each constant as Choices of data types for the others are as follows: namespace hgcal {
namespace econd {
namespace ToTStatus {
constexpr uint8_t ZeroSuppressed = 0x0, ...;
} // namespace ToTStatus
} // namespace econd
namespace backend {
namespace ECONDPacketStatus {
constexpr uint8_t Normal = 0x0, ...;
} // namespace ECONDPacketStatus
} // namespace backend
namespace ECOND_FRAME {
constexpr uint32_t HEADER_POS = 23, ...;
} // namespace ECOND_FRAME
namespace BACKEND_FRAME {
constexpr uint32_t CAPTUREBLOCK_RESERVED_MASK = 0x7f, ...;
} // namespace BACKEND_FRAME
namespace DIGI_FLAG {
constexpr uint16_t
FULL_READOUT = 0x0000, ...;
} // namespace DIGI_FLAG
} // namespace hgcal |
Hi @IzaakWN, thank you for making the changes, I think the changes look good to me |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All looks reasonable to me, thanks for implementing!
…sVersion+checksum
…ncludes & couts; pass reference to fill_SoA_column to avoid copy
…lRecHitHost; Remove deprecated HGCalFlaggedECONDInfo* & comments;
…-loop with std::accumulate
…r instead of shared_ptr; NULL → nullptr; initialize optional override; memcp -> std::copy
Tested with ssh lxplus9
source /cvmfs/cms.cern.ch/cmsset_default.sh
# install
export SCRAM_ARCH="el9_amd64_gcc12"
cmsrel CMSSW_14_1_0_pre4
cd CMSSW_14_1_0_pre4/src/
cmsenv
# checkout our development branch
git cms-merge-topic -u CMS-HGCAL:dev/CMSSW_14_1_X_PR_suggestions
# load external tools & data
git clone https://gitlab.cern.ch/hgcal-dpg/hgcal-comm.git HGCalCommissioning
git fetch origin CMSSW_14_1_X_PR_suggestions
git checkout CMSSW_14_1_X_PR_suggestions
# compile
scram b -j 10
# test that things are up and running (run unit tests)
scram b runtests
# if debug information is needed, instead of `scram b -j 10`, do
#USER_CXXFLAGS=-DEDM_ML_DEBUG scram b -j 10
cmsRun $CMSSW_BASE/src/HGCalCommissioning/SystemTestEventFilters/test/test_raw2reco.py charMode=0 dqmOnly=True prescale=100 maxEvents=5000 output=output_RunRUN.root inputFiles=/eos/cms/store/group/dpg_hgcal/tb_hgcal/2024/BeamTestAug/HgcalBeamtestAug2024/Relay1722965128/Run1722965131_Link1_File000000000[01].bin |
* Implement TH2Poly in DQM Services for HGCal DQM * Apply changes from code quality checks * Add a patch based on makortel's comments * enum -> namespace + constexpr uint{8,16,32}_t * Matti suggestion: remove '_t' postfix from HGCalFEDReadoutSequence_t, HGCalROCConfig_t, HGCalECONDConfig_t, HGCalFedConfig_t structs * Matti suggestions & code checks: remove redundant comments; silence debugging cout or use MessageLogger; move includes; remove trailing '_' from function arguments * Matti suggestion: reuse typecodeMap_.find() to avoid searching twice (TODO: throw exception if not found?) * Matti suggestion: use SET_PORTABLEHOSTCOLLECTION_READ_RULES; add ClassVersion+checksum * Matti suggestion: iRecord.getRecord<...>. -> iRecord. * Matti suggestion: iRecord.getRecord<...>. -> iRecord.; rm redundant includes & couts; pass reference to fill_SoA_column to avoid copy * Matti suggestions: use SET_PORTABLEHOSTCOLLECTION_READ_RULES for HGCalRecHitHost; Remove deprecated HGCalFlaggedECONDInfo* & comments; * Matti suggestion: throw exception if type code not found; replace for-loop with std::accumulate * Matti suggestion: remove TestObjects * Matti suggestions: remove config as member data and produce unique_ptr instead of shared_ptr; NULL → nullptr; initialize optional override; memcp -> std::copy * Matti suggestions: use file in path * removing soafiller and updating buildfile (#139) * address comments on HGCalRecHitCalibrationAlgorithms.dev.cc * irot as uint8_t instead of char * restore DQM without TH2Poly * apply code formats --------- Co-authored-by: ywkao <ywkao@hep1.phys.ntu.edu.tw> Co-authored-by: Pedro <psilva@cern.ch>
Code format HGCalMappingESSourceTester.cc Removing unused includes and delarations after PR comments Correction: missing :: for namespace Fix code style indentation record descriptions Re-introducing headers.h file needed for serialization classes remove data and change locations to cms-data folder moving to single record as all conds produced have the same IOV re-introducing 1 record per payload, addressing additional code comments moving to printf, removing unused product, and message logger code checks remove unused iomanip adding digi SoAs update collection names moving emulators to SimCalorimetry/HGCalSimAlgos, adapting paths moving digi soa filler to alpaka and adding extra tools some progress with soafiller updating fixing but in module es producer (assignment of capture block index instead of capture block) add dense index map for HGCalElectronicsId add getMaxDataSize & getMaxERxSize functions fix getDenseIndex bug; add typcode map to ModuleIndexer (to retrieve module dense index offset); use regexp for type code add getFedAndModuleIndex; rename getDenseIndex -> getIndexForModule(Data) for elecID add calibrations Alpaka ESProducer rename ES_HGCal*Parameter.cc -> HGCal*ParameterHostCollection.cc for clarity int -> uint32_t; uncomment debugging print out clean add HGCal configuration ESProducers + record + tester rename ES_HGCal*Parameter.cc -> HGCal*ParameterHostCollection.cc for clarity rename HGCal*ParameterHostCollection.cc -> HGCal*ParameterDeviceCollection.cc for clarity rename 'HGCal.*(Device|Host)Collection -> (Device|Host)' for readability; move calib SoAs to CondFormats/HGCalObjects/; move TestHGCalRecHitESProducers to test rename 'HGCal.*(Device|Host)Collection -> HGCal.*(Device|Host)' for readability add HGCalRecHit add DIGI2RECO calibration algorithms (see https://gitlab.cern.ch/hgcal-dpg/hgcal-comm/-/merge_requests/3)[C; remove detId/elecId from SoAs (should be redundant since common indexer); add HGCROCChannelDataFrame<HGCalDetId> to classes_def.xml Move HGCalRawDataDefinitions.h. [WIP] Refactor unpacker. parse arrays of gain-dependent calibration parameters in eRx-blocks add missing Noise attribute updating usage of Rcd fixing cfg files Convert ECON-D payload to 32b words. [Unpacker] rebase the unpacker on denseIdx and SoA digis Update unpacker code. SoA format of HGCal ECON-D header Unpack header information to HGCalECONDInfoSoA; Add in quality checks for ECON-D remove unused add typecode regexp; pass typecode to modIndexer_.processNewModule update ESProducers update record fix ESInputTags prefix filename with T_EventSetup_ add HGCalConfiguration & HGCalConfigurationESProducer Update and rename HGCalRecHitProducer.cc to HGCalRecHitProducers.cc fill HGCal config structs for unpacker (header marks, charMode) update calibration kernels read HGCal FED & ECON-D configuration from JSON Update unpacker to use HGCalConfiguration structure make manual overrides truly optional fix compilation issue due to initialization of const gain Change to LogDebug adding dense index info producer adding hgcalmapping customise fix misunderstanding in max data size expected from indexer Implement TH2Poly in DQM based on 14e856f Update HGCalConfigurationESProducer.cc update unpacker and HGCalConfiguration Add definitions of flags in digi SOA update the other part to most recent commit Add flags for passthrough ECON-Ds Debug: loop through every denseIdx Fix the number of channel in eRx;Add default flag for digi, representing digi not read in raw data fully read config parameters from JSONs for config SoA; add sanity checks; add HGCalMappingModuleIndexer::getNFED; rename getFedAndModuleIndex -> getIndexForFedAndModule; clean code; ... rename macro silence debugging message in ESProducers silence cout add HGCalMappingModuleIndexer::getMaxModuleSize(fedid) typo; SystemTestEventFilters/data -> Configuration/data bugfix in the cm Sum fixing dense index issue Update comment in HGCalMappingParameterSoA.h Some additional fixes (#132) - remove_if was missing a call to std::vector::erase to effectively remove unused entries - typecodemap was missing a reset with the final indices - usage of command line options was missing in module map tester - do not use moduleLUT in DenseIndexInfoESProducer!! it has the full dimensions before trimming to the actual number of ECON-Ds in a FED Add parsing function for econd flag; renaming HGCalECONDInfo to HGCalECONDPacketInfo Add irot to module mapper (#134) * - remove_if was missing a call to std::vector::erase to effectively remove unused entries - typecodemap was missing a reset with the final indices - usage of command line options was missing in module map tester * do not use moduleLUT in DenseIndexInfoESProducer!! * Adding optional irot to module mapper file - If existing it will be put in the module info SoA - othwerwise a default irot=0 (baseline is used) Suppress "unused variable" warning: remove CellIndexer in HGCalDigiSoaFiller.cc apply code formats (#135) Apply suggestions from code review Apply easy suggestions by Matti. (Looking into other comment & suggestions.) Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch> 'const json' to fix compilation issue for Matti's suggestion gain index should be taken as is, not subtracted by 1 make use of valid at rechit level / set rechit valids fill rec hitflags update kernel after closure Implement suggestions for CMSSW_14_1_X PR cms-sw#45665 (#137) * Implement TH2Poly in DQM Services for HGCal DQM * Apply changes from code quality checks * Add a patch based on makortel's comments * enum -> namespace + constexpr uint{8,16,32}_t * Matti suggestion: remove '_t' postfix from HGCalFEDReadoutSequence_t, HGCalROCConfig_t, HGCalECONDConfig_t, HGCalFedConfig_t structs * Matti suggestions & code checks: remove redundant comments; silence debugging cout or use MessageLogger; move includes; remove trailing '_' from function arguments * Matti suggestion: reuse typecodeMap_.find() to avoid searching twice (TODO: throw exception if not found?) * Matti suggestion: use SET_PORTABLEHOSTCOLLECTION_READ_RULES; add ClassVersion+checksum * Matti suggestion: iRecord.getRecord<...>. -> iRecord. * Matti suggestion: iRecord.getRecord<...>. -> iRecord.; rm redundant includes & couts; pass reference to fill_SoA_column to avoid copy * Matti suggestions: use SET_PORTABLEHOSTCOLLECTION_READ_RULES for HGCalRecHitHost; Remove deprecated HGCalFlaggedECONDInfo* & comments; * Matti suggestion: throw exception if type code not found; replace for-loop with std::accumulate * Matti suggestion: remove TestObjects * Matti suggestions: remove config as member data and produce unique_ptr instead of shared_ptr; NULL → nullptr; initialize optional override; memcp -> std::copy * Matti suggestions: use file in path * removing soafiller and updating buildfile (#139) * address comments on HGCalRecHitCalibrationAlgorithms.dev.cc * irot as uint8_t instead of char * restore DQM without TH2Poly * apply code formats --------- Co-authored-by: ywkao <ywkao@hep1.phys.ntu.edu.tw> Co-authored-by: Pedro <psilva@cern.ch> restore release base DQM classes (#140) move to chrono calls instead of struct inside namespace (#141) address comments (#142) update include (#143) remove ECON-D tests as they'll need to be re-written in a different form (#144) Declare HGCalRecHitCalibrationAlgorithms::calibrate as const method (#145) * remove ECON-D tests as they'll need to be re-written in a different form * enable const patching unpacker for padding of capture blocks within FED move check of padding to start of capture block Fix the throw, so the processing won't stop at the first event with error Update HGCalRecHitCalibrationAlgorithms.dev.cc CM sum -> 0.5*CM sum Change the throws to specific returns Optimize the producer Debugging the padding of eRx subpackets roll back (#149) apply code checks and remove dqm
Code format HGCalMappingESSourceTester.cc Removing unused includes and delarations after PR comments Correction: missing :: for namespace Fix code style indentation record descriptions Re-introducing headers.h file needed for serialization classes remove data and change locations to cms-data folder moving to single record as all conds produced have the same IOV re-introducing 1 record per payload, addressing additional code comments moving to printf, removing unused product, and message logger code checks remove unused iomanip adding digi SoAs update collection names moving emulators to SimCalorimetry/HGCalSimAlgos, adapting paths moving digi soa filler to alpaka and adding extra tools some progress with soafiller updating fixing but in module es producer (assignment of capture block index instead of capture block) add dense index map for HGCalElectronicsId add getMaxDataSize & getMaxERxSize functions fix getDenseIndex bug; add typcode map to ModuleIndexer (to retrieve module dense index offset); use regexp for type code add getFedAndModuleIndex; rename getDenseIndex -> getIndexForModule(Data) for elecID add calibrations Alpaka ESProducer rename ES_HGCal*Parameter.cc -> HGCal*ParameterHostCollection.cc for clarity int -> uint32_t; uncomment debugging print out clean add HGCal configuration ESProducers + record + tester rename ES_HGCal*Parameter.cc -> HGCal*ParameterHostCollection.cc for clarity rename HGCal*ParameterHostCollection.cc -> HGCal*ParameterDeviceCollection.cc for clarity rename 'HGCal.*(Device|Host)Collection -> (Device|Host)' for readability; move calib SoAs to CondFormats/HGCalObjects/; move TestHGCalRecHitESProducers to test rename 'HGCal.*(Device|Host)Collection -> HGCal.*(Device|Host)' for readability add HGCalRecHit add DIGI2RECO calibration algorithms (see https://gitlab.cern.ch/hgcal-dpg/hgcal-comm/-/merge_requests/3)[C; remove detId/elecId from SoAs (should be redundant since common indexer); add HGCROCChannelDataFrame<HGCalDetId> to classes_def.xml Move HGCalRawDataDefinitions.h. [WIP] Refactor unpacker. parse arrays of gain-dependent calibration parameters in eRx-blocks add missing Noise attribute updating usage of Rcd fixing cfg files Convert ECON-D payload to 32b words. [Unpacker] rebase the unpacker on denseIdx and SoA digis Update unpacker code. SoA format of HGCal ECON-D header Unpack header information to HGCalECONDInfoSoA; Add in quality checks for ECON-D remove unused add typecode regexp; pass typecode to modIndexer_.processNewModule update ESProducers update record fix ESInputTags prefix filename with T_EventSetup_ add HGCalConfiguration & HGCalConfigurationESProducer Update and rename HGCalRecHitProducer.cc to HGCalRecHitProducers.cc fill HGCal config structs for unpacker (header marks, charMode) update calibration kernels read HGCal FED & ECON-D configuration from JSON Update unpacker to use HGCalConfiguration structure make manual overrides truly optional fix compilation issue due to initialization of const gain Change to LogDebug adding dense index info producer adding hgcalmapping customise fix misunderstanding in max data size expected from indexer Implement TH2Poly in DQM based on 14e856f Update HGCalConfigurationESProducer.cc update unpacker and HGCalConfiguration Add definitions of flags in digi SOA update the other part to most recent commit Add flags for passthrough ECON-Ds Debug: loop through every denseIdx Fix the number of channel in eRx;Add default flag for digi, representing digi not read in raw data fully read config parameters from JSONs for config SoA; add sanity checks; add HGCalMappingModuleIndexer::getNFED; rename getFedAndModuleIndex -> getIndexForFedAndModule; clean code; ... rename macro silence debugging message in ESProducers silence cout add HGCalMappingModuleIndexer::getMaxModuleSize(fedid) typo; SystemTestEventFilters/data -> Configuration/data bugfix in the cm Sum fixing dense index issue Update comment in HGCalMappingParameterSoA.h Some additional fixes (#132) - remove_if was missing a call to std::vector::erase to effectively remove unused entries - typecodemap was missing a reset with the final indices - usage of command line options was missing in module map tester - do not use moduleLUT in DenseIndexInfoESProducer!! it has the full dimensions before trimming to the actual number of ECON-Ds in a FED Add parsing function for econd flag; renaming HGCalECONDInfo to HGCalECONDPacketInfo Add irot to module mapper (#134) * - remove_if was missing a call to std::vector::erase to effectively remove unused entries - typecodemap was missing a reset with the final indices - usage of command line options was missing in module map tester * do not use moduleLUT in DenseIndexInfoESProducer!! * Adding optional irot to module mapper file - If existing it will be put in the module info SoA - othwerwise a default irot=0 (baseline is used) Suppress "unused variable" warning: remove CellIndexer in HGCalDigiSoaFiller.cc apply code formats (#135) Apply suggestions from code review Apply easy suggestions by Matti. (Looking into other comment & suggestions.) Co-authored-by: Matti Kortelainen <matti.kortelainen@cern.ch> 'const json' to fix compilation issue for Matti's suggestion gain index should be taken as is, not subtracted by 1 make use of valid at rechit level / set rechit valids fill rec hitflags update kernel after closure Implement suggestions for CMSSW_14_1_X PR cms-sw#45665 (#137) * Implement TH2Poly in DQM Services for HGCal DQM * Apply changes from code quality checks * Add a patch based on makortel's comments * enum -> namespace + constexpr uint{8,16,32}_t * Matti suggestion: remove '_t' postfix from HGCalFEDReadoutSequence_t, HGCalROCConfig_t, HGCalECONDConfig_t, HGCalFedConfig_t structs * Matti suggestions & code checks: remove redundant comments; silence debugging cout or use MessageLogger; move includes; remove trailing '_' from function arguments * Matti suggestion: reuse typecodeMap_.find() to avoid searching twice (TODO: throw exception if not found?) * Matti suggestion: use SET_PORTABLEHOSTCOLLECTION_READ_RULES; add ClassVersion+checksum * Matti suggestion: iRecord.getRecord<...>. -> iRecord. * Matti suggestion: iRecord.getRecord<...>. -> iRecord.; rm redundant includes & couts; pass reference to fill_SoA_column to avoid copy * Matti suggestions: use SET_PORTABLEHOSTCOLLECTION_READ_RULES for HGCalRecHitHost; Remove deprecated HGCalFlaggedECONDInfo* & comments; * Matti suggestion: throw exception if type code not found; replace for-loop with std::accumulate * Matti suggestion: remove TestObjects * Matti suggestions: remove config as member data and produce unique_ptr instead of shared_ptr; NULL → nullptr; initialize optional override; memcp -> std::copy * Matti suggestions: use file in path * removing soafiller and updating buildfile (#139) * address comments on HGCalRecHitCalibrationAlgorithms.dev.cc * irot as uint8_t instead of char * restore DQM without TH2Poly * apply code formats --------- Co-authored-by: ywkao <ywkao@hep1.phys.ntu.edu.tw> Co-authored-by: Pedro <psilva@cern.ch> restore release base DQM classes (#140) move to chrono calls instead of struct inside namespace (#141) address comments (#142) update include (#143) remove ECON-D tests as they'll need to be re-written in a different form (#144) Declare HGCalRecHitCalibrationAlgorithms::calibrate as const method (#145) * remove ECON-D tests as they'll need to be re-written in a different form * enable const patching unpacker for padding of capture blocks within FED move check of padding to start of capture block Fix the throw, so the processing won't stop at the first event with error Update HGCalRecHitCalibrationAlgorithms.dev.cc CM sum -> 0.5*CM sum Change the throws to specific returns Optimize the producer Debugging the padding of eRx subpackets roll back (#149) apply code checks and remove dqm
[Work in progress.]
Implement suggestions by @makortel in PR cms-sw#45665 to official CMSSW:
enum
withnamespace
&constexpr uint{8,16,32}_t
inDataFormats/HGCalDigi/interface/HGCalRawDataDefinitions.h
._t
postfix fromHGCalFEDReadoutSequence_t
,HGCalROCConfig_t
,HGCalECONDConfig_t
,HGCalFedConfig_t
structs.RecoLocalCalo/HGCalRecAlgos/interface/alpaka/HGCalRecHitCalibrationAlgorithms.h
toRecoLocalCalo/HGCalRecAlgos/plugins/alpaka/HGCalRecHitCalibrationAlgorithms.dev.cc
._
from function argumentsn_blocks_
&n_threads_
, and add to private data members.scram build code-checks
orscram build code-format
.Needs to be run together with
hgcal-dpg/hgcal-comm!45
.