Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TOOLFILES] Check mergebility after merging with latest cmsdist #7110

Closed

Conversation

mrodozov
Copy link
Contributor

@mrodozov mrodozov commented Jul 7, 2021

this of course cant be tested yet

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 7, 2021

A new Pull Request was created by @mrodozov (Mircho Rodozov) for branch IB/CMSSW_12_0_X/master.

@cmsbuild, @smuzaffar, @mrodozov, @iarspider can you please review it and eventually sign? Thanks.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.
cms-bot commands are listed here


done \
tool_script=${SCRAM_TOOLS_ROOT}/scram-tools/$tool/$(echo $tool | tr '[A-Z]' '[a-z]').sh \
if [ -f $tool_script ]; then ($tool_script %i/tools/selected); fi
Copy link
Contributor

@smuzaffar smuzaffar Jul 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrodozov , I was thinking to move all this CopyScramTools logic in a script which we can call like ${SCRAM_TOOLS_ROOT}/get_tools <tool_name> <outdir>. This tool then can process the <toolname>/<toolname>*.xml files and replace @TOOL_ROOT@ and @TOOL_VERSION@ and can internally call ${SCRAM_TOOLS_ROOT}/get_tools/<toolname>/<toolname>.sh if available

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll move that part in a file to be called as a script with args then

Copy link
Contributor

@smuzaffar smuzaffar Jul 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the ${SCRAM_TOOLS_ROOT}/get_tools script can take care of common variables e.g.

TOOL_ROOT
TOOL_VERSION
PYTHONV
PYTHON3V

and rest can go in tool's specific script. The tool's specific script should only set the environment variables e.g.

COMPILER_NAME_SUFFIX
OS_CSHAREDOBJECTFLAGS
ARCH_CSHAREDOBJECTFLAGS
BOOST_CHRONO_LIB
BOOST_DATE_TIME_LIB

and the get_tools script should do the replacement. e.g. get_tools can do source gcc/gcc.sh ( which should set the env) and then get_tools should do the replacement of @VARIABLE@

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrodozov , please take note of my comment #7110 (comment) too

@mrodozov
Copy link
Contributor Author

mrodozov commented Jul 7, 2021

please test
lets check the available externals

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 7, 2021

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68cf0c/16546/summary.html
COMMIT: f614add
CMSSW: CMSSW_12_0_X_2021-07-06-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7110/16546/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation warning when building: See details on the summary page.

%ifarch x86_64
Requires: dmtcp-toolfile
Requires: tkonlinesw-toolfile
Requires: oracle-toolfile
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrodozov , we need to keep the dependencies but should replace <tool>-toolfile with <tool> only.
This should also be done for coral and fwlite tool-conf.spec

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file is containing a short list because so I first finish pkgs with any script logic, the short list if for not deploying all xmls every time (not to build the whole cmssw-tool-conf)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this type of changes are better to test locally as this will help with debugging.

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 7, 2021

Pull request #7110 was updated.

@mrodozov
Copy link
Contributor Author

mrodozov commented Jul 7, 2021

please test
lets see the updated url

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 7, 2021

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68cf0c/16548/summary.html
COMMIT: 489f32a
CMSSW: CMSSW_12_0_X_2021-07-06-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7110/16548/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation warning when building: See details on the summary page.

@cmsbuild
Copy link
Contributor

Pull request #7110 was updated.

@cmsbuild
Copy link
Contributor

Pull request #7110 was updated.

@mrodozov
Copy link
Contributor Author

mrodozov commented Jul 16, 2021

test parameters:

  • full_cmssw=true
  • pull_request = cms-sw/scram-tools#7

@mrodozov
Copy link
Contributor Author

please test

@mrodozov
Copy link
Contributor Author

abort

@mrodozov
Copy link
Contributor Author

please test

@mrodozov
Copy link
Contributor Author

please test for CMSSW_12_0_X/slc7_ppc64le_gcc9

@mrodozov
Copy link
Contributor Author

please test for slc7_aarch64_gcc9

%post
echo "%{BaseTool}_ROOT='$CMS_INSTALL_PREFIX/%{pkgrel}'" > $RPM_INSTALL_PREFIX/%{pkgrel}/etc/profile.d/init.sh
echo "set %{BaseTool}_ROOT='$CMS_INSTALL_PREFIX/%{pkgrel}'" > $RPM_INSTALL_PREFIX/%{pkgrel}/etc/profile.d/init.csh
echo "%{BaseTool}_PKGREQUIRED='%pkgreqs'" >> $RPM_INSTALL_PREFIX/%{pkgrel}/etc/profile.d/init.sh
echo "set %{BaseTool}_PKGREQUIRED='%pkgreqs'" >> $RPM_INSTALL_PREFIX/%{pkgrel}/etc/profile.d/init.csh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrodozov once test are done then please remove these lines, I think these are not needed any more ... right?

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68cf0c/16901/summary.html
COMMIT: 14e1e98
CMSSW: CMSSW_12_0_X_2021-07-15-2300/slc7_ppc64le_gcc9
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7110/16901/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68cf0c/16901/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68cf0c/16901/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test test_PrepareInputDb had ERRORS
---> test test_MpsWorkFlow had ERRORS
---> test TestDQMServicesDemo had ERRORS
---> test testFWCoreUtilities had ERRORS
and more ...

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68cf0c/16899/summary.html
COMMIT: 14e1e98
CMSSW: CMSSW_12_0_X_2021-07-15-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7110/16899/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68cf0c/16899/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68cf0c/16899/git-merge-result

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 38
  • DQMHistoTests: Total histograms compared: 2786302
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2786279
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 37 files compared)
  • Checked 160 log files, 37 edm output root files, 38 DQM output files
  • TriggerResults: no differences found

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests RelVals AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68cf0c/16903/summary.html
COMMIT: 14e1e98
CMSSW: CMSSW_12_0_X_2021-07-15-2300/slc7_aarch64_gcc9
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7110/16903/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68cf0c/16903/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-68cf0c/16903/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test TestFWCoreServicesDriver had ERRORS
---> test TestDQMServicesDemo had ERRORS
---> test testUploadConditions had ERRORS
---> test TestDQMOfflineConfiguration_160 had ERRORS
and more ...

RelVals

AddOn Tests

  • read312RVcmsRun /home/cmsbuild/jenkins_a/workspace/ib-run-pr-addon/CMSSW_12_0_X_2021-07-15-2300/src/Utilities/ReleaseScripts/scripts/read312RV_cfg.py : FAILED - time: date Sat Jul 17 06:23:02 2021-date Sat Jul 17 06:23:01 2021 s - exit: 32512
  • fastsimcmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - time: date Sat Jul 17 06:23:04 2021-date Sat Jul 17 06:23:03 2021 s - exit: 256
  • fastsim1cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc_l1stage1 --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns : FAILED - time: date Sat Jul 17 06:23:08 2021-date Sat Jul 17 06:23:07 2021 s - exit: 256
Expand to see more addon errors ...

@cmsbuild
Copy link
Contributor

Pull request #7110 was updated.

@smuzaffar
Copy link
Contributor

closing in favor of #7147

@smuzaffar smuzaffar closed this Jul 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants