Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CPU service calls to ResourceInformationService for non-Intel CPUs #40441

Merged

Conversation

dan131riley
Copy link

@dan131riley dan131riley commented Jan 6, 2023

PR description:

The changes made to the CPU service in #37831 to fill the ResourceInformationService CPU models list fails on non-Intel CPUs where "model name" doesn't appear in /proc/cpuinfo. This PR integrates the fallback to the cpufeatures library in a more transparent way that fixes the immediate problem and should reduce the possibility of future issues.

Fixes TestResourceInformationService unit test failures on non-Intel platforms.

PR validation:

aarch64 output of FWCore/Services/test/test_resourceInformationService_cfg.py before:

    cpu models:
        None
    gpu models:
        None
    acceleratorTypes:
        GPU
    nvidiaDriverVersion:
    cudaDriverVersion: 0
    cudaRuntimeVersion: 0
    cpuModelsFormatted: aarch64 0 0
    cpuAverageSpeed: 0

After:

    cpu models:
        aarch64 0 0
    gpu models:
        None
    acceleratorTypes:
        GPU
    nvidiaDriverVersion:
    cudaDriverVersion: 0
    cudaRuntimeVersion: 0
    cpuModelsFormatted: aarch64 0 0
    cpuAverageSpeed: 0

@cmsbuild cmsbuild added this to the CMSSW_13_0_X milestone Jan 6, 2023
@dan131riley dan131riley changed the title CPU service to accomodate non-Intel CPUs Update CPU service calls to ResourceInformationService for non-Intel CPUs Jan 6, 2023
@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 6, 2023

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40441/33588

  • This PR adds an extra 12KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 6, 2023

A new Pull Request was created by @dan131riley (Dan Riley) for master.

It involves the following packages:

  • FWCore/Services (core)

@cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please review it and eventually sign? Thanks.
@makortel, @missirol, @wddgit, @fwyzard this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@makortel
Copy link
Contributor

makortel commented Jan 6, 2023

Thanks Dan, this was an oversight (of me) in review of #37831. Would it be feasible to resolve the duplication already now? I'm thinking something like changing the getModels() to return a std::set<std::string> (that would include the return value of getModelFromCPUFeatures() when necessary), and adding another (e.g.) formatModels() that converts the set to a comma-separated string.

@dan131riley
Copy link
Author

@makortel that looks cleaner than what I was thinking of, will update the PR

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2023

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40441/33593

  • This PR adds an extra 16KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2023

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40441/33594

  • This PR adds an extra 16KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2023

Pull request #40441 was updated. @cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please check and sign again.

@dan131riley
Copy link
Author

PR updated, PR description updated, and since this is now more than a minimal update, made some other gratuitous cleanups of whatever caught my eye. Should be more maintainable now.

FWCore/Services/plugins/CPU.cc Outdated Show resolved Hide resolved
FWCore/Services/plugins/CPU.cc Outdated Show resolved Hide resolved
@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2023

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40441/33622

  • This PR adds an extra 12KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2023

Pull request #40441 was updated. @cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please check and sign again.

@makortel
Copy link
Contributor

makortel commented Jan 9, 2023

@cmsbuild, please test

@makortel
Copy link
Contributor

makortel commented Jan 9, 2023

@cmsbuild, please test for el8_aarch64_gcc11

@makortel
Copy link
Contributor

makortel commented Jan 9, 2023

@cmsbuild, please test for el8_ppc64le_gcc11

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2023

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7d764d/29857/summary.html
COMMIT: 3df8ae5
CMSSW: CMSSW_13_0_X_2023-01-08-2300/el8_ppc64le_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40441/29857/install.sh to create a dev area with all the needed externals and cmssw changes.

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 9, 2023

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7d764d/29856/summary.html
COMMIT: 3df8ae5
CMSSW: CMSSW_13_0_X_2023-01-09-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40441/29856/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 36 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3555538
  • DQMHistoTests: Total failures: 1194
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3554322
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 211 log files, 162 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7d764d/29857/summary.html
COMMIT: 3df8ae5
CMSSW: CMSSW_13_0_X_2023-01-08-2300/el8_ppc64le_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40441/29857/install.sh to create a dev area with all the needed externals and cmssw changes.

@makortel
Copy link
Contributor

+1

Comparison differences are in 11634.7 (#39803)

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 6593af1 into cms-sw:master Jan 10, 2023
@dan131riley dan131riley deleted the resourceInformationService-non-intel branch January 10, 2023 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants