Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nexus: should default binding option on Summit be -b rs? #3222

Closed
Paul-St-Young opened this issue Jun 4, 2021 · 3 comments
Closed

Nexus: should default binding option on Summit be -b rs? #3222

Paul-St-Young opened this issue Jun 4, 2021 · 3 comments

Comments

@Paul-St-Young
Copy link
Contributor

Describe the bug
Nexus' default bind option on Summit is -b rs.
I observed that -b rs oversubscribes the cores in a pure MPI run.

To Reproduce
Steps to reproduce the behavior:

  1. Launch interactive session on Summit
    bsub–W 10 –nnodes1 –P $OLCF_PROJECT_ID –Is $SHELL
  2. Load job-step-viewer
    module load job-step-viewer
  3. View jsrun with -b rs

Expected behavior
All MPI tasks of a resource set are bound to its first core.
Command: export OMP_NUM_THREAEDS=1; jsrun -n 6 -c 7 -a 7 -b rs -d packed
brs

The problem can be solved by using -b packed:$OMP_NUM_THREADS instead of -b rs
Command: export OMP_NUM_THREAEDS=1; jsrun -n 6 -c 7 -a 7 -b packed:1 -d packed
bind1

Is this a real problem or did I misunderstand jsrun?

System:

  • summit
@Paul-St-Young
Copy link
Contributor Author

I should add that -b rs works as intended when each resource set is purely OpenMP
export OMP_NUM_THREAEDS=7; jsrun -n 6 -c 7 -a 1 -b rs -d packed
appropriately binds one thread to each core

@ye-luo
Copy link
Contributor

ye-luo commented Jun 4, 2021

You may use qmc-check-affinity to access exact affinity. SMT1 not used?

@prckent
Copy link
Contributor

prckent commented Jan 2, 2025

Closing since Summit is no more.

@prckent prckent closed this as completed Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants