Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow specifying fractions as RMM pool initial/maximum size #1021

Merged
merged 5 commits into from
Oct 20, 2022
Merged

Allow specifying fractions as RMM pool initial/maximum size #1021

merged 5 commits into from
Oct 20, 2022

Conversation

pentschev
Copy link
Member

This is already supported by memory_limit/device_memory_limit, and this has been raised in #1020 during discussions on how to make Dask Profiles usable in Dask-CUDA.

@pentschev pentschev requested a review from a team as a code owner October 19, 2022 11:58
@github-actions github-actions bot added the python python code needed label Oct 19, 2022
@pentschev pentschev mentioned this pull request Oct 19, 2022
Open
@pentschev pentschev added 3 - Ready for Review Ready for review by team feature request New feature or request non-breaking Non-breaking change labels Oct 19, 2022
@codecov-commenter
Copy link

codecov-commenter commented Oct 19, 2022 •
edited
Loading

Codecov Report

❗ No coverage uploaded for pull request base (branch-22.12@acc96a5). Click here to learn what that means.
Patch has no changes to coverable lines.

❗ Current head 116c484 differs from pull request most recent head 14daf7e. Consider uploading reports for the commit 14daf7e to get more accurate results

Additional details and impacted files 
@@              Coverage Diff               @@
##             branch-22.12   #1021   +/-   ##
==============================================
  Coverage                ?   0.00%           
==============================================
  Files                   ?      17           
  Lines                   ?    2213           
  Branches                ?       0           
==============================================
  Hits                    ?       0           
  Misses                  ?    2213           
  Partials                ?       0

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

wence-
wence- approved these changes Oct 19, 2022
View reviewed changes
Copy link
Contributor

@wence- wence- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very minor, if you want to make the changes go ahead (or disagree), looks good.

dask_cuda/utils.py Outdated Show resolved Hide resolved
dask_cuda/local_cuda_cluster.py
if self.rmm_pool_size is not None:
self.rmm_pool_size = parse_bytes(self.rmm_pool_size)
if self.rmm_maximum_pool_size is not None:
self.rmm_maximum_pool_size = parse_bytes(self.rmm_maximum_pool_size)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does the pool size now get set? That is, where does the parsing into an int happen?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In https://github.com/rapidsai/dask-cuda/pull/1021/files#r999492018, as you noted yourself. 🙂

dask_cuda/utils.py Show resolved Hide resolved
@pentschev @wence-
Minor code simplification
14daf7e 
Co-authored-by: Lawrence Mitchell <wence@gmx.li>
pentschev
pentschev commented Oct 19, 2022
View reviewed changes
Copy link
Member Author

@pentschev pentschev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @wence- for the review, I've addressed your suggestion and question. Could you take another look when you have the chance?

dask_cuda/local_cuda_cluster.py
if self.rmm_pool_size is not None:
self.rmm_pool_size = parse_bytes(self.rmm_pool_size)
if self.rmm_maximum_pool_size is not None:
self.rmm_maximum_pool_size = parse_bytes(self.rmm_maximum_pool_size)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In https://github.com/rapidsai/dask-cuda/pull/1021/files#r999492018, as you noted yourself. 🙂

dask_cuda/utils.py Show resolved Hide resolved
@wence-
Copy link
Contributor

wence- commented Oct 20, 2022

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 713df2d into rapidsai:branch-22.12 Oct 20, 2022
@pentschev
Copy link
Member Author

Thanks @wence- for the review!

@pentschev pentschev deleted the rmm-pool-size-fraction branch October 20, 2022 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wence- wence- wence- approved these changes

Assignees
No one assigned
Labels
3 - Ready for Review Ready for review by team feature request New feature or request non-breaking Non-breaking change python python code needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@pentschev @codecov-commenter @wence-