[RELEASE] dask-cuda v0.17 #469
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[gpuCI] Auto-merge branch-0.16 to branch-0.17 [skip ci]
[gpuCI] Auto-merge branch-0.16 to branch-0.17 [skip ci]
[gpuCI] Auto-merge branch-0.16 to branch-0.17 [skip ci]
* use string for bandwidth output when using a remote cluster for cudf merge benchmark * Fix output in CuPy benchmark tests when using scheduler address Co-authored-by: Peter Andreas Entschev <peter@entschev.com>
Fix typo in benchmark utils
#420) * add all to all and test * lint * use all-to-all and move wait_for_client in cudf merge * make all-to-all optional * fix all-to-all logic * remove cleanup command and simplify all-to-all * remove cleanup from test
Update changelog for 0.16
Revert "Update changelog for 0.16"
[gpuCI] Auto-merge branch-0.16 to branch-0.17 [skip ci]
* add disable-rmm option * lint * cleanup * ignore dask-worker-space directories * disable reinitializing the pool when disable-rmm-pool is set. Also, allow for setting the rmm-pool-size * update benchmark to use disable_rmm_pool * change default chunk size * revert default chunksize
See #431 for CI failures
* Decrease data size to shorten spilling tests time * Fix formatting
Add codecov.yml and ignore uncovered files
Do not skip DGX/TCP tests when ucp is not installed
* Allow parsing CUDA_VISIBLE_DEVICES with UUID Add new parse_cuda_visible_device utility function to parse UUIDs * Add test CUDA_VISIBLE_DEVICES parsing with UUID * Move cuda_visible_devices to utils.py * Fix formatting * Fix parse_cuda_visible_device doc typo Co-authored-by: Mads R. B. Kristensen <madsbk@gmail.com>
Co-authored-by: Peter Andreas Entschev <peter@entschev.com> * Initial implementing of ObjectProxy * Added basic tests of ObjectProxy * Implemented some more proxy attributes * Added spilling of proxy object optional * Re-added dask_serialize for DeviceSerialized * Added support of __array__ * Added __sizeof__ * Added some spill_proxy tests in test_device_host_file.py * Checking len() instead of .size() * Added dispatch support of hash_object_dispatch and group_split_dispatch * Added "*args, **kwargs" to dispatch of ObjectProxy * Added dispatch of make_scalar * Added dispatch of concat_dispatch * meta.yaml: added pandas dependency * meta.yaml: depend on dask (not only dask-core) * Added jit-unspill worker option * meta.yaml: removed pandas * Using explicit args for the dispatch functions * ObjectProxy._obj_pxy_serialize(): takes serializers * serializers replaces is_serialized * Supporting cuda serializers * Added a lot of operators * fixed typos * Support and test of a proxy object of a proxy object * test_spilling_local_cuda_cluster(): added some extra checks * Added _obj_pxy_is_cuda_object() * asproxy(): added subclass argument * fixed type in test_spilling_local_cuda_cluster check * Added test of communicating proxy objects * Making ObjectProxy threadsafe * renamed ObjectProxy => ProxyObject * Never re-serialize proxy objects * Test: setting device_memory_limit="1B" to force serialization * test: added an explicit client shutdown * Added some str/repr tests * added some more checks in test_proxy_object_of_numpy * ProxyObject: added docs * added unproxy() * Added ValueError when serializers isn't specified * Style and spelling fixes * ProxyObject.__sizeof__(): use dask.sizeof() * Serializer: convert to tuples before comparing
* Add CPU support to CuPy transpose sum benchmark * Add back NumPy import to CuPy transpose sum benchmark
* Add `MANIFEST.in` to include versioneer components Some of the components of versioneer may not have been included during packaging. To fix this we add a `MANIFEST.in` file to ensure these pieces are included. * Always use versioneer Make sure that we always query versioneer for the version. It already handles cases where the package does not live in version control (like being installed or in a tarball).
* Drop `x.x` syntax This has been deprecated for a while in Conda-Build and isn't strictly needed any more. So go ahead and drop it. * Replace `build` with `host` Conda-Build 2 added the `host` environment for things that would be needed to link against so as to distinguish it from `build` tools, whose CLIs were only need (like CMake). Given this, go ahead and switch this to `host` to match with that usage.
* update builddocs dependency requirements * only need pandas * add dask back
Specify `master` branch to bypass `pip` solver. Authors: - John Kirkham <jakirkham@gmail.com> Approvers: - Benjamin Zaitlen - AJ Schmidt URL: #454
* Allow setting number of threads, defaulting to 1 * Add threads per worker tests * Fix black/flake8 formatting
* serializing using: on_error="raise" * Implemented _obj_pxy_get_init_args() * Implemented __reduce__
Add note on RMM pool docs mentioning its per worker [skip ci]
Reverts some `setup.py` changes that are necessary for PyPI uploads.
Looks like `setup.py` expects to find `requirements.txt` in the `sdist`, but we don't include that there. To fix that, this adds `requirements.txt` to `MANIFEST.in` to ensure it shows up in packages.
Gpuciscripts clean and update
Codecov Report
@@ Coverage Diff @@
## main #469 +/- ##
===========================================
+ Coverage 56.89% 89.83% +32.93%
===========================================
Files 19 15 -4
Lines 1443 1121 -322
===========================================
+ Hits 821 1007 +186
+ Misses 622 114 -508
Continue to review full report at Codecov.
|
Update changelog for 0.17
Restore `setup.py` bits for PyPI
Fix PyPI Uploads [skip-ci]
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
❄️ Code freeze for
branch-0.17and v0.17 releaseWhat does this mean?
Only critical/hotfix level issues should be merged into
branch-0.17until release (merging of this PR).What is the purpose of this PR?
branch-0.17intomainfor the release