Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pbapply cl argument can't be an integer when using windows #105

Open
dblodgett-usgs opened this issue Jun 30, 2024 · 0 comments
Open

pbapply cl argument can't be an integer when using windows #105

dblodgett-usgs opened this issue Jun 30, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@dblodgett-usgs
Copy link
Collaborator

dblodgett-usgs commented Jun 30, 2024

Describe the bug
From pbapply docs:

A cluster object created by makeCluster, or an integer to indicate number of child-processes (integer values are ignored on Windows) for parallel evaluations (see Details on performance).

I'll put in a little work around, but we should probably figure out how to allow a person to pass in their own cluster object rather than using an integer number of workers.

There is a bit of overhead to spin up a cluster and shut it down. With future, the cluster gets created a-priori, which is akin to dask or the cluster from parallel. I think it's probably going to be best to accept that kind of cluster input in some way in addition to this simple setting of the level of parallelism on a per call basis.

@dblodgett-usgs dblodgett-usgs added the bug Something isn't working label Jun 30, 2024
@dblodgett-usgs dblodgett-usgs self-assigned this Jul 1, 2024
dblodgett-usgs added a commit to dblodgett-usgs/pizzarr that referenced this issue Jul 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant