Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add ants worker pool #268

Closed
plastikfan opened this issue May 27, 2024 · 0 comments · Fixed by #270
Closed

add ants worker pool #268

plastikfan opened this issue May 27, 2024 · 0 comments · Fixed by #270
Assignees
Labels
feature New feature or request

Comments

@plastikfan
Copy link
Contributor

plastikfan commented May 27, 2024

the worker pool in ants looks ok-ish to user but it has some deficiencies:

  • each job is simply a function func() and there are 2 problems with this:
  1. no function result; what happens if we wish to submit a job that needs a result, eg a calculation
  2. no error returned
  • propensity to use interface{}; not type safe; prefer to use generics instead
  • no top level context; so how do we cancel?
  • submitting jobs requires invoking Pool.Submit, with this, we lose out on the benefit of loose coupling. Now the client must have access to the pool, rather than sending jobs into a channel (our wrapper can fix this)
  • no concept of a sequence no for each job executed; I'll add this as a feature
  • ants uses its own logger, when it should just use the one provided by the client

The source code will be copied into lorax and heavily modified, but its core will remain the same. I have no need for a multi pool, so this wil not be used.

This task will port the stretchr tests to use ginko (but actually as we already know the worker-pool works, do not implement all the tests straight away, just do a couple of key tests. Only once we have established that we can make our own changes to the core, will we implment all the tests; I don't want to repeat the mistake I made with rxgo, where I implememnted all the units tests ajust to find out at the end, that rxgo was unsuitable).

The idea behind the use of ants will be to retain most of the user interface we already defined in the legacy work pool but use the ants worker pool under the covers. One change we'll make will be to allow the job to specific an executive function that will override the one defined on the pool; this will allow many different types of execution to occur in the ppol, not just the default one defined on the pool.

@plastikfan plastikfan added the feature New feature or request label May 27, 2024
@plastikfan plastikfan self-assigned this May 27, 2024
@plastikfan plastikfan linked a pull request May 27, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant