Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MoreStream utility methods that accept future suppliers #329

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from

Conversation

msg430
Copy link

@msg430 msg430 commented May 23, 2024

Currently the non-deprecated versions of inCompletionOrder and blockingStreamWithParallelism require a an executor to run each transformation on, which means that that transformation will be a blocking call. This is not ideal when the actual call is itself async (i.e. hitting a method which returns a ListenableFuture already). This PR adds versions of those methods that require a future supplier, that is a method that transforms the keys into the listenable futures of the result. This method will be called by the thread that is pulling from the Stream.

This is similar to earlier problematic methods that accepted an input a stream of futures. Those were deprecated because they did not enforce actual max parallelism. That is, the underlying stream supplier may kick off more futures than the parallelism intended to allow. I believe this solution avoids those issues because we only create new futures from within the buffering spliterator, which enforces the max parallelism.

@palantirtech
Copy link
Member

Thanks for your interest in palantir/streams, @msg430! Before we can accept your pull request, you need to sign our contributor license agreement - just visit https://cla.palantir.com/ and follow the instructions. Once you sign, I'll automatically update this pull request.

@changelog-app
Copy link

changelog-app bot commented May 23, 2024

Generate changelog in changelog/@unreleased

What do the change types mean?
  • feature: A new feature of the service.
  • improvement: An incremental improvement in the functionality or operation of the service.
  • fix: Remedies the incorrect behaviour of a component of the service in a backwards-compatible way.
  • break: Has the potential to break consumers of this service's API, inclusive of both Palantir services
    and external consumers of the service's API (e.g. customer-written software or integrations).
  • deprecation: Advertises the intention to remove service functionality without any change to the
    operation of the service itself.
  • manualTask: Requires the possibility of manual intervention (running a script, eyeballing configuration,
    performing database surgery, ...) at the time of upgrade for it to succeed.
  • migration: A fully automatic upgrade migration task with no engineer input required.

Note: only one type should be chosen.

How are new versions calculated?
  • ❗The break and manual task changelog types will result in a major release!
  • 🐛 The fix changelog type will result in a minor release in most cases, and a patch release version for patch branches. This behaviour is configurable in autorelease.
  • ✨ All others will result in a minor version release.

Type

  • Feature
  • Improvement
  • Fix
  • Break
  • Deprecation
  • Manual task
  • Migration

Description

Add MoreStream utility methods that accept future suppliers

Check the box to generate changelog(s)

  • Generate changelog entry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants