Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple output streams for trident #731

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

alienrobotwizard
Copy link

Addresses issue 638 (http://github.com/nathanmarz/storm/issues/638). @mridulj @mrflip

Consider:

Map<String, Fields> outputStreamMapping = new HashMap<String, Fields>();
outputStreamMapping.put("streamA", new Fields("field_in_a"));
outputStreamMapping.put("streamB", new Fields("field_in_b"));

Stream s = topology.newStream("testStream", spout);
StreamCollection multiStream = s.each(
  s.getOutputFields(),
  new DebugMultiStream("streamA", "streamB"),
  new StreamMap(outputStreamMapping)            
  );

Stream streamA = multiStream.getStream("streamA")
streamA.each(new Fields("field_in_a"), new Debug());

@nathanmarz
Copy link
Owner

I've only glanced at this, but I have two initial comments:

  1. This needs tests
  2. I'm not so sure about the API. I think that when the multistream each is used, it should return a "MultiStream" object instead of a stream object. And then the MultiStream object can be queried for the particular streams.

@alienrobotwizard
Copy link
Author

Nathan,

  1. Yes, absolutely. That's the next thing. I just wanted to get the patch out asap since @mridulj was asking about it.
  2. The multi each returns a StreamCollection object that has a 'getStream' method. It could just as easily be called MultiStream if that seems like a clearer name.

@alienrobotwizard
Copy link
Author

Oh I see the confusion. I've been writing my topologies in jruby which is why I can reuse 's' up there for both a Stream and a StreamCollection. I just did a quick conversion as I was typing. But my previous comment holds, it does return a collection in the way you're asking,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants