Function Pipelines Feedback & Questions

[davidkohn88]

Give feedback on the blog post about function pipelines here, as well as feedback/ questions on how to use the feature!

[jbylund]

Is there any tooling in place to facilitate keeping multiple versions of pipelines elements in sync between version control and the db? I can imagine doing something like creating schemas per release or per git sha but I feel like I'd be making a lot of custom (read: buggy) glue to keep my git repo and the db in sync?

[WireBaron]

While we have these elements in our experimental schema, we're assuming they're likely to see enough changes that we're not going to be able to keep old versions up to date. In a few release cycles, as these changes become less frequent, we'll move them out of experimental, at which point we're committing to maintain backward compatibility.

[erikvanoosten]

One thing that kept me busy for months in a recent job was the following task:

• Given a stream of millions of events per day, with each event indicating the new state, for hundreds of thousands of devices,
• for a given period (e.g. a month), report the amount of time a device (or a selection of devices) was in each of the 4 allowed states (optionally grouped by hour or day).
  And do it fast enough for a dashboard, with any sub-selection of devices.

Is that a use-case that could be eloquently written with the new pipeline elements?

[jfjoly]

@erikvanoosten the state_agg function should help with measuring the amount of time a device spent in each state. Check it if you have a moment and don't hesitate to open a forum discussion or a new issue if you have any questions about the state_agg function.