Guide: David Greenberg

[dgrnbrg]

I'm David Greenberg, a functional and distributed systems programmer. My passions are data structures, concurrency, and distributed systems, and especially their intersection with functional programming concepts. I take a very relaxed approach to types: I equally enjoy Rust & Clojure, and wrote Clojure professionally for many years. I take a stochastic approach to progamming--I assume everything I write will have bugs, and so I like to think about how to use simulation & generative testing to make my software more robust (as opposed to encoding invariants in types).

I've written a variety of functional libraries, including a generatively test CRDT library, an asynchronous HTTP server library, an FPGA DSL for Clojure, and the book Building Applications on Mesos.

There's two topics I'm most excited about talking about, but I need your help to fine-tune what we'll discuss:

• Hitchhiker Trees - Hitchhiker trees are a purely functional and remote I/O optimized variation of fractal trees, which are themselves I/O optimized variations of B+ Trees. We have a prototypical implementation that we could hack on, adding things like distributed GC or alternative indices. We could spend time discussing the various techniques used to make data structures functional, and the various techniques used to make functional data structures fast.
• Distributed functional programming - Most functional programming currently exists on a single machine, and relies on garbage collection to ensure that functional data structures can remain efficient. How do we get the same advantages of functional programs when we scale up to dozens or hundreds of machines? We can discuss individual properties of functional programming and which things will work as we scale up. Some concepts we could investigate are monotonicity, idempotency, and multi-actor consistency considerations.

If you're interested in other aspects of concurrency, parallelism, and distribution, and their interaction with functional programming, so am I--let's spend the day learning!

If you have other ideas for what I should speak on at the Winter Retreat, please post them below! Vote on ideas by using the thumbs up / thumbs down emojis. Thanks in advance for all of your feedback, and I look forward to being a guide at LambdaConf Winter Retreat 2017!

[mudphone]

I'm really excited by the technologies you've mentioned, especially CRDTs. I'd be very excited about a discussion of "Distributed FP." I have some experience with Clojure, and am interested in learning Rust. Did I mention I'm excited about a discussion of CRDTs?

[haroldcarr]

+1 for Distributed Functional Programming

[jdegoes]

👍 for distributed functional programming.

[rezalesmana]

+1 for Distributed Functional Programming. But, would really love if you could talk about that by also taking examples from your book Building Applications on Mesos. 👍

[purefunctions]

Cloud Haskell
Elixir
How the above compare to Clojure and Akka!

[ErrorNullPointer]

+1 for Distributed Functional Programming

[LambdaMathematician]

+1 to Distributed Functional Programming

[dgrnbrg]

Here we go! This is the planned session description:

If you’re coming to the retreat, you probably are interested in FP, if not already a full-blown user of FP and its techniques. Unfortunately, when it comes time to work on systems that scale beyond one machine, many techniques in functional programming no longer seem to be possible. For example, distributed garbage collection isn’t generally available, which makes it difficult to work with immutable data structures across multiple machines. Another issue is function composition—RPC semantics are difficult to get right, and the additional network latency incurred makes many FP design pattens unrealistic in distributed environments.

In this session, we’ll recast various ideas from FP to see their analogies in distributed systems. We will discuss how monotonic functions can offer many of the benefits of pure functions. We’ll look at dependencies and causality: ways our programs express the results of one computation being the input to the next computation, and performant techniques that achieve similar results in distributed systems. We’ll also analyze distributed functional data representations. By deconstructing the things we love most about FP, we’ll see how we can preserve their essence but make them work in distributed environments.

You should be familiar with pseudocode—we won’t be digging into specific languages with large amounts of code, but we will look at general ways to express these ideas. You should come eager to discuss, as we’ll learn best by sharing our thoughts and revelations as we break down FP concepts and see how they can be mapped into the distributed domain.