Async

Control flow tools

Once upon a time there was a paradisiac userland where functions received arguments and gave back values. So, for instance you had this function: sum(a, b); you knew that feeding it with values 3 and 4 gave you back a 7. Simple and straightforward.

That paradise is no more, gone forever in the mist of event driven programming model. So where you used to have values, now you have promises, "something that will have a value at an unknown point in the future". What this means is that when a function returns, it may not have the value you're waiting for, so the sync code:

    var n1, n2, result1, n3, result2, n4, result3;
    result1 = sum(n1, n2); // if sum executes asynchronously, result1 is undefined
    result2 = sum(result1, n3); // crashes or returns a meaningless value.
    result3 = sum(result2, n4); // and so on
    console.log("Result is:", result3); // boom

has to resort to a callback style, that is, you must provide a function that will receive the primary function result as an argument when it becomes available, turns into this async code:

    var n1, n2, result1, n3, result2, n4, result3;
    sum(n1, n2, function(result1) {
        sum(result1, n3, function(result2) {
            sum(result2, n4, function(result3) {
                console.log("Result is:", result3); // Slowly descending into the "callback hell"
                });
        });  
    }); 

This is a trivial example, with only 2 levels of nested callbacks, but enough to explain what Trevor Burnham, author of Async Javascript calls "trouble in paradise" in the introduction of the book. You can easily figure that a more complicated workflow, with added levels of nesting - and forking - can easily twist one's mind (resemblance with Python's async library, Twisted Matrix is probably not a coincidence).

This is not to say that the async nature of Node event loop is a burden in itself, on the contrary, that async nature is what makes it able to continue processing requests while the callbacks wait for a response from a most likely IO bound procedure. The burden is imposed on the programming style, and so, over time a number of intended solutions emerged in order to alleviate this problem.

I really think that these things should be handled by the runtime, be it the language or its standard library, so one could be able to continue programming in the usual straightforward style, and the async oddities should remain under the hood, without the application programmer even having to think about them. But state of the art hasn't reached that point yet, even though some advance is being made in that direction, for instance:

• Javascript 1.7 generators - language level
• syncify.js - library
• streamlinejs - library, see also Asynchronous javascript - the tale of Harry

The present state of affairs shows solutions that, even though they leave the application programmer in asyncland, make her life much easier by introducing syntactic sugar in a dose enough to fulfill the following goals:

1. Write less code (in some cases, much less)

2. Make it prettier by reducing or even eliminating the levels of indentation typical of callback style - and yes, this is important -

3. As a consequence of the former two points, mind process of the programmer is greatly simplified, leaving room to think about the problem at hand and not being stuck in implementation details.

These solutions to asynchronous handling in general and control flow in particular can be broadly classified in two groups, although they not necessarily pursue the same goals:

• Promises based: Kris Kowal's Q, cujoJS when and the highly debatable for its non-standard implementation jQuery promises, to mention a few.

• Not promises based: Caolan McMahon's async and Tim Caswell's Step are two fine examples.

The list is by no means exhaustive, rather it reflects my personal preferences/experience. Consider studying/trying them by yourself, it will surely be a rewarding experience.

Async control flow and NSR

So far, so good. All of the previous can be put to good use in a NSR driven web app, as NSR is agnostic to the control flow tool which is used within its routing functions. In fact, I have done it myself this way when programming an online comics reading site, built with NSR and using async, or this RAR handling tool, in which I used when.

As I stated before, I really think that the language/framework/tool you use should provide some built-in utility to handle async methods, that is, if the tool relieves you from the "pyramid of doom", all the better. The thing here is that NSR must live to its promise: "no dependencies, only node.js and NSR and you're done", hence the decision to provide NSR with some async handling mechanism.

But which flavor should be chosen? Well, as probably many of the readers know, reinventing the wheel may not be advisable in certain enviroments, but eventually it may be highly educative, and more than once you get a ton of fun while you're at it. So the decision was made, NSR would be provided with:

• Async like methods, at least a subset.

• An implementation of Promises/A+

Async implementation

Async functions can be accessed through your instantiated router object by means of var async = router.utils.async;, or if you don't need Router functions, it can be accessed stand-alone like so:

    var async = require('node-simple-router').async; 
    //Either case, you can then invoke async functions:
    async.map(myArray, finalCb);

As has been said, async implements a subset of the functions present in Async, namely async.some, async.every, async.map, async.filter, async.reduce, async.waterfall, async.series and async.parallel

Obviously the base implementation from Caolan McMahon is much richer regarding the number of the methods it exposes as well as the quality of its inner workings, so by all means that's what you should use. The API for the methods exposed is exactly the same that in Caolan's , and regarding to why the API was "copied", all I have to say is that the original is so hugely popular, and rightly so, that it has become a spec in itself. NSR implementation source code resides in src/async.coffee

Promises implementation

Like in async case, promises can be accessed in two manners:

1. Through the router object: router.utils.defer, in which case you get defer(), a function to be invoked with no parameters which acts as a "Deferred object factory", that is, it retrieves an instance of a Deferred object, which in turn retrieves an instance of Promise object to which you can later add fulfill/rejection handlers, like so:

    var deferred = router.utils.defer();
    var promise = deferred.promise();
    promise.then(function (answerToAll) {console.log("Answer to all is:", answerToAll);});
    setTimeout(function() {deferred.resolve(42);}, 3000); // In 3 seconds we'll know the answer.

2. Requiring it as a stand alone:

   var deferred = require('node-simple-router/lib/promises').defer();
   // ...

In any case, all the details that are needed can be found at Promises A+ spec

Wrapping up

Both lib/async.js and lib/promises.js, when are invoked through the command line, for example: node lib/async.js, run a test that provides a neat listing of the current working directory. They look both exactly the same, though in the first case the implementation uses async.waterfall to "cascade" a group of async operations, and in the second case it does the same using promises as the control flow mechanism. It's interesting to examine the code to see what implementations look like, what they have in common and how they differ.