question: does dredd support chain-calls (testing flows)?

[ducin]

I'm working on a frontend that will rely on a big corporate API and I'm considering using dredd.

The API supports flows. A flow is an operation divided into few steps, each having its own view and a corresponding AJAX call. The execution is based on promises. For example: creating a user would take 3 ajax calls that have to be done in a specific order (the server stores session data, which is used to combine next step as a continuation of the previous one). Result from one step will be used as parameters for another one (and it is matched on server-side). You can't succeed in any different order (e.g. start from the second step).

My question is: does dredd support such AJAX request chain? Can I tell dredd that "create user" consists of 3 steps and output of N-step should be the input for N+1? Or, more generally, is there any way to customize dredd to make it aware that REST resources are related to each other somehow? My case is that resources are not strictly "plain". There are some resources (which I want to have tested automatically of course), which are available only after another resource has been fetched/created/started.

If there's no such built-in mechanism, are there any hooks or any methods to override so that I can provide such behavior?

[netmilk]

Hi,

this is an interesting one :)

Generally, at this moment Dredd is more focused on isolated testing of each entry in the blueprint (resource action examples) and hooks on to serve for doing setup and teardown between each transaction. It's because API Blueprint is designed as an API reference documentation format and it respects REST paradigm in its DSL. So I'm thinking about each action example as as it is a unit.

I'm totally in for support of workflow oriented documentation and testing this workflows with Dredd, but it will need support in API Blueprint syntax. It's discussed here in the API Blueprint project.

But if you want to just pass data from one transaction to another, you can use response stash and modify request in the before hook. You can use same pattern for mixing in header with session data. You can also organise your blueprints in separate files and then run them all by specifying multiple paths with -p switch or by a glob. I hope it will be useful in this case.

[ducin]

Again, thank you for your response, Adam.

I was kinda afraid that you'll write that the tests are isolated due to REST nature. But the real world is more complicated than REST which seems a little utopian. In big systems you have to separate a process into steps and this breaks REST philosophy willy-nilly. And since such big interfaces are SPA-based, they use kinda-REST, which is not fully supported. And that's a big pitty.

It will take me some time until I figure out whether this response stash thing or multiple files approach suits my needs. But it would be a great thing if API blueprint supported related resource - it would become the ultimate docs & test solution for enterprise interfaces.

[netmilk]

@tkoomzaaskz, thanks a lot for valuable feedback! Here in the dredd-example repo is an example how to pass data between transactions in the hooks - from previous response to next request.

[netmilk]

Hi,
I'd like to demonstrate one possible approach how to test chain-calls. As I mentioned above, main objective of API Blueprint is to describe REST APIs. But on the other hand, you can use API Blueprint's simple syntax for writing down any HTTP calls.

For example you have following blueprint in blurpint.md:

FORMAT: 1A

# My scenario

## POST /login

+ Request (application/json)

        {  
          "username": "john",
          "password":"d0e"
        }


+ Response 200 (application/json)

        {
            "token": "s3cr3t"
        }

## GET /cars

+ Response 200 (application/json)

        [
          {
            "id": "42",
            "color": "red"  
          }
        ]

## PATCH /cars/{id}

+ Parameters
  + id (string,required, `1`)

+ Request

        {
          "color": "yellow"
        }

+ Response 200

        [
          {
            "id": 42,
            "color": "yellow"  
          }
        ]

Now, get transaction names for hooks from Dredd CLI:

$ dredd blueprint.md http://localhost:3000 --names
info: Beginning Dredd testing...
info:  > /login > POST
info:  > /cars > GET
info:  > /cars/{id} > PATCH
complete: 0 passing, 0 failing, 0 errors, 0 skipped, 3 total
complete: Tests took 4ms

Create a hook file hooks.coffee using these names:

hooks = require 'hooks'

# setup response stash
stash = {}

hooks.after ' > /login > POST', (transaction) ->

  # stash transaction response for later usage
  stash[transaction.name] = JSON.parse transaction.actual

hooks.before ' > /cars > GET', (transaction) ->

  # add retrieved authentication token to headers
  loginResponseBody = stash[' > /login > POST']['body']
  transaction.request.headers['X-Api-Key'] = loginResponseBody['token']

hooks.after ' > /cars > GET', (transaction) ->

  # stash transaction response for later usage
  stash[transaction.name] = JSON.parse transaction.actual

hooks.before ' > /cars > GET', (transaction) ->

  # add retrieved authentication token to headers
  carsResponse = stash[' > /cars > GET']['body']

  # replace id in URL with ID retrieved in previous response body
  newId= carsResponse.body[0].id
  transaction.request.url = transaction.request.url.gsub(1, newId)

Finally run Dredd with hooks:

$ dredd blueprint.md http://localhost:3000 --hookfiles ./hooks.coffee

Please let me know if this works for you and if does it fit your needs for testing of chained AJAX calls.

[ducin]

Wow, thank you very much for this extensive tutorial, Adam! Great work!

The entire API blueprint stack is in the roadmap of the project I'm designing architecture for, but I'll need to do the blueprint/dredd research as soon as possible, so again, thank you. I need to decode it from CoffeeScript first:

var hooks, stash;

hooks = require('hooks');

stash = {};

hooks.after(' > /login > POST', function(transaction) {
  return stash[transaction.name] = JSON.parse(transaction.actual);
});

hooks.before(' > /cars > GET', function(transaction) {
  var loginResponseBody;
  loginResponseBody = stash[' > /login > POST']['body'];
  return transaction.request.headers['X-Api-Key'] = loginResponseBody['token'];
});

hooks.after(' > /cars > GET', function(transaction) {
  return stash[transaction.name] = JSON.parse(transaction.actual);
});

hooks.before(' > /cars > GET', function(transaction) {
  var carsResponse, newId;
  carsResponse = stash[' > /cars > GET']['body'];
  newId = carsResponse.body[0].id;
  return transaction.request.url = transaction.request.url.gsub(1, newId);
});

As far as I can see, each AJAX call chain would have its own separate hooks file and what you're doing is registering resource-call-callbacks, right? I mean - you are creating something analogical to a deferred that you register .then() callbacks to execute next AJAX calls and so on and so on and when the first AJAX is fired, the chain is processed. Is that more or less what is happening here?

The way I imagine testing API with Dredd is creating directories like test/dredd/* that would consist of js/coffee scenarios (implementing hooks), where dredd command line tool is the actual test runner. Does that make sense?

[netmilk]

@tkoomzaaskz Dredd command is testing data from API blueprint (HTTP tranasctions), and hooks are used to pass data from real responses to next requests. I'm really interested in what are you doing and I'm trying to achieve it. I'm available for a skype call if you want to discuss it, anytime.

[LinusU]

@tkoomzaaskz I'm trying to accomplish something very similar and have gotten a bit on the way. Check out LinusU/mocha-api-blueprint-example for something that actually works right now.

You use case is very simliar to what I've done in Usage.apib and which is then automatically tested in test/rest.js.

[netmilk]

Hi @LinusU,
I'm sorry, but I have to repeat, that all functionality needed to achieve this is already present in Dredd and re-implementing Dredd in Mocha is in my opinion very hard and time consuming and it still doesn't make lot of sense for me. Can you please articulate the motivation behind this your effort and what you can't do using Dredd and you can with your approach?

Thanks a lot!
🍬

[LinusU]

Hi @netmilk,

No problem, I've been moving very fast forward to try and land on what I want so it's been a bit all over the place :)

1. Stricter validation of the values returned from my REST API.

This is a snippet of code from Usage.apib. The idea here is to both generate a "quick-start" tutorial that my users can follow, and it should also be tested.

## /messages
### Shout your message to the world [POST]

Now that you are logged in your are free to post messages for others to see.

+ Request
  + Headers
    Content-Type: application/json
  + Body
    { "text": "Listen to me!" }

+ Response 201
  + Headers
    Content-Type: application/json
  + Body
    { "id": "{messageId}", "text": "Listen to me!" }

### Listen in on the conversation [GET]

Listen to the conversation by retreiving all the messages.

+ Response 200
  + Headers
    Content-Type: application/json
  + Body
    [
      { "id": "{messageId}", "text": "Listen to me!" }
    ]

As you can see I have specified "{messageId}" in two places, this is actually validated to be the same. All the other values are checked with deepEqual which ensures that it has the exact same values, instead of only being the same type as dredd currently does.

2. I want to send my request myself

I would like to send my request away to the server myself, and then be responsible for passing the answer back. As I have made it in my repo I get to specify the function for doing this.

The signature for that function looks like this: (transaction, done). It gets the transaction, with .response, .request, and a function to call when it's done. It also provides a function for validating the response, suite.validateResponse(actual, expected).

This can be seen in lib/rest.js line 14-20.

What is really cool with this approach is that I don't actually have to send the request over HTTP. More often than not the server is being run in the same process as the tests and I can just pipe them right in, without hitting the network stack. I can also skip the serialization and deserialization of the request body.

This brings potentially huge speed boosts to the tests (I just did this for multer and we went from 5 seconds to 0.1 seconds), which is really helpful when working with test driven development.

3. I would like it to integrate with my test framework of choosing

This is a very nice to have since it lowers the barrier for people who already are using a test framework. mocha is wildly popular in the Node.js community but I don't at all think that it should limit itself to one framework.

I want dredd to be a strong foundation which it should be easy to click it in to other frameworks, but it shouldn't at all require them.

This could either be accomplished by shipping glue for the most popular frameworks, or exposing good enough hooks so that it's easy accomplished.

Here is how I did it in my tests. Basically it has to call describe for every ResouceGroup and Resource that it enters, and call it for every Action. Below is the beautiful output that it yields, do note that this is 100% generated from the API Blueprints Spec.apib and Usage.apib.

I hope that this sheds some light over what I want. I'm very open to contributing code and having good discussions.

Here are some related pull requests that I have submitted: apiaryio/protagonist#69, apiaryio/api-blueprint#198, apiaryio/snowcrash#332.

[LinusU]

Hi @netmilk, if you could give some thoughts on the three points I provided it would really help me to move forward.

Thanks 👍

[netmilk]

@LinusU answered here in #192

[netmilk]

Just a little remark: Example for handling session in hooks was added to the documentation

[netmilk]

Tracking now in #358