JSON formatter specification

[charlierudolph]

Do we have a document anywhere that documents what we would like cucumber implementations to export in the JSON formatter?

In cucumber-js we were mainly following cucumber-ruby and I made a few changes in 2.0 (removing the "type" from scenarios as that is the only thing exported and converting the duration from nanoseconds to milliseconds) that I have received a few issues on

If we would like this behavior to be shared between the different cucumber implementations I would prefer we have a JSON schema somewhere that the implementations can test themselves against.

I'm happy to lead this and create an initial schema that is based off of the current output of the different projects.

@aslakhellesoy @jbpros @mattwynne

[mattwynne]

Briefly: see #172

The idea is to move away from the big blog of JSON that combines AST with test results, and use a series of smaller events instead. I'd like to deprecate then remove the JSON formatter from Cucumber Ruby ASAP.

If we can standardise around one protocol, we can build a single set of renderers / formatters that consume the protocol and produce human-readable output.

I'm glad to have you in the loop on this @charlierudolph.

[brasmusson]

I think the new style of the JSON formatter (which creates one JSON file - as opposed to the set of JSON event in the event protocol), should be base the (list of) pickles emitted by the compiler. This data need to be augmented with the data of the step definitions, hooks and results.

That being said, since there are tools parsing the current output of the JSON formatter, that option needs still to be available (for some time), but deprecated.

[mattwynne]

IMO the JSON formatter will be replaced by the event protocol, and we shouldn't need both in the end. I don't think there's any need to write a new JSON formatter.

[aslakhellesoy]

That being said, since there are tools parsing the current output of the JSON formatter, that option needs still to be available (for some time), but deprecated.

We could write a small utility that builds the legacy JSON report from events

[brasmusson]

We could write a small utility that builds the legacy JSON report from events

Yes, a formatter, because that is what a formatter is - a utility that build a report from events.

[brasmusson]

IMO the JSON formatter will be replaced by the event protocol

One thing that always made me a bit uneasy related to the discussion of "the event protocol" is the suspicion that we mix up concept/data/events belonging in different bounded context in the same bucket.

AFAIK "protocol" includes the concept of allowed sequence of things. Mentally I imagine massive parallel execution of pickles on a grid of executors. In such situation there could be 528 unmatched testStepStarting events when a testStepFinished event arrives, I'm not sure that it is reasonable to demand that every utility the receives events can handle such interleaving of events coming from parallel execution. That could be a use of a test case result json (with a corresponding event), so that the protocol for an executor is simple, pickle events are received and testCaseResult events are sent. The underlying primitive events could the be recreated, but serialized to some utilities can work with the illusion of single threaded execution.

[mattwynne]

@brasmusson interesting feedback. Would we be better having that discussion in #172?

[aslakhellesoy]

Closing this since the JSON format is being phased out

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.