Option to abort on first failure

[KittyGiraudel]

Current behavior:

All tests are run even if one fails.

Expected behavior:

Providing an option to abort the test suite as soon as a test fails.

How to reproduce the current behavior:

Make a test fail.

Environment

• Cypress Version: 0.19.2

[KittyGiraudel]

In the mean time, this does the trick:

Cypress.on('fail', () => Cypress.runner.abort())

[brian-mann]

In 0.20.0 this became:

Cypress.on('fail', () => {
  Cypress.stop()
})

[KittyGiraudel]

From what we’ve seen, Cypress.stop() exits the process with a 0 code, which means even if a test fails, this will make it behave like everything went fine. We believe Cypress.runner.abort() did exit with a non-0 code in case of a failure.

Are we mistaken?

[brian-mann]

I didn't try it, but Cypress.stop() does do more things than aborting just the runner.

Try this:

Cypress.runner.stop()

That's where the abort code moved to, but even this does a bit more than it used it. May work though.

[tomaswitek]

Any update here?
Both Cypress.runner.stop() and Cypress.stop() behave like everything went fine.

[brian-mann]

Nope. PR's are welcome.

[ThomasDecavele]

This is a much desired feature!

In the meanwhile I managed to get the workaround working though, here's what's in my "support/index.js":

import './commands'
import { OperationCanceledException } from 'typescript';

Cypress.on('fail', () => {
    Cypress.stop();
    throw new OperationCanceledException();
});

[nanoflat]

I think it will work in support/index.js, not in plugins/index.js

[01taylop]

The use-case I have for this is that I need my authentication tests to pass first - if they don't all subsequent tests will fail. Although the other tests will fail in the before hooks, it still slows things down unnecessarily (30 seconds per test file).

I have to name my test 1auth so that it runs first and whitelist a cookie which contains a token required for the other tests to pass.

Cypress.Cookies.defaults({
  whitelist: 'token',
})

I was hoping to do something like this in my auth tests to stop running tests on fail;

  before(() => {
    Cypress.on('fail', () => {
      Cypress.runner.stop()
    })
  })

It does stop the test runner, but it doesn't fail any tests so I get 3/3 tests passed and a green build which is not the intended outcome. I also tried adding am intentional failing test before stopping cypress, but then the test runner continues.

[danh91]

I have a temporary solution that I think can help someone.

I just add an after each in my utilities file imported by all my specs.

afterEach(function() {
  if (this.currentTest.state === 'failed') {
    Cypress.runner.stop()
  }
});

I take advantage of Mocha offering a way to check the state of the last test.
And the nice part is that the process exit with a failing code (not 0)

[sanderdejong88]

@danh91 I've been following this thread for months, and finally someone found a way! Awesome!
I've put your code in the support file which also seems to work.

[harvitronix]

This no longer works for me with Cypress 3.x.

[jennifer-shehane]

@harvitronix We changed how specs are run in 3.0 so that the files are run isolated. This means the before runs before each spec file - so I am guessing that it is stopping and failing that 1 spec file and then continuing to the next file. Is this what is happening?