How to skip a test by Helper / hooks?

[zordius]

What are you trying to achieve?

I try to create a helper to help me to skip some test cases for specific browser, but I can not find document about how to skip a test case. Any suggestion?

• So far I can use beforeStep (step) to know what's the next step and the arguments, but I don't know how to skip the step (or all the steps) . <= this maybe anoterh way to help me skip "all steps in a test case" , not so good but may work.

• Or, refer to Add a way of a programmatically skipping a test mochajs/mocha#1901 , is there any similiar api in codeceptjs?

[APshenkin]

@zordius
Hi!
Unfortunately there is no ability to skip tests from scenario code now. We use promise chain for scenarios, so async method this.skip is not working as expected for it.
Maybe somebody will help us with it (or after some research I will implement it later)

But now you can use grep feature to mark tests, that you want to run

You should place your test for different browsers in different files and add tags in Feature (e.g. @chrome, @firefox, @all, etc.)
And then use this tags as filters

`codeceptJS run --grep='@chrome|@ALL'

[zordius]

Ya, we already adopt this practice, but the practice can just skip a test file and I am looking for a way to control test case more programmatically.

Here are some possible designs:

• A new recorder API to insert a CANCEL action BEFORE the queue as first action.
• A way to let _before() function to do CANCEL, use the previous API.
• something like options.filter = function (testCase) {} in config to filtering test cases by any logic <= this may be more easy

[thiagodp]

Maybe using decorators to tag features and scenarios is a interesting solution. Other tags, such as the mentioned @firefox, @chrome, etc., could be added too.

[alexperezmmt]

Hi,
I know that I arrive more than a year late, but I found a workaround for it.
If in your hook/helper you want to skip a test, I override the function run as

test.run = function skip() {
          this.skip()
        }

And it skipped the test.
Is not the most elegant way, but it works

[mirao]

I'm using this for skipping of scenario (e.g. when the scenario is unreliable in Firefox):

if (require("codeceptjs").config.get().helpers.WebDriver.browse == "firefox") {
    return;
}

It's placed at the begin of Scenario(). Of course you can have a "shortcut" method for that in a step object's/helper method if you want to call in more scenarios.

[Vorobeyko]

I think we should create method like SkipIf(() => 2 > 2).Scenario

[ryust-ax]

Hi,
I know that I arrive more than a year late, but I found a workaround for it.
If in your hook/helper you want to skip a test, I override the function run as

test.run = function skip() {
          this.skip()
        }

And it skipped the test.
Is not the most elegant way, but it works

Where/How do you add this? I'm familiar with Page Objects and Helper (extend codeceptjs.helper) but I'm not yet familiar with building hooks.
I'm trying to skip a BDD/Gherkin setup but haven't found a solution that doesn't involve reverting from using Given/When/Then style and back to Scenario style.

[themarcelor]

@ryaxc , you just need to register a script to define your hooks and capture events. Just add the following block to your codecept.conf.js:

  hooks: [
    'hooks/my_hooks.js',
  ],

and, inside the hooks/my_hooks.js' file, add some code like this:

const { event } = require('codeceptjs');

event.dispatcher.on(event.test.before, function (test) {
    if (test.tags.includes('@skip') {
      test.run = function skip() {
        this.skip()
      }
    }
});

Hope that helps.

But it would be nice to introduce a proper SkipIf() function like @Vorobeyko was suggesting.

[ngraf]

Consider this as a workaround as well:

(skipConditionIsMet ? xScenario : Scenario) ('title', async () => {
     // test code
})

It's a bit ugly to read, but it works as expected.

[idxn]

@ngraf This is good and I prefer to this way :)

[shemeshsapir]

Does anyone know how do I skip an entire feature/suite in codeceptjs?
I tried this way below - but it is not enter to the event listener at all

event.dispatcher.on(event.suite.before, async (suite) => {
if(suitesSet.has(${suite.title})){
xFeature(suite.title);
}
});

[kobenguyent]

I guess just simply add xFeature then the whole Scenarios inside that feature would be skipped tho.

[shemeshsapir]

I guess just simply add xFeature then the whole Scenarios inside that feature would be skipped tho.

Thanks for the answer..I'm trying to do it dynamically - in run time
which means that I want to do the skip through the event listener.
but from some reasons .I'm even not enter to the event.

[ryust-ax]

I wound up doing the following:

In codecept.conf.js

exports.config = {
  hooks: ["./hooks/ccjs-skip.js"],
};

In hooks/ccjs-skip.js

function decideSkip(tags) {
  if (testTags.includes("@skip")) {
    return true;
  }
}

module.exports = function () {
  event.dispatcher.on(event.test.before, function (test) {
    const skipThisTest = decideSkip(test.tags);

    if (skipThisTest) {
      test.run = function skip() {
        this.skip();
      };
      return;
    }
  });
};

Then for bdd feature files

Feature: Some Feature Defined
  In order to get some goal
  I want to do some task
 
  @skip
  Scenario: Do some task
    Given I start somewhere
    When I do something
    Then I see result

And for test files

Scenario("Do a test @skip", async ({ I }) => {
  await I.doSomethingHere();
});

I've got some additional output.log or other logging going on inside my hook, but the above is essentially how it's working for me.
I also have some more sophisticated checks inside or with helper methods used by decideSkip(); in my case I'm also leveraging a feature-flag system to give some external control based on what sort of @flagValue I add to my scenarios.

[shemeshsapir]

test.run = function skip() {
        this.skip();
      };

Hi,Thank you!
I actually tried this way , the problem is that when I run with multiple threads,
its not works well , in addition, I'm looking on the test's status in 'allure' and the status of those tests that skipped, are not clearly .. the state it not really "skipped" it's 'unknown' as I saw.

[ryust-ax]

test.run = function skip() {
        this.skip();
      };

Hi,Thank you! I actually tried this way , the problem is that when I run with multiple threads, its not works well , in addition, I'm looking on the test's status in 'allure' and the status of those tests that skipped, are not clearly .. the state it not really "skipped" it's 'unknown' as I saw.

Good to know! I hadn't yet started running multiple threads but that's a near-term goal. I've started dabbling with writing some custom helpers extending the built-ins (Playwright so far), I wonder if there's something with skip() that needs an enhancement.

[mirao]

I tried using #661 (comment) for skipping of a Feature for a selected browser, but it works in run mode only and doesn't work in run-multiple mode.
#661 (comment) works in both modes.

[idxn]

@mirao which one does work in both modes? I'm a bit confused. You stated the same 661.

[mirao]

@idxn The comments have the same name, but refer to different links. It's a GitHub feature :). Just try to click ...
I just wanted to say that the @ngraf 's solution works in both modes (run and run-multiple).

[ryust-ax]

I don't know if this is as good as @ngraf or how it would perform with the different run styles, but I have iterated to also use the following custom helper:

const { output } = require("codeceptjs");
const Helper = codeceptjs.helper;

class SkipperHelper extends Helper {
  _before(test) {
    this.currentTest = test;
  }

  /**
   * Who knows when you want to do something with the test
   *
   */
  getMyTest() {
    return this.currentTest;
  }

  /**
   * Usage:
   *    await I.skipTest();
   *    return;
   * Or:
   *   return I.skipTest();
   * 
   * Use the "return;" if adding a conditional execution above a block of code
   * This code will not immediately interrupt the full test scenario and is
   * ideally used only in code-based Scenario() instead of the BDD step
   * definitions.
   */
  async skipTest() {
    try {
      if (!this.currentTest.tags.includes("@skip")) {
        this.currentTest.tags.push("@skip");
      }

      // eslint-disable-next-line func-name-matching
      this.currentTest.run = function skip() {
        this.currentTest.skip();
      };

      await this.currentTest.skip();
    } catch (error) {
      output.log("Working around non-programmatic skip: ", error);
    }
  }
}

module.exports = SkipperHelper;

Then in codecept.conf.js

exports.config = {
  /* ... */
  helpers: {
    /* ... */
    Skipper: {
      require: "../ui/helpers/skipper.js",
    },
    /* ... */
  },
  /* ... */
};

Then in a tests/file.test.js:

/* import, injects, dotenv constants including APP_ENVIRONMENT import/assignment */

Scenario(
  "Execute a test @someRelevantTag",
  async () => {
    if (!APP_ENVIRONMENT.includes("staging")) {
      await I.say(`Skipping test.  Being executed on environment other than staging [${APP_ENVIRONMENT}]`);
      return I.skipTest();
    }
   /* do some testing if not staging */
  }
);