Implement per-test, not per-browser hooks

[chadmyers]

Are you requesting a feature or reporting a bug?

Requesting a feature

What is the current behavior?

Fixture hooks are run once per fixture, test hooks are run once per test per browser (if using the all option or something like chrome,firefox).

What is the expected behavior?

I'd like a third type of hook which runs once per test for all browsers. So if I'm running a fixture with 2 tests in it against chrome,firefox browsers, I'd like it to do:

• fixture setup
• test1 pre-browser setup hook
• test1 setup hook for Chrome
• test1 against Chrome
• test1 setup hook for Firefox
• test1 againt Firefox
• test2 pre-browser setup hook
• test2 setup hook for Chrome
• test2 against Chrome
• test2 setup hook for Firefox
• test2 against Firefox

[inikulin]

Hello @chadmyers,

We've had discussion before about this feature in context of #903, but decided to postpone it's implementation since where was no demand for it at the time. The question is should this hooks have access to TestController and thus to tested page or not (like fixture.before/after hooks)? Can you share your use case, please, so we'll be able to figure out basic capabilities of this feature?

BTW, for currently you can use the following workaround to achieve desired functionality:

function once(fn) {
    var executed = false;

    return function() {
          if (!executed) {
               executed = true;
               return fn.apply(this, arguments);
          }
    };
}


fixture `My fixture`
     .beforeEach(once(async t => {
           // ...
      }));

test
.before(once(async t => {
//...
}))
('My test', async t => {
//...
});

[chadmyers]

@inikulin Thank you for the response! I'm loving TestCafe, btw.

I'll definitely check out the "once()" trick. That may do what I need.

You asked for my use case: Our testing philosophy for UI tests (what we call "regression tests" since we follow the unit/integration/regression test hierarchy scheme) is that we reset the DB back to a known state before each test. So before each test, I want to run our "clear()" routine. The problem I ran into is that if 4 browsers are running the test simultaneously and they fire off clear() at various times (due to asynchronous execution) they can stomp on each other. So I'd like to run the clear once for all four browsers, run the test for all browsers, then move onto the next test.

The workaround I used was to do all the test setup in the fixture hook, but that means I can only have one test per fixture and we usually have more tests per fixture.

It's possible I may be bringing my prior-tool philosophy into this tool and that's why I'm having this difficulty, so feel free to tell me I'm "doing it wrong" and I'll readjust my understanding accordingly :)

Thanks again for a great tool!

[chadmyers]

I understand the conundrum about the TestController access in the "beforeAll" (or whatever it would be called) method. For my purposes, I do not need access to the browser in the beforeAll(), in fact, that would be the entire point of this method would be "Before each test, before the browsers are spawned". The before hook satisfies the "before each test, after the browser is spawned" use case very well.

[inikulin]

@chadmyers Thank you for the feedback. We need to figure out naming now. I wonder if test.beforeOnce and fixture.beforeEachOnce fits well.

[dapperdandev]

If I'm not mistaken, it looks like the once workaround above will cause everything in the fixture.beforeEach to run only once. I wasn't able to accomplish what I wanted with that solution. I was, however, able to implement a similar solution that worked really well for me:

fixture(`My Fixture`)
    .before(async ctx => {
        ctx.hasExecuted = false;
        // Server-side actions
    })
    .beforeEach(async t => {
        // Before each test

        if (!t.fixtureCtx.hasExecuted) {
            t.fixtureCtx.hasExecuted = true;
            // Client side actions to run once.
        }
    })

[AlexanderMoskovkin]

@djbreen7 The approach with fixtureCtx looks well.

[qualityshepherd]

I'm trying to use djbreen7's solution to login once and run multiple tests while logged in. Ie. behavior that would normally be handled in a beforeAll. Sadly, Testcafe seems to log me out between tests, and thus, this solution fails.

I tried using Roles as well... no joy.

Any ideas on how to work around this? I really just want to login once, then run a couple tests (same file).

Thanks!

[dapperdandev]

@qualityshepherd Do you have sample code somewhere? I've had a lot of luck with Roles.

[AlexKamaev]

@qualityshepherd
The Roles mechanism can help you solve this issue. If you have some problems with it, feel free to create a separate ticker using this form
We would really appreciate if you provide a sample project which demonstrates the issue.

[qualityshepherd]

Ahhh... figured out the problem I was having with Roles. I'm using page objects and had a signInAsRole method that I was calling t.useRole(admin) in... that was no bueno. it seems useRole must be called in the test or beforeEach to work properly (ie. re-use one login for multiple tests).

My solution is now to have a roles.js that I export various roles from and import that file into my tests and call useRole with reckless abandon within. Eg.

// roles.js
import { Role } from 'testcafe';
import signInPage from '../pages/signInPage';

export const admin = Role(signInPage.url, async t => {
        await signInPage.signIn('admin@example.com', 'password');
    }, { preserveUrl: true });

And then...

// test.js
import { admin } from 'roles';

fixture `Sign In`
    test('should sign in', async t => {
        await t.useRole(admin);
...

[miherlosev]

fixture.beforeAll
fixture.afteAll
or
fixture.beforeEachOnce
fixture.afterEachOnce

[alexschwantes]

Regarding the question of should the is should the Fixture .before and .after hooks have access to TestController, I would say that it would be very useful to have this.

Our example is as follows:

1. Say we have a test suite with 6 tests that test a particular function of the site. All of these tests require the same prerequisite.
2. Currently the prerequisite setup in the Fixture.beforeEach() which runs a series of steps to set it up. This is run for all 6 tests. This takes about 20 seconds for each test.

Now we could of course develop a custom hook to insert this into the database directly but this has other security implications. We could also go direct to the database, but again security issues and it means the test can't be run in production or controlled environments. Also the solutions suggested to abuse the .beforeEach() hook to run only once don't seem like they would work as the setup takes 20 seconds and you could enter a situation where other tests start running before the prerequisite has been setup.

Allowing .before and .after to have access to the TestController will save the execution time of 100 seconds for this fixture. When you add it up across all the fixtures, it does result in TestCafe running slower than had the tests been run in other frameworks that support this.

[qualityshepherd]

Please... I BEG of you... give me a beforeAll. PRETTY PLEASE.

But seriously, do.

[LavrovArtem]

@elimence, Thank you for your opinion.

[vsaetton]

Hello everybody. Do you have any news on this ticket?

Thanks in advance.

[miherlosev]

Hi @vicentesaettone09

Thank you for your inquiry. At present, the feature is not implemented.
The current development sprint is already planned. If this feature is important for you, the PR will be greatly appreciated.

[qualityshepherd]

With every test I write... I get more an more annoyed by the lack of a beforeAll/afterAll. It forces you to make bad decisions... do I write long, multi-assert tests or do I couple them. I always opt for the latter because the former is more problematic. I really want to see this on the roadmap.

[omartaghlabi]

Is there currently a workaround to login once at the fixture level, and execute multiple tests within that fixture?
I'm currently using the beforeEach which is not ideal as it adds significant time during test runs.

[qualityshepherd]

@omartaghlabi try out Testcafe's User Roles. They're a little janky (sorry guys, but #fact) but they do work. I have an example project where I use them. Take a peek and see if that helps...

[omartaghlabi]

I'll try that out. Thanks @qualityshepherd!

[AlexKamaev]

I researched the issue and created a separate testcafe-once-hook module, which allows you to emulate the required behavior.

Please check the following github repository to see how the module works: https://github.com/AlexKamaev/testcafe-once-hook-example

The testcafe-once-hook module exports two functions:

• oncePerFixture runs in the first specified browser before and after all tests in a fixture. Unlike the fixture.before and fixture.after hooks designed for server-side operations, oncePerFixture allows you to execute test actions in the browser.
• oncePerTest runs in the first specified browser before and after each test in a fixture, while regular beforeEach and afterEach hooks run in every browser.

[qualityshepherd]

Nice... will give it a go... only thing is I also need an onceAfterFixture for cleanup...

[qualityshepherd]

Oh... I get how you're working this now that I've looked at the tests...

[AlexKamaev]

I'll close the issue since now we have a separate module to support the described behavior. If there are any issues that cannot be covered by the module, feel free to describe them here.

[lock]

This thread has been automatically locked since it is closed and there has not been any recent activity. Please open a new issue for related bugs or feature requests. We recommend you ask TestCafe API, usage and configuration inquiries on StackOverflow.