Using --grep will call every defined beforeAll() in your suite/spec

[wburgess-datical]

When using --grep to delineate individual tests across a suite of multiple specs, each defined beforeAll() will be called, even those belonging to specs that do not contain a captured test. This is troublesome as it adds significant setup time to a test run that is unnecessary, and can lead to workflow corruption.

Note that superfluous beforeEach()s are not called (just the appropriate ones). Also note that this is directly related to issues experienced by users on #164, but that has been closed without response.

Ideally for each test that is captured via --grep, it will look upwards and run the corresponding beforeAll() and beforeEach(), run the captured test, then run afterEach() and afterAll(), then move to the next captured test and repeat. This will avoid running "extra" beforeAll()s.

I really love the --grep functionality and would love to see it better supported! The iit/xit/fit paradigm requires manually editing potentially hundreds of tests across a ton of files, which is tedious and does not fit well into fully automated CI. --grep (plus a suite of official tags) is my current approach and typically works very well, except for this one inconvenience.

Example:

spec01.js:

describe('Spec 1.1', function() {

    beforeAll(function() {
        console.log('1.1 beforeAll executed');
    });

    beforeEach(function() {
        console.log('1.1 beforeEach executed');
    });

    it('Test case 1.1.1', function() {
        console.log('1.1.1 test case executed');
    });

    it('Test case 1.1.2', function() {
        console.log('1.1.2 test case executed');
        expect(1).toEqual(1);
    });
});

describe('Spec 1.2', function() {

    beforeAll(function() {
        console.log('1.2 beforeAll executed');
    });

    it('Test case 1.2.1', function() {
        console.log('1.2.1 test case executed');
    });
});

-------------------------------------------------
spec02.js:


describe('Spec 2.1', function() {

    beforeAll(function() {
        console.log('2.1 beforeAll executed');
    });

    beforeEach(function() {
        console.log('2.1 beforeEach executed');
    });

    it('Test case 2.1.1 #RunThisTestOnly', function() {
        console.log('2.1.1 test case executed');
    });
});

Let's assume these files are captured in the "all" suite, and then run with: protractor conf.js --suite=all --grep="#RunThisTestOnly"

Output:

Spec started
1.1 beforeAll executed

1.2 beforeAll executed
2.1 beforeAll executed
2.1 beforeEach executed
2.1.1 test case executed

The main problem here is this is adding x-minutes of superfluous setup time to my test run. Also, in theory, if your beforeAll()s were written correctly, you shouldn't have conflicts...but it is entirely possible, and will require some babysitting.

Thank you and keep up the great work!

[juliemr]

I believe this is just a feature request for Jasmine (since we just forward to them for handling the test cases). Let me make sure and follow up.

[wburgess-datical]

Ahh right, my bad. Thank you!

[eileenzheng]

jasmine/jasmine#1473 There is a related pr in jasmine that attempts to address this issue, but not sure if it will be accepted.

I was wondering if there's something that can be done on protractor's side, such as making the grep excluded tests disabled instead of pending? @juliemr

[eileenzheng]

I'm wondering if we can change this line to
spec.disable();
And that may resolve the issue...?