Tests: Try to migrate server and integration tests to Jest #14364

gziolo · 2017-05-23T10:46:36Z

This PR migrates server and integration tests from Mocha to Jest.

Integration tests

npm run test-integration
npm run test-integration:ci - version tailored for CircleCI
npm run test-integration:watch - run with watch mode enabled, by default it executes tests only for modified files

I skipped code coverage for integration tests, because it isn't important at all in this context.

Server tests

npm run test-server
npm run test-server:ci - version tailored for CircleCI
npm run test-server:coverage - run with code coverage enabled
npm run test-server:watch - run with watch mode enabled, by default it executes tests only for modified files

Summary

~~This also breaks other test runners because I modified the helpers from the /test/ folder to make it work with Jest. Mocha uses before, but Jest wants beforeAll :(~~

I followed this awesome article to make test helpers work with Mocha and Jest at the same time.

Code coverage result for `/server/` folder

TODO

Make skipped server test suites pass
Make client tests work with updated test helpers
Find a way to use mocha-junit-reporter like reporter on Circle CI
Find a way to distribute server tests between container on Circle CI
Add support for code coverage
Migrate integration tests to Jest
Distribute integration tests between CircleCi nodes

matticbot · 2017-05-23T10:46:37Z

Test live: ~~https://calypso.live/?branch=try/jest-server~~

markryall · 2017-05-25T01:47:00Z

Might it be possible to do this more cautiously by introducing jest alongside mocha and then incrementally migrate?

Perhaps we can use a mutually exclusive pattern for test filenames (like 'spec' instead of 'test')

We could introduce and discuss some new tests (specs!) that demonstrate use of jest features (such as snapshots), communicate these more broadly and then start the process for migration once the idea has momentum.

Codemods might then be used to get us the rest of the way.

gziolo · 2017-05-30T08:30:31Z

server/devdocs/test/index.js

-		} );
-
-		it( 'should return components stats only', done => {
-			getComponentsUsageStats( ( err, res ) => {


I'm wondering how it was passing before ...

It had empty body in the identical call in the previous test which calls the same function:

expect( res.body ).not.to.be.empty;

gziolo · 2017-05-30T09:59:47Z

circle.yml

@@ -27,8 +27,7 @@ test:
          - server/**/*.jsx
    - MOCHA_FILE=./test-results-client.xml npm run test-client -- -R mocha-junit-reporter -t $CIRCLE_NODE_TOTAL -i $CIRCLE_NODE_INDEX:
        parallel: true
-    - MOCHA_FILE=./test-results-server.xml npm run test-server -- -R mocha-junit-reporter -t $CIRCLE_NODE_TOTAL -i $CIRCLE_NODE_INDEX:
-        parallel: true
+    - npm run test-server:ci


With this line tests are executed only on node 0, which isn't what I would expect. This needs to be further investigated.

gziolo · 2017-05-30T17:12:01Z

@markryall thanks for sharing good ideas 👍

Might it be possible to do this more cautiously by introducing jest alongside mocha and then incrementally migrate?

I stumbled upon this article https://medium.com/@RubenOostinga/combining-chai-and-jest-matchers-d12d1ffd0303 this week. It helped me to alias Mocha specific lifecycle methods to make them work with Jest. I hope it allows to use Mocha syntax and Jest runner with the client side code, too :)

Perhaps we can use a mutually exclusive pattern for test filenames (like 'spec' instead of 'test')

We could introduce and discuss some new tests (specs!) that demonstrate use of jest features (such as snapshots), communicate these more broadly and then start the process for migration once the idea has momentum.

I don't like the idea of having two different file extensions, because it only makes everything more complicated. I like a lot the rest of suggestion though. It would be nice to be able to run the same tests with Mocha and Jest at the same time, to validate the transition. I'm still not sure if Jest is going to be faster because it initiates testing env for every test suite to ensure they are isolated. It runs tests concurrently, which might make it perform better locally, but it might be no go on Circle CI.

Codemods might then be used to get us the rest of the way.

Yes, they are very granular and have interactive mode. So you can migrate everything step by step.

markryall · 2017-05-31T05:00:17Z

@gziolo that's awesome - if we can just make existing mocha tests work with jest using this aliasing trick then that certainly makes the transition easier.

We'd still have to make the before vs beforeAll change everywhere wouldn't we? That could also probably be handled with some test config. Maybe the same for 'context'.

I'd had performance issues with jest when using it a couple of years ago but we had automocking enabled. I think the useMockery approach we already have for mocks is superior (although less succinct) - jest mocks seem needlessly complicated to me.

gziolo · 2017-05-31T07:04:05Z

test/setup-test-framework.js

+	return Object.assign( chaiMatchers, originalMatchers );
+};
+
+// Make sure we can share test helpers between Mocha and Jest


We'd still have to make the before vs beforeAll change everywhere wouldn't we? That could also probably be handled with some test config. Maybe the same for 'context'.

@markryall this is enough to make Jest work with Mocha helpers. We can add the same for context :)

gziolo · 2017-05-31T07:05:33Z

I'd had performance issues with jest when using it a couple of years ago but we had automocking enabled. I think the useMockery approach we already have for mocks is superior (although less succinct) - jest mocks seem needlessly complicated to me.

Fortunately they disabled automocking last year and introduced several other performance improvements.

We'd still have to make the before vs beforeAll change everywhere wouldn't we? That could also probably be handled with some test config. Maybe the same for 'context'.

There is also a codemod that is able to rename all before, after and context occurrences with what Jest likes.

gziolo · 2017-05-31T07:08:25Z

@blowery parallel and files in the CircleCI config file is going to work perfectly fine with Jest. Even if their version of glob fails and provides too many files, like we encountered it in the past, Jest will filter out all tests that don't match its own glob based matcher. 👍

gziolo · 2017-06-01T13:11:28Z

server/jest.config.ci.js

+module.exports = Object.assign(
+	{},
+	{
+		testResultsProcessor: './node_modules/jest-junit-reporter',


It was quite easy to integrate a reporter that is going to play nicely with Circle CI 🎉

gziolo · 2017-09-25T12:35:08Z

@gwwar I rebased with master, updated Jest to the latest version and fixed some new issues. Can you review the last 3 commits before this gets merged? :)

gwwar · 2017-09-25T23:17:38Z

@nb @gziolo are we good with moving forward with Jest again? If so, should we wait until they actually publish a version without the patent clause?

gziolo · 2017-09-26T08:40:53Z

We can wait, it may take a couple of days until they update repository. The patent clause was the only blocker. As far as I know, people are very excited about Jest.

gziolo · 2017-09-27T11:03:07Z

Okey, it is now rebased and updated to use Jest with MIT license 🎉

gziolo · 2017-09-28T14:56:20Z

@blowery @aduth it's about time to shed a tear, our custom test runner code for Mocha gets removed with 04a8861 :)

blowery · 2017-09-28T15:51:12Z

package.json

@@ -234,8 +234,6 @@
    "test-server:ci": "cross-env TEST_REPORT_FILENAME=./test-results-server.xml jest -c=test/server/jest.config.ci.js -w=2",
    "test-server:coverage": "npm run -s test-server -- --coverage",
    "test-server:watch": "npm run -s test-server -- --watch",
-    "test-test": "cross-env NODE_ENV=test NODE_PATH=test:client:client/extensions TEST_ROOT=test node test/runner.js",


blowery · 2017-09-28T15:51:20Z

package.json

@@ -269,9 +267,6 @@
    "jscodeshift": "0.3.30",
    "md5-file": "3.1.0",
    "mixedindentlint": "1.2.0",
-    "mocha": "3.1.0",
-    "mocha-junit-reporter": "1.12.0",
-    "mockery": "1.7.0",


…h IDE

gziolo self-assigned this May 23, 2017

gziolo requested review from markryall, coderkevin, gwwar and samouri May 23, 2017 10:46

matticbot added the [Size] XL Probably needs to be broken down into multiple smaller issues label May 23, 2017

gziolo added Testing [Tests] Includes Tests [Type] Janitorial [Status] In Progress labels May 23, 2017

gziolo force-pushed the try/jest-server branch 3 times, most recently from b55424e to 8f157da Compare May 24, 2017 05:04

gziolo force-pushed the try/jest-server branch 2 times, most recently from 1669c2f to 3c0d65c Compare May 30, 2017 08:28

gziolo commented May 30, 2017

View reviewed changes

gziolo force-pushed the try/jest-server branch 3 times, most recently from f9ec6bd to 8bba761 Compare May 30, 2017 09:26

gziolo commented May 30, 2017

View reviewed changes

gziolo commented May 31, 2017

View reviewed changes

gziolo force-pushed the try/jest-server branch from 60772ab to e1a7c99 Compare May 31, 2017 07:59

gziolo commented Jun 1, 2017

View reviewed changes

gziolo added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Jun 1, 2017

gziolo added [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. and removed [Status] Needs Rebase labels Sep 25, 2017

gziolo force-pushed the try/jest-server branch from a80eab1 to c01624e Compare September 27, 2017 07:07

gziolo force-pushed the try/jest-server branch 2 times, most recently from 0daac6a to 04a8861 Compare September 28, 2017 14:54

blowery reviewed Sep 28, 2017

View reviewed changes

gziolo added 14 commits September 29, 2017 07:37

Tests: Try using Jest for server tests

56ec4dc

Tests: Refactor all server tests files to avoid using useMockery

69773fb

Tests: Make Jest test runner work with Circle CI

7ea2f11

Tests: Remove unnecessary override for Jest assertion methods

da78af5

Tests: Speed up server pages test

4a85d85

Tests: Add jest run with coverage enabled for server tests

22f1fe1

Tests: Migrate integration tests from Mocha to Jest

aa49374

Tests: Move devdocs test suite to integration tests

ab54d12

Tests: Refactor Jest config files to make it possible to use them wit…

3340053

…h IDE

Tests: Enable parallel execution of integration tests

ff3e325

Tests: Migrate newly added tests to work with Jest

f380197

Tests: Fix failing api test by using Promises

ebc465d

Tests: Move server api tests to integration tests

b33f7f8

Tests: Perform cleanup after finished Jest migration

11be7ed

gziolo force-pushed the try/jest-server branch from 04a8861 to 11be7ed Compare September 29, 2017 05:38

gziolo merged commit 2459c50 into master Sep 29, 2017

gziolo deleted the try/jest-server branch September 29, 2017 05:57

matticbot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Sep 29, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tests: Try to migrate server and integration tests to Jest #14364

Tests: Try to migrate server and integration tests to Jest #14364

gziolo commented May 23, 2017 •

edited

Loading

matticbot commented May 23, 2017 •

edited by folletto

Loading

markryall commented May 25, 2017

gziolo May 30, 2017 •

edited

Loading

gziolo May 30, 2017

gziolo commented May 30, 2017

markryall commented May 31, 2017 •

edited

Loading

gziolo May 31, 2017

gziolo commented May 31, 2017

gziolo commented May 31, 2017

gziolo Jun 1, 2017

gziolo commented Sep 25, 2017

gwwar commented Sep 25, 2017

gziolo commented Sep 26, 2017

gziolo commented Sep 27, 2017

gziolo commented Sep 28, 2017 •

edited

Loading

blowery Sep 28, 2017

blowery Sep 28, 2017

Tests: Try to migrate server and integration tests to Jest #14364

Tests: Try to migrate server and integration tests to Jest #14364

Conversation

gziolo commented May 23, 2017 • edited Loading

Integration tests

Server tests

Summary

Code coverage result for /server/ folder

TODO

matticbot commented May 23, 2017 • edited by folletto Loading

markryall commented May 25, 2017

gziolo May 30, 2017 • edited Loading

Choose a reason for hiding this comment

gziolo May 30, 2017

Choose a reason for hiding this comment

gziolo commented May 30, 2017

markryall commented May 31, 2017 • edited Loading

gziolo May 31, 2017

Choose a reason for hiding this comment

gziolo commented May 31, 2017

gziolo commented May 31, 2017

gziolo Jun 1, 2017

Choose a reason for hiding this comment

gziolo commented Sep 25, 2017

gwwar commented Sep 25, 2017

gziolo commented Sep 26, 2017

gziolo commented Sep 27, 2017

gziolo commented Sep 28, 2017 • edited Loading

blowery Sep 28, 2017

Choose a reason for hiding this comment

blowery Sep 28, 2017

Choose a reason for hiding this comment

gziolo commented May 23, 2017 •

edited

Loading

Code coverage result for `/server/` folder

matticbot commented May 23, 2017 •

edited by folletto

Loading

gziolo May 30, 2017 •

edited

Loading

markryall commented May 31, 2017 •

edited

Loading

gziolo commented Sep 28, 2017 •

edited

Loading