Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: replace Jasmine with Jest as the testing framework #847

Merged
merged 35 commits into from
Jan 13, 2022

Conversation

0xJuancito
Copy link
Contributor

@0xJuancito 0xJuancito commented Jan 10, 2022

Description

Replace Jasmine testing framework with Jest in order to improve the development experience, and allow features like global setup and teardown functionalities, as well as parallel processing.

Changes

  • Remove Jasmine and replace with Jest
  • Use ts-jest library to support Jest with TypeScript
  • Add option to silence logs to the console to improve readability
  • Create global setup and teardown for Docker containers in integration tests

.vscode/extensions.json Outdated Show resolved Hide resolved
@menduz menduz self-requested a review January 10, 2022 18:21
import { E2ETestEnvironment } from './test/integration/E2ETestEnvironment'
import { isCI } from './test/integration/E2ETestUtils'

export default async (): Promise<void> => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use named exports

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reading the whole PR, I'd like to suggest this:

Wouldn't it be better to be more in control and use a high order function to test with a DB? These hooked files add a lot of magic and force us to have to separate unit and integration tests. Making coverage reports very hard to unify.

export function describeWithDb(nameOfSuite: string, testFn: () => void) {
  describe(nameOfSuite, () => {
    it('initialize db', () => ...)
    testFn()
    it('teardown db', () => ...)
  })
}

With that function, the only thing needed would be to use describeWithDb instead of describe

describeWithDb('my test', () => {
  it('does something with the DB', () => ...)
})

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise I'd suggest to move it back to the E2ETestEnvironment to not scatter the logic around many files handled by runners, that makes the debugging and understanding of the tests flow hard to follow

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change only applies to development env, preventing E2ETestEnvironment to spin up and delete a container every test, but not for initializing the DB.
DB initialization is still happening in E2ETestEnvironment where it creates a database and run the migrations.

Also there is no problem to unify unit and integration tests, this change only ensure you have a postgres instance up and running, which is the same behavior as CircleCI having a service for postgres.

After this change:

  • setup postgres instance
  • run all tests (every test uses its own database)
  • tear down

Before this change:

  • for each test
    • start a container
    • run test (uses its own database)
    • delete container
      This is causing a lot of noise when containers are not available or failing tests not deleting containers.

I agree if you are talking about removing E2ETestEnvironment, but it means a lot of refactor, probably worth to think it in another PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I completely agree on unifying coverage reports. On that line, I introduced Jest projects here. We can have a single config file for both unit and integration tests and have specific configurations for each. If we run them together, it will produce a unified coverage report.

A PR for unifying content tests with parallelism on the CI is being developed here: #850

loggerMock.log = () => {}
loggerMock.warn = () => {}
logComponentMock.getLogger = jest.fn(() => loggerMock)
return logComponentMock
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not ultimately necessary, but currently our CI logs are bloated with unnecessary data, making them hard to read like here. I believe this could be mitigated in the past by setting the LOG_LEVEL env variable, but I understand the current logger implementation is lacking that capability.
So, the point of these changes is to improve readability of the tests by mocking the current logger.

What would you suggest in this case?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd sacrifice readability in the console runner to benefit debug capabilities. Now that every test is passing bloated with unnecessary data is a valid argument, but when you are trying to fix a hairy bug then you really need those logs.

I'd make it optional with an env var to enable local debugging with logs or add the log level env var to here https://github.com/well-known-components/logger/blob/2a1765ec3a3b5236c691f91a6083a9c5d1745694/src/helpers.ts#L12 if that suffices your use case

logComponentMock.getLogger = jest.fn(() => loggerMock)
return logComponentMock
})
jest.spyOn(process.stdout, 'write').mockImplementation(() => true)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this necessary also?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the same line as with the logger mock changes, this change is not ultimately necessary, but it aims to remove console logs done in the tests and improve readability of the CI logs.
What would you suggest in this case?

content/tsconfig.json Outdated Show resolved Hide resolved
jest.config.json Outdated Show resolved Hide resolved
commons/jest.config.json Outdated Show resolved Hide resolved
@0xJuancito 0xJuancito force-pushed the test/replace-jasmine-with-jest branch from e9900ea to 92b3b9c Compare January 13, 2022 12:36
@0xJuancito 0xJuancito merged commit ad8fbd9 into main Jan 13, 2022
@0xJuancito 0xJuancito deleted the test/replace-jasmine-with-jest branch January 13, 2022 12:57
@menduz
Copy link
Member

menduz commented Jan 13, 2022

🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants