Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(integ-runner): support config file #22937

Merged
merged 5 commits into from
Nov 17, 2022

Conversation

mrgrain
Copy link
Contributor

@mrgrain mrgrain commented Nov 16, 2022

Enables support for yarg's config file, replacing the current unused IntegrationTests.fromFile() method.

This also fixes a bug where --list would ignore test provided as args and via --from-file and just list all tests.


All Submissions:

Adding new Unconventional Dependencies:

  • This PR adds new unconventional dependencies following the process described here

New Features

  • Have you added the new feature to an integration test?
    • Did you use yarn integ to deploy the infrastructure and generate the snapshot (i.e. yarn integ without --dry-run)?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@gitpod-io
Copy link

gitpod-io bot commented Nov 16, 2022

@github-actions github-actions bot added the p2 label Nov 16, 2022
@aws-cdk-automation aws-cdk-automation requested a review from a team November 16, 2022 15:47
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Nov 16, 2022
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

@mrgrain mrgrain added the pr-linter/exempt-integ-test The PR linter will not require integ test changes label Nov 16, 2022
@aws-cdk-automation aws-cdk-automation dismissed their stale review November 16, 2022 16:26

✅ Updated pull request passes all PRLinter validations. Dissmissing previous PRLinter review.

@@ -75,6 +75,7 @@ export class WorkList<A> {

public done() {
this.remaining.clear();
this.stopTimer();
Copy link
Contributor Author

@mrgrain mrgrain Nov 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a bug currently where the runner would not finish immediately if no tests are found.
(It was only completing after the 60s timeout)

Comment on lines -93 to -169

describe('from file', () => {
const configFile = 'integ.config.json';
const writeConfig = (settings: IntegrationTestsDiscoveryOptions, fileName = configFile) => {
writeFileSync(fileName, JSON.stringify(settings, null, 2), { encoding: 'utf-8' });
};

test('find all', async () => {
writeConfig({});
const integTests = await tests.fromFile(configFile);

expect(integTests.length).toEqual(3);
expect(integTests[0].fileName).toEqual(expect.stringMatching(/integ.integ-test1.js$/));
expect(integTests[1].fileName).toEqual(expect.stringMatching(/integ.integ-test2.js$/));
expect(integTests[2].fileName).toEqual(expect.stringMatching(/integ.integ-test3.js$/));
});


test('find named tests', async () => {
writeConfig({ tests: ['test-data/integ.integ-test1.js'] });
const integTests = await tests.fromFile(configFile);

expect(integTests.length).toEqual(1);
expect(integTests[0].fileName).toEqual(expect.stringMatching(/integ.integ-test1.js$/));
});


test('test not found', async () => {
writeConfig({ tests: ['test-data/integ.integ-test16.js'] });
const integTests = await tests.fromFile(configFile);

expect(integTests.length).toEqual(0);
expect(stderrMock.mock.calls[0][0]).toContain(
'No such integ test: test-data/integ.integ-test16.js',
);
expect(stderrMock.mock.calls[1][0]).toContain(
'Available tests: test-data/integ.integ-test1.js test-data/integ.integ-test2.js test-data/integ.integ-test3.js',
);
});

test('exclude tests', async () => {
writeConfig({ tests: ['test-data/integ.integ-test1.js'], exclude: true });
const integTests = await tests.fromFile(configFile);

const fileNames = integTests.map(test => test.fileName);
expect(integTests.length).toEqual(2);
expect(fileNames).not.toContain(
'test/test-data/integ.integ-test1.js',
);
});

test('match regex', async () => {
writeConfig({ testRegex: ['1\.js$', '2\.js'] });
const integTests = await tests.fromFile(configFile);

expect(integTests.length).toEqual(2);
expect(integTests[0].fileName).toEqual(expect.stringMatching(/integ.integ-test1.js$/));
expect(integTests[1].fileName).toEqual(expect.stringMatching(/integ.integ-test2.js$/));
});

test('match regex with path', async () => {
writeConfig({ testRegex: ['other-data/integ\..*\.js$'] });
const otherTestDir = new IntegrationTests('.');
const integTests = await otherTestDir.fromFile(configFile);

expect(integTests.length).toEqual(1);
expect(integTests[0].fileName).toEqual(expect.stringMatching(/integ.other-test1.js$/));
});

test('can set app command', async () => {
writeConfig({ app: 'node --no-warnings {filePath}' });
const integTests = await tests.fromFile(configFile);

expect(integTests.length).toEqual(3);
expect(integTests[0].appCommand).toEqual('node --no-warnings {filePath}');
});
});
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed because fromFile() is not a thing anymore.

@@ -1,21 +1,26 @@
// Exercise all integ stacks and if they deploy, update the expected synth files
import { promises as fs } from 'fs';
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was no point in having the test list file being async

@mergify
Copy link
Contributor

mergify bot commented Nov 17, 2022

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 41c962f
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Contributor

mergify bot commented Nov 17, 2022

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 4f49efe into aws:main Nov 17, 2022
@mrgrain mrgrain deleted the mrgrain/feat/integ-runner-config branch November 17, 2022 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS. p2 pr-linter/exempt-integ-test The PR linter will not require integ test changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants