Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: [M3-8517] - Cloud changes for component test CI #10926

Merged
merged 10 commits into from
Sep 12, 2024
Merged

test: [M3-8517] - Cloud changes for component test CI #10926

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
b11a504
Add component test Jenkinsfile
Sep 11, 2024
4fbd54b
Load fonts from index template, add `cy:component:run` command
Sep 11, 2024
09d3fc1
Add component test runner
Sep 11, 2024
31ebf9b
Fix region select component test
Sep 11, 2024
1b8dac6
Allow explicit suite name to be specified in JUnit report
Sep 11, 2024
eea4de4
Expose env, discard passed test recordings, and enable JUnit output f…
Sep 11, 2024
078267b
Pass env vars to container
Sep 11, 2024
785bc70
Enable JUnit Jenkins mode for component tests
Sep 11, 2024
dfe13c7
Set JUnit separator char
Sep 11, 2024
2612e1c
Add changeset, make slight improvements to Docker Compose docs
Sep 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Jenkinsfile-component-tests.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
library 'ui-builder'

testManagerComponents()
12 changes: 12 additions & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@ x-e2e-runners:
condition: service_healthy
env_file: ./packages/manager/.env
volumes: *default-volumes
# TODO Stop using entrypoint, use CMD instead.
# (Or just make `yarn` the entrypoint, but either way stop forcing `cy:e2e`).
entrypoint: ['yarn', 'cy:e2e']

services:
Expand Down Expand Up @@ -118,6 +120,16 @@ services:
<<: *default-env
MANAGER_OAUTH: ${MANAGER_OAUTH}

# Component test runner.
# Does not require any Cloud Manager environment to run.
component:
<<: *default-runner
depends_on: []
environment:
CY_TEST_DISABLE_RETRIES: ${CY_TEST_DISABLE_RETRIES}
CY_TEST_JUNIT_REPORT: ${CY_TEST_JUNIT_REPORT}
entrypoint: ['yarn', 'cy:component:run']

# End-to-end test runner for Cloud's synthetic monitoring tests.
# Configured to run against a remote Cloud instance hosted at some URL.
e2e_heimdall:
Expand Down
1 change: 1 addition & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
"cy:ci": "yarn cy:e2e",
"cy:debug": "yarn workspace linode-manager cy:debug",
"cy:component": "yarn workspace linode-manager cy:component",
"cy:component:run": "yarn workspace linode-manager cy:component:run",
"cy:rec-snap": "yarn workspace linode-manager cy:rec-snap",
"changeset": "node scripts/changelog/changeset.mjs",
"generate-changelogs": "node scripts/changelog/generate-changelogs.mjs",
Expand Down
5 changes: 5 additions & 0 deletions packages/manager/.changeset/pr-10926-tests-1726156342482.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@linode/manager": Tests
---

Support running component tests via CI ([#10926](https://github.com/linode/manager/pull/10926))
10 changes: 9 additions & 1 deletion packages/manager/cypress.config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,14 @@ export default defineConfig({
specPattern: './cypress/component/**/*.spec.tsx',
viewportWidth: 500,
viewportHeight: 500,

setupNodeEvents(on, config) {
return setupPlugins(on, config, [
loadEnvironmentConfig,
discardPassedTestRecordings,
enableJunitReport('Component', true),
Copy link
Contributor Author

@jdamore-linode jdamore-linode Sep 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abailly-akamai For the UI tests, we added the CY_TEST_DISABLE_FILE_WATCHING env var to prevent the Cypress UI from automatically stopping and re-running tests when it detects file changes.

Do you think that behavior should apply to the component tests as well, or do we want to keep file watching enabled since developing these tests is more iterative / the tests run quickly?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's okay to make the behavior apply to both.
As far as I know, I don't ever set CY_TEST_DISABLE_FILE_WATCHING. I like having my files be watched when developing 👀

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i do use CY_TEST_DISABLE_FILE_WATCHING, it's very helpful when working on resource intensive CY test and saving files often (rate limiting etc). Doesn't really apply as much for component tests as you mentioned, but applying the behavior to both makes sense to me as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to pose one last option: we can break this into 2 env vars so they can be configured independently, so @abailly-akamai you could keep file watching disabled for UI tests and enabled for component tests

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kinda prefer to have it control both so it's one less variable to remember (and also very few people seem to use it), but I can go either way

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense too -- there's already an overwhelming number of configuration options.

I'll make it apply to both, but this is easy to change so if it ever becomes a pain point we can always revisit 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I spoke too soon -- with component tests, it looks like the hot reloading is handled by Vite/our Vite config.

I think it could be changed dynamically using the CY_TEST_DISABLE_FILE_WATCHING env var, but the little bit of messing around I was doing with it started to get real hacky so I figure I'll just hold off unless this becomes a pain.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good with me!

]);
},
},

e2e: {
Expand Down Expand Up @@ -85,7 +93,7 @@ export default defineConfig({
regionOverrideCheck,
logTestTagInfo,
splitCypressRun,
enableJunitReport,
enableJunitReport(),
generateTestWeights,
]);
},
Expand Down
6 changes: 3 additions & 3 deletions packages/manager/cypress/component/poc/region-select.spec.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -397,9 +397,9 @@ componentTests('RegionSelect', (mount) => {
.should('be.visible');
});
regionsWithoutObj.forEach((region) => {
ui.autocompletePopper
.findByTitle(`${region.label} (${region.id})`)
.should('not.exist');
ui.autocompletePopper.find().within(() => {
cy.findByText(`${region.label} (${region.id})`).should('not.exist');
});
});
});

Expand Down
1 change: 1 addition & 0 deletions packages/manager/cypress/support/component/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width,initial-scale=1.0">
<link rel="stylesheet" type="text/css" href="fonts/fonts.css">
<title>Cloud Manager Components</title>
</head>
<body>
Expand Down
3 changes: 0 additions & 3 deletions packages/manager/cypress/support/component/setup.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,6 @@ import { ThemeName } from 'src/foundations/themes';

import 'cypress-axe';

// Load fonts using Vite rather than HTML `<link />`.
import '../../../public/fonts/fonts.css';

/**
* Mounts a component with a Cloud Manager theme applied.
*
Expand Down
51 changes: 33 additions & 18 deletions packages/manager/cypress/support/plugins/junit-report.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,27 +8,42 @@ const capitalize = (str: string): string => {
};

/**
* Enables and configures JUnit reporting when `CY_TEST_JUNIT_REPORT` is defined.
* Returns a plugin to enable JUnit reporting when `CY_TEST_JUNIT_REPORT` is defined.
*
* If no suite name is specified, this function will attempt to determine the
* suite name using the Cypress configuration object.
*
* @param suiteName - Optional suite name in the JUnit output.
*
* @returns Cypress configuration object.
*/
export const enableJunitReport: CypressPlugin = (_on, config) => {
if (!!config.env[envVarName]) {
const testSuite = config.env['cypress_test_suite'] || 'core';
const testSuiteName = `${capitalize(testSuite)} Test Suite`;
export const enableJunitReport = (
suiteName?: string,
jenkinsMode: boolean = false
): CypressPlugin => {
return (_on, config) => {
if (!!config.env[envVarName]) {
// Use `suiteName` if it is specified.
// Otherwise, attempt to determine the test suite name using
// our Cypress configuration.
const testSuite = suiteName || config.env['cypress_test_suite'] || 'core';

const testSuiteName = `${capitalize(testSuite)} Test Suite`;

// Cypress doesn't know to look for modules in the root `node_modules`
// directory, so we have to pass a relative path.
// See also: https://github.com/cypress-io/cypress/issues/6406
config.reporter = '../../node_modules/mocha-junit-reporter';
// Cypress doesn't know to look for modules in the root `node_modules`
// directory, so we have to pass a relative path.
// See also: https://github.com/cypress-io/cypress/issues/6406
config.reporter = '../../node_modules/mocha-junit-reporter';

// See also: https://www.npmjs.com/package/mocha-junit-reporter#full-configuration-options
config.reporterOptions = {
mochaFile: 'cypress/results/test-results-[hash].xml',
rootSuiteTitle: 'Cloud Manager Cypress Tests',
testsuitesTitle: testSuiteName,
jenkinsMode: false,
};
}
return config;
// See also: https://www.npmjs.com/package/mocha-junit-reporter#full-configuration-options
config.reporterOptions = {
mochaFile: 'cypress/results/test-results-[hash].xml',
rootSuiteTitle: 'Cloud Manager Cypress Tests',
testsuitesTitle: testSuiteName,
jenkinsMode,
Copy link
Contributor Author

@jdamore-linode jdamore-linode Sep 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't remember why we never set Jenkins mode to begin with, but it makes the output in the Jenkins run "Test" section a lot more readable. Sadly we can't just set this globally now that we have other teams like SRE who depend on our JUnit test results, so for right now this will only be set for the component test JUnit output

suiteTitleSeparatedBy: jenkinsMode ? '→' : ' ',
};
}
return config;
};
};
1 change: 1 addition & 0 deletions packages/manager/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@
"cy:e2e": "cypress run --headless -b chrome",
"cy:debug": "cypress open --e2e",
"cy:component": "cypress open --component",
"cy:component:run": "cypress run --component --headless -b chrome",
"cy:rec-snap": "cypress run --headless -b chrome --env visualRegMode=record --spec ./cypress/integration/**/*visual*.spec.ts",
"typecheck": "tsc --noEmit && tsc -p cypress --noEmit",
"coverage": "vitest run --coverage && open coverage/index.html",
Expand Down
Loading