-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make expectSnapshot available in all functional test runs #82932
Conversation
6f898b0
to
f2adbb4
Compare
f2adbb4
to
6e95ee7
Compare
4fc3d38
to
7cee115
Compare
Hmm, I think I'll need some help with the types - looks like the Mocha types are conflicting with the Jest types. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks great, I plan to run this and try it out in some test locally later today but the code looks great.
packages/kbn-test/src/functional_test_runner/lib/mocha/load_test_files.js
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only thing I feel like I'm missing from this implementation of snapshots is the summary at the bottom of the test output that indicates how many snapshots were checked/failed/updated, etc. If you can produce some summary information that's tracked by the mocha reporter then you should be able to include information like that in the epilogue
@@ -141,6 +150,7 @@ export function runFtrCli() { | |||
--exclude-tag=tag a tag to be excluded, pass multiple times for multiple tags | |||
--test-stats print the number of tests (included and excluded) to STDERR | |||
--updateBaselines replace baseline screenshots with whatever is generated from the test | |||
--updateSnapshots replace inline and file snapshots with whatever is generated from the test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I definitely reached for the -u
flag here, maybe it would be nice to have an --update,-u
flag that sets both updateBaselines
and updateSnapshots
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done - I tried to add -u
only where necessary, but let me know if I missed anything.
@spalger I can't quite figure out how to collect stats. I'm not sure if I can get access to the reporter, or emit events that the reporter can use. Any clues where I should begin? |
@spalger Looks like the typings in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
packages/kbn-test/src/functional_test_runner/lib/snapshots/decorate_snapshot_ui.ts
Outdated
Show resolved
Hide resolved
@@ -4,7 +4,7 @@ | |||
"tsBuildInfoFile": "../build/tsbuildinfo/test", | |||
"types": ["node", "mocha", "flot"] | |||
}, | |||
"include": ["**/*", "../typings/elastic__node_crypto.d.ts", "typings/**/*"], | |||
"include": ["**/*", "../typings/elastic__node_crypto.d.ts", "typings/**/*", "../packages/kbn-test/types/ftr_globals/**/*"], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Went ahead and unified the global mocha and snapshot types under this directory in @kbn/test
and included them in both the test/tsconfig.json and x-pack/test/tsconfig.json projects.
💚 Build SucceededMetrics [docs]
History
To update your PR or re-run it, just comment with: |
Thanks for all the help @spalger! |
* master: (60 commits) Forward any registry cache-control header for files (elastic#83680) Revert "[Alerting] Add `alert.updatedAt` field to represent date of last user edit (elastic#83578)" [Security Solution][Detections] Fix adding an action to detection rules (elastic#83722) Make expectSnapshot available in all functional test runs (elastic#82932) Skip failing cypress test Increase bulk request timeout during esArchiver load (elastic#83657) [data.search] Server-side background session service (elastic#81099) [maps] convert VectorStyleEditor to TS (elastic#83582) Revert "[App Search] Engine overview layout stub (elastic#83504)" Adding documentation for global action configuration options (elastic#83557) [Metrics UI] Optimizations for Snapshot and Inventory Metadata (elastic#83596) chore(NA): update lmdb store to v0.8.15 (elastic#83726) [App Search] Engine overview layout stub (elastic#83504) [Workplace Search] Update SourceIcon to match latest changes in ent-search (elastic#83714) [Enterprise Search] Rename React Router helpers (elastic#83718) [Maps] Add 'crossed' & 'exited' events to tracking alert (elastic#82463) Updating code-owners to use new core/app-services team names (elastic#83731) Add Managed label to data streams and a view switch for the table (elastic#83049) [Maps] Add query bar inputs to geo threshold alerts tracked points & boundaries (elastic#80871) fix(NA): search examples kibana version declaration (elastic#83182) ...
* master: (60 commits) Forward any registry cache-control header for files (elastic#83680) Revert "[Alerting] Add `alert.updatedAt` field to represent date of last user edit (elastic#83578)" [Security Solution][Detections] Fix adding an action to detection rules (elastic#83722) Make expectSnapshot available in all functional test runs (elastic#82932) Skip failing cypress test Increase bulk request timeout during esArchiver load (elastic#83657) [data.search] Server-side background session service (elastic#81099) [maps] convert VectorStyleEditor to TS (elastic#83582) Revert "[App Search] Engine overview layout stub (elastic#83504)" Adding documentation for global action configuration options (elastic#83557) [Metrics UI] Optimizations for Snapshot and Inventory Metadata (elastic#83596) chore(NA): update lmdb store to v0.8.15 (elastic#83726) [App Search] Engine overview layout stub (elastic#83504) [Workplace Search] Update SourceIcon to match latest changes in ent-search (elastic#83714) [Enterprise Search] Rename React Router helpers (elastic#83718) [Maps] Add 'crossed' & 'exited' events to tracking alert (elastic#82463) Updating code-owners to use new core/app-services team names (elastic#83731) Add Managed label to data streams and a view switch for the table (elastic#83049) [Maps] Add query bar inputs to geo threshold alerts tracked points & boundaries (elastic#80871) fix(NA): search examples kibana version declaration (elastic#83182) ...
…ode-details * 'master' of github.com:elastic/kibana: fixed pagination in connectors list (elastic#83638) Forward any registry cache-control header for files (elastic#83680) Revert "[Alerting] Add `alert.updatedAt` field to represent date of last user edit (elastic#83578)" [Security Solution][Detections] Fix adding an action to detection rules (elastic#83722) Make expectSnapshot available in all functional test runs (elastic#82932) Skip failing cypress test Increase bulk request timeout during esArchiver load (elastic#83657) [data.search] Server-side background session service (elastic#81099) [maps] convert VectorStyleEditor to TS (elastic#83582) Revert "[App Search] Engine overview layout stub (elastic#83504)" Adding documentation for global action configuration options (elastic#83557) [Metrics UI] Optimizations for Snapshot and Inventory Metadata (elastic#83596) chore(NA): update lmdb store to v0.8.15 (elastic#83726) [App Search] Engine overview layout stub (elastic#83504) [Workplace Search] Update SourceIcon to match latest changes in ent-search (elastic#83714) [Enterprise Search] Rename React Router helpers (elastic#83718)
Closes #80292.
Haven't figured out a way to test everything as jest-snapshot sidesteps mocking of fs, added some smoke tests where possible.