Separately run Accessibility browser tests #1378
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
jonrohan
changed the title
jonrohan
changed the title
khiga8
approved these changes
Sep 16, 2022
| @@ -129,3 +128,25 @@ jobs: | |||
| bundle config path vendor/bundle | |||
| bundle install --jobs 4 --retry 3 | |||
| bundle exec rake coverage:report | |||
| system: | |||
| name: Accessibility | |||
There was a problem hiding this comment.
Should we just generally call this System or Browser? Though the a11y test, doc_examples_axe_test is taking the most time, there's more general component tests as well for testing behavior.
jonrohan
added a commit
that referenced
this pull request
Sep 16, 2022
* Separate run for slow tests * same version * run yarn * combine fast tests
jonrohan
added a commit
that referenced
this pull request
Sep 17, 2022
* Separate run for slow tests * same version * run yarn * combine fast tests
jonrohan
added a commit
that referenced
this pull request
Sep 17, 2022
…1319) * Create 0252-build-component-css-with-ruby-code.md * update per comments * Adding release canary npm workflow (#1359) * Adding canary preview * Ignore changeset * changeset ignore * actions permissions * update * remove name * ignore dependabot * on push * Setup preview deploys for pull requests (#1357) * Adding deploy preview script * change url to app name base * port 80 * setting key * not https * Not using this variable * clear up variables * set port? * variable updates * Fixing app_name * setup key * Wrong config * try passing master key var * setup params * fix typo * Don't run on changesets * use branches-ignore * setup paths to run on * Adding destroy workflow * Use docker image * Use env. * Renaming workflows to make more sense * We don't have the container tagged * Ignore changeset * make sure markdownlint sees the file * Bump @primer/css from 20.4.4 to 20.4.5 in /docs (#1360) Bumps [@primer/css](https://github.com/primer/css) from 20.4.4 to 20.4.5. - [Release notes](https://github.com/primer/css/releases) - [Changelog](https://github.com/primer/css/blob/main/CHANGELOG.md) - [Commits](primer/css@v20.4.4...v20.4.5) --- updated-dependencies: - dependency-name: "@primer/css" dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump @primer/css from 20.4.4 to 20.4.5 in /demo (#1361) Bumps [@primer/css](https://github.com/primer/css) from 20.4.4 to 20.4.5. - [Release notes](https://github.com/primer/css/releases) - [Changelog](https://github.com/primer/css/blob/main/CHANGELOG.md) - [Commits](primer/css@v20.4.4...v20.4.5) --- updated-dependencies: - dependency-name: "@primer/css" dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump @primer/css from 20.4.4 to 20.4.5 (#1362) Bumps [@primer/css](https://github.com/primer/css) from 20.4.4 to 20.4.5. - [Release notes](https://github.com/primer/css/releases) - [Changelog](https://github.com/primer/css/blob/main/CHANGELOG.md) - [Commits](primer/css@v20.4.4...v20.4.5) --- updated-dependencies: - dependency-name: "@primer/css" dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * demo preview small tweaks * Add Primer::Alpha::ToggleSwitch to the docs build task (#1364) * added toggleswitch to docs.rake, and rebuilt arguments.yml * added changeset * Remove pull step * Remove label.md from git * Bump @primer/gatsby-theme-doctocat from 4.0.0 to 4.1.0 in /docs (#1369) Bumps @primer/gatsby-theme-doctocat from 4.0.0 to 4.1.0. --- updated-dependencies: - dependency-name: "@primer/gatsby-theme-doctocat" dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Version Packages (#1356) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Delete the deprecated Primer::ButtonGroup component (#1372) * Removing deprecated ButtonGroup * docs: build docs * Create eight-pants-yell.md Co-authored-by: Actions Auto Build <actions@github.com> * Delete the deprecated Primer::BaseButton component (#1371) * Delete the deprecated Primer::BaseButton component * Updating docs link * Create friendly-rats-wait.md * Convert the rest of the primer storybook stories to view component previews (#1365) * Removing Deprecated Primer::Tooltip story * converting SpinnerComponent story to preview * Remove deprecated BlankslateComponent story * Primer::Box doesn't have controls * Primer::ButtonComponent is deprecated * Converting ClipboardCopy to preview * Removing value * Removing deprecated dropdown stories * Convert dropdown to preview * Setup menu stories * Delete FlexComponent deprecated stories * Convert HellipButton * Removing imagecrop stories * Remove label component * Delete animation stories * Converting LayoutComponent stories * Convert LinkComponent preview * Converting OcticonComponent * Converting ProgressBarComponent * Removing TabContainer stories * Convert SubheadComponent preview * Remove deprecated Truncate * Convert StateComponent preview * Convert Markdown preview * Convert TimelineItemComponent preview * Converting MenuComponent * Converting PopoverComponent preview * Put back stories until we remove storybook server * Demo lookbook (#1374) * add lookbook to demo * Updating assets and routes for docker * Removing javascript_importmap_tags * copy entire app folder * try docker_image * use tag argument * don't use lookbook unless rails is more than 7 * push all tags * if rails is greater than 7 * Setup IMAGE_TAG variable * Missed a quote * push all tags * try tag param * image.tags? * check rails_version * better if * check version * try manual push * or main * Does connecting in manifest fix path? * only js right now * connect to pvc css * Try pulling cache to speed up? * Fixing gem updates * pull from image_url ignore if it doesn't exist * switch to unless * only if Lookbook is defined * use application import * if defined * Remove lookbook params * load form paths * install postcss * Copy postcss mixins * nocov on routes config * Copy forms to docker file also * don't ignore forms Co-authored-by: Joel Hawksley <joel@hawksley.org> * Deleting the old lookbook folder (#1375) * Deleting the old lookbook folder * Add hotwire-livereload * Remove from gemfile * Fixing markdown headings * Setting env to test for rake tasks * Adding octicon option * Specify tag to deploy * just use latest * Adding lookbook links to docs output (#1376) * Adding lookbook links to docs output * remove component suffix * Making sure everything has a default * remove hidden default * Fixing remaining previews * Also remove storybook_url * exclude generated docs previews * Fixing auto_complete test * Separate run for slow tests (#1378) * Separate run for slow tests * same version * run yarn * combine fast tests * Run axe scanner on written previews in test/previews (#1377) * Running axe scan on written previews * put raise back * Can't run raise here * Ignore standalone * Run deprecated job as a test instead of ci job (#1379) * Run deprecated job as a test * Update component_test.rb * Use system name * Simplify CI configuration to a single build per Ruby/Rails version. (#1381) * Simplify CI configuration to a single build per Ruby/Rails version. Our existing CI configuration runs a multi-build matrix for the main test suite. I think this is overkill for our needs. In my experience, our matrix builds have mainly served us by testing each version of Rails and Ruby we support, not every combination of Ruby and Rails we support. I think we can get by with just 3 builds. * use matrix for test and benchmark builds * Create tricky-tomatoes-thank.md * Docs change to fix typo and descriptions in dialog.rb (#1382) * Update dialog.rb Fixed a small typo and what may have been missed copy pasta for the position & position_narrow attributes * Create neat-baboons-do.md Co-authored-by: Jon Rohan <rohan@github.com> * Accepted Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Joel Hawksley <joel@hawksley.org> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: River Lynn Bailey <river.bailey@testdouble.com> Co-authored-by: GitHub Design Systems Bot <30705008+primer-css@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Actions Auto Build <actions@github.com> Co-authored-by: Ru Levin <kangarruu@github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Separating out faster component tests from the system tests.
Currently all the tests are running together in the ci job, under a ruby and rails version matrix.
My goal was to speed up the matrix test builds by removing the accessibility tests into their own run. I don't believe these tests need to be run on every ruby version and rails version since they're looking at browser previews.