-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Automate WCAG testing #987
Conversation
- add github action to automate wcag test on release, hotfix, support, and develop branches - add axe-core/puppeteer to repo, and run analysis on all pages used in percy snapshot tests J=1644 TEST=manual/auto - ran `npm run wcag` with puppeteer's browser headless mode disabled, see that all pages are loaded and analyzed by axe-core, see that the violation results are console log out.
} | ||
|
||
|
||
async _analyzeUniversalSearch() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems to mimic the exact flow of the Photographer
class. I think we should do some re-factoring here. For both classes, we go to the same pages, run the same queries, etc. If we add a new page or want to run a new query, we will have to update both files.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wasn't sure if we would always test the same page flow between percy and wcag. But I guess anything UI related snapshots would be best to run by wcag analysis as well. I will do some refactoring
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we move towards a CAC approach? Something like:
interface TestCommand {
type: 'click',
params?: any[]
}
interface TestLocation {
pageType: 'vertical' | 'universal',
page?: string
name: string
viewPort?: {
height?: number,
width?: number
}
commands?: TestCommand[]
}
const testingLocations : TestLocation[] = [
{
name: 'universal-search',
pageType: 'universal'
},
{
name: 'vertical-grid-search',
pageType: 'vertical',
page: 'people'
},
{
name: 'universal-search-faq-accordion',
pageType: 'universal',
commands: [
{ type: 'click', params: ['.HitchhikerFaqAccordion-toggle']}
]
}
]
Rather than an array of TestLocations, we could remove the name from TestLocation
and have a Record<name,TestLocation>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, that makes sense. Although we wouldn't have nice interface/typing since theme is only in js right now. Yeah I think having a PageOperator that runs through these test configs would be good.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How would this TestLocation[]
fit into what we have currently? Would the Photographer
and WCAGReporter
classes take them in? Then, for each entry in the array, the respective class would know what action to take? Another approach is to simply have a class that does all the navigating, clicking, etc. and takes the snapshot and does the WCAG analysis. For example:
await this._pageNavigator.gotoUniversalPage({ query: 'what if i forget my password?'});
await this._pageNavigator.click('.HitchhikerFaqAccordion-toggle')
await this._camera.snapshot('universal-search--faq-accordion');
this.results.push(await this._analyzer.analyze());
This approach is fairly straightforward and we don't need to create a CaC framework.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'd need to create some new code such as the PageOperator that Yen mentioned which would parse the file and run the commands. I like the idea of the CaC approach, but if we'd rather close this out sooner, then it probably makes more sense to do as you suggested
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the CaC approach is pretty neat, and it might also help us out with organizing the multilangphotographer's tests as well. I imagine it's something like this:
while(pageOperator.hasNextTestLocation()) {
const testContext = await pageOperator.nextTestLocation();
this._camera.snapshot(testContext); OR this.results.push(await this._analyzer.analyze());
}
I think I have most of the pageOperator code that would parse the file and run the commands, but yes there would need to be more changes in other files and writing up the testLocations itself.
Perhaps, we can keep this PR as is for the format, and make a new item to address the CaC or page navigation refactoring in general?
@tmeyer2115 @cea2aj
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that works, I'll add the CaC idea to the 'ideas for product doc' and we can keep this PR on the current approach
## Version 1.26.0 ### Features - Add support for the auth token, with a new global_config `token` field (#976) - Add support for the visitor, which can be set or changed at runtime (#970) - Add support for generic thumbs up/down feedback buttons to all cards (#973, #1005) - Handle page and template names with spaces (#988) ### Changes - Update styling of navigation ‘More’ menu (#989) - Update hover color of search bar buttons (#1000) - Internal repo changes (automate WCAG testing (#987)) ### Bugfixes - Fix console error which would appear on google maps (#955) - Fix extra query when loading in iframe integrations (#986) - Fix bug that displayed links that didn’t exist (#990) - Allow microphone access for voice search in iframe (#991) - Fix issue with custom search icon size (#992) - Fix bug with new prop comments not being added (#1004) - Fix distance alignment on location cards (#1006)
J=1644
TEST=manual/auto
npm run wcag
with puppeteer's browser headless mode disabled, see that all pages are loaded and analyzed by axe-core, see that the violation results are console log out.wcag test
ran in this pr