Define issue triage process

[mcking65]

Objective: Enable timely and more efficient triage and prioritization of bugs on example pages with:

1. Clear steps and ability to assign people to those steps
2. Consistent labeling process

We have a steady cadence of new issues being raised and huge backlog of old issues. We need a clear process for ensuring everything is well prioritized and we can point people to the most important work. Bugs on example pages are often most difficult to triage and most important to resolve, so this process is focused on that category of issues.

[css-meeting-bot]

The ARIA Authoring Practices (APG) Task Force just discussed Define bug triage process · Issue #2906 · w3c/aria-practices.

The full IRC log of that discussion

<jugglinmike> subtopic: Define bug triage process · Issue #2906 · w3c/aria-practices
<jugglinmike> github: https://github.com//issues/2906
<jugglinmike> Matt_King: On January 16, I listed 8 different sources of work
<jugglinmike> Matt_King: My goal in this issue is to identify certain areas of work where I think it's very possible for us to do better with the team that we have
<jugglinmike> Matt_King: ...and make it easier for everybody to help out in the ways that they can
<jugglinmike> Matt_King: For instance, wouldn't it be awesome if we had more people helping out with the process of triaging issue?
<jugglinmike> Matt_King: I think issues with the examples are the most common
<jugglinmike> Matt_King: I'd like to have a clearly-defined bug triage process that allows us to do it more efficiently and more asynchronously. That should free up time during these meetings
<jugglinmike> Matt_King: Ultimately, this issue will end up on a wiki page
<jugglinmike> Matt_King: Things we need to do: first, agree that the issue is a bug in the APG code (not in the browser or in the screen reader).
<jugglinmike> Matt_King: ...if we know that it's reproducible within our scope, we then need to assign a severity and a priority
<jugglinmike> Matt_King: ...with that information, we'll be able to easily create queries that surface the most important information for this group
<Jem> regret+ mark, bryan, curt
<jugglinmike> Matt_King: I believe that verifying that the bug is reproducible and "in scope" is something someone can do outside of this meeting
<jugglinmike> Matt_King: Provided they can label and comment on the issue
<jugglinmike> Matt_King: From there, as a group, we can use the information that was collected and reported asynchronously to make decisions about prioritization
<jugglinmike> Jem: WCAG does something similar to this
<jugglinmike> Jem: I think it's a good idea
<jugglinmike> jugglinmike: it might be tough for non-developers to determine whether a bug is truly in the APG's application code (rather than a browser bug or a screen reader bug)
<Jem> +1 Daniel for pairing or partnership
<jugglinmike> Matt_King: I think we can document a fairly repeatable process for non-technical folks, at least for ruling out browser bugs (e.g. opening in multiple browsers)
<jugglinmike> dmontalvo: Maybe assigning two people of differing abilities to triage will be helpful
<jugglinmike> Matt_King: I've been thinking of adding a new label called "reproducible" and using that as a signal that this first step of the process is complete
<Jem> https://github.com/w3c/aria-practices/labels
<jugglinmike> Matt_King: This group could review bugs labeled "reproducible" and decide if they are dependent on assistive technologies (and close them with an appropriate label in that case)
<jugglinmike> jugglinmike: Sometimes triaging takes a fair amount of analysis to understand what the reporter is saying. We should document as part of this process that the person verifying may need to formally document their steps to reproduce
<jugglinmike> Matt_King: Good point!
<jugglinmike> Jem: Also, we need environment information (e.g. browser version, AT version, OS version, etc.), but reporters do not always provide them. The person performing the triage should also capture that if it isn't present
<jugglinmike> Matt_King: Great point!

[mcking65]

also discussed in feb 20 meeting

[mcking65]

Still working on how to resolve process gaps discussed last week.

Made some updates to:

• Issue Triage Process · w3c/aria-practices Wiki
  • Improve overview
  • Start working on solving how to identify issues that need to go through intake
• Labels · w3c/aria-practices
  • Add label for "ready to prioritize", P1, P2, and P3
  • Removed some more obsolete labels (appendix, needs review)

[css-meeting-bot]

The ARIA Authoring Practices (APG) Task Force just discussed Update on triage process definition.

The full IRC log of that discussion

<jugglinmike> Topic: Update on triage process definition
<jugglinmike> github: https://github.com//issues/2906
<jugglinmike> Matt_King: I spent a fair amount of time on this, but I haven't yet addressed everything we dicussed last week. I'm getting close, though
<jugglinmike> Matt_King: I started working a little more on our labels
<jugglinmike> Matt_King: I added a label named "ready to prioritize", and I also added labels, "p1", "p2", and "p3"
<jugglinmike> Matt_King: If something is "ready to prioritize", that means it has completed the "intake" step, and if it's a bug, it has also completed the "bug reproduction documentation" step in the triage process
<jugglinmike> Matt_King: Once something is prioritized, we would replace the "ready to prioritize" label with either "p1", "p2", or "p3"
<jugglinmike> Matt_King: I'm still updating the documentation to reflect this
<jugglinmike> Matt_King: Then it will be ready to go, I think
<jugglinmike> Matt_King: I need to add labels for issue severity.
<jugglinmike> Matt_King: I think this will all be done by next week
<jugglinmike> Jem: Thank you for all the work you've put into the triage process!
<jugglinmike> Jem: What would be the timeline to start working on triage itself?
<jugglinmike> Matt_King: I think I'd like try some of it in this meeting today. That will help use ensure that we're all aligned on which issue labels we need
<jugglinmike> Matt_King: Otherwise, I'll leave it up to you to decide how you want to set up a schedule and how you want to document that schedule
<jugglinmike> Matt_King: I think it would be possible for you to kick it off next week
<jugglinmike> Jem: Okay, so who wants to contribute to the triage process?
<jugglinmike> Matt_King: Last week, we recorded three volunteers, I believe
<jugglinmike> dmontalvo: I was one of them
<jugglinmike> Matt_King: My proposal would be that, when people are on rotation (perhaps two times a month), they commit to an hour of independent triage--either the "intake" process or the "bug reproduction" process
<jugglinmike> Matt_King: Obviously, we don't get bug reports every single week, so eventually, we would be able to begin revisiting prior bug reports using this process
<jugglinmike> Matt_King: There appear to be about 30 bugs already existing. This means that we could have really solid documentation on them all by some time in June
<jugglinmike> jongund: How do we sign up? Is there a wiki page?
<jugglinmike> Jem: I need to create a place to manage this, but I haven't selected one, yet
<jugglinmike> Jem: Let's use the GitHub Wiki associated with the aria-apg repository
<jugglinmike> Matt_King: That means folks will need to be comfortable working with GitHub Wikis, but I think that's a reasonable requirement