Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fuchsia][a11y] Set explicit hit regions per compositor layer on GFX. #33852

Merged
merged 1 commit into from
Jun 16, 2022

Conversation

abrush21
Copy link
Contributor

@abrush21 abrush21 commented Jun 6, 2022

This change modifies the GFX external view embedder class to set hit regions per compositor layer, which correspond to the bounds of the paint operations performed on the underlying skia canvas. The purpose of this change is to enable more accurate accessibility hit testing of flutter overlays. Today, scenic (fuchsia's graphics compositor) assumes all overlay content is full-screen, even if the overlay only paints content in a small region. With this change, scenic has enough information to determine accurately where overlay content is located.

fxbug.dev/100825

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • [X ] I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
  • I listed at least one issue that this PR fixes in the description above.
  • I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See testing the engine for instructions on
    writing and running engine tests.
  • I updated/added relevant documentation (doc comments with ///).
  • I signed the CLA.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@abrush21 abrush21 marked this pull request as ready for review June 6, 2022 20:55
@arbreng arbreng requested review from arbreng and akbiggs June 8, 2022 18:36
@chinmaygarde
Copy link
Member

cc @akbiggs

Copy link
Contributor

@arbreng arbreng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets go over this via GVC, there is some complex stuff I need to explain to you in here

layer->second.recorder->finishRecordingAsPicture();
FML_CHECK(picture != nullptr);

canvas->setMatrix(SkMatrix::I());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The complex part I was talking about before....

Lines 302-306 need to happen after the Present() below

We pipeline the Skia rendering work with the scenic IPC, and the fences passed to Present() make sure Scenic delays internally until Skia is finished

Moving the Skia rendering work up here will likely cause a performance regression

You really only need to move "finishRecordingAsPicture" up here, in order to populate the RTree. Can you break this part into 2 pieces, before and after Present()? It would also help to add a comment below, on the Skia code after Present, explaining the pipelining (my bad for not doing that originally)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation. I broke up the pre- and post-Present() operations as you suggested.

I had to cache the sk_sp returned by finishRecordingAsPicture at the beginning of the method, so that I could draw it to the canvas after the Present() call at the end of the method. Is that safe to do?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that should be safe to do.

finishRecordingAsPicture doesn't actually do any GPU work, it just records the commands from the SkRecordingCanvas into an SkPicture (a type of command buffer).

The canvas->drawPicture after Present() is what actually does the GPU work

@abrush21 abrush21 force-pushed the flutter-overlay-hit-test-fix-cleanup branch from 1257737 to 1531372 Compare June 10, 2022 22:43
@abrush21 abrush21 force-pushed the flutter-overlay-hit-test-fix-cleanup branch from 1531372 to 31466f4 Compare June 14, 2022 19:04
@@ -677,4 +717,121 @@ TEST_F(GfxExternalViewEmbedderTest, SceneWithOneView) {
loop().RunUntilIdle();
}

TEST_F(GfxExternalViewEmbedderTest, SimpleSceneDisjointHitRegions) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something I find useful for tests is to leave a comment at the top of the test explaining what behavior the test is validating since sometimes it can be hard to tell from the name alone.

@abrush21 abrush21 force-pushed the flutter-overlay-hit-test-fix-cleanup branch from 31466f4 to 8d386f3 Compare June 14, 2022 22:21
@akbiggs akbiggs merged commit 0720dc1 into flutter:main Jun 16, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jun 16, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jun 16, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jun 16, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jun 16, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jun 17, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jun 17, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jun 17, 2022
naudzghebre pushed a commit to naudzghebre/engine that referenced this pull request Jun 28, 2022
naudzghebre added a commit that referenced this pull request Jun 28, 2022
abrush21 added a commit to abrush21/engine that referenced this pull request Jun 29, 2022
naudzghebre pushed a commit that referenced this pull request Jun 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants