Accessibility Design

The purpose of the accessibility code in the embedder repo is to act as an intermediary between the Flutter engine and the Fuchscia Semantics Manager. The AccessibilityBridge class implements this functionality.

Flutter Engine <--> Accessibility Bridge <--> Fuchsia Semantics Manager

Responsibilities

The accessibility bridge must forward information between Flutter and Fuchsia. Its specific responsibilities can be broken down by functionality:

Functionality	Code Path	A11y Interface	Flutter Engine Interface
Enable/Disable Semantics	Fuchsia --> Bridge --> Flutter	OnSemanticsModeChanged	FlutterEngineUpdateSemanticsEnabled
Perform Accessibility Actions	Fuchsia --> Bridge --> Flutter	OnAccessibilityActionRequested	FlutterEngineDispatchSemanticsAction
Add Semantics Update	Flutter --> Bridge --> Fuchsia	UpdateSemanticNodesDeleteSemanticNodesCommitUpdates	update_semantics_callback
Hit Test	Fuchsia --> Bridge	HitTest	N/A
Message Announcement	Flutter --> Bridge --> Fuchsia	RequestAnnounceSendSemanticEvent	FlutterPlatformMessageCallback
Set Pixel Ratio	Fuchsia --> Bridge	SetPixelRatio	N/A

Notes

Pixel Ratio

This is the factor used to convert between a view’s logical coordinate space (assigned by scenic) and its allocated buffer size.

For example, on Nest Hub Max, Flutter views have a logical size of (1024x640) and an allocated size of (1280x800), so they have a pixel scale of 1.25.

Testing

1. Use emulator and dump semantic tree
   1. Start emulator
   2. ffx setui accessibility set -s true (note, this will temporarily disable touch/mouse input)
   3. ffx inspect show core/ui/a11y_manager
   4. Ensure the labels are filled out correctly with the words you see on the screen
2. Use a smart display device to test the screen reader

Custom Semantic Actions

Custom semantic actions are not supported yet.