Pointer Input Design

At a high level the embedder‘s job is to handle the input events from Scenic and transform the fuchsia specific FIDL events into a generic type (FlutterPointerEvent, defined in embedder.h) and send it to the flutter engine via the Embedder API’s FlutterEngineSendPointerEvent function. The flutter engine implements this function in embedder.cc.

Scenic --(Touch/Mouse Event)--> Embedder --(FlutterPointerEvent)--> Flutter Engine`

There are two forms of pointer input (Touch and Mouse), and each has some unique differences that are described below.

Touch Input

Embedder must convert the TouchEvent from Scenic's TouchSource protocol into a generic FlutterPointerEvent that we can send to the flutter engine.

Touch input is unique from mouse input in several ways:

  1. There can be multiple fingers touching the screen at once. In order to support “multi-touch” we need to encode both fuchsia's device_id and pointer_id into the device field in the FlutterPointerEvent object that is sent to the flutter engine.

  2. Touch input requires adhering to the Gesture Disambiguation Protocol. The embedder must buffer new interactions and only handle them once Scenic has determined who is the true owner of this event.

Responsibilities

  1. Initialize the TouchSourceHandle
  2. Call Watch() on the channel to begin listening for events
  3. Implement a handler function that converts fuchsia::ui::pointer::TouchEvent types to FlutterPointerEvent types
  4. Handler should follow the rules of Gesture Disambiguation
  5. Send the pointer events to the flutter engine using the embedder API's FlutterEngineSendPointerEvent function

Mouse Input

Embedder must convert the MouseEvent from Scenic's MouseSource protocol into a generic FlutterPointerEvent that we can send to the flutter engine.

Mouse input is unique from touch input because it has buttons that can be clicked or scrolled. Therefore, the mouse input channel processor needs to keep track of additional state (i.e. whether any buttons are down or up) and relay this to the flutter engine.

Responsibilities

  1. Initialize the MouseSourceHandle
  2. Call Watch() on the channel to begin listening for events
  3. Implement a handler function that converts fuchsia::ui::pointer::MouseEvent types to FlutterPointerEvent types
  4. Send the pointer events to the flutter engine using the embedder API's FlutterEngineSendPointerEvent function

Usage & Implementation

The TouchDelegate and MouseDelegate classes implement the responsibilities for touch and mouse input respectively. Both are initialized on the embedder‘s main thread directly on the stack of main(). These objects lifetime is therefore tied to the scope of the main function, which will run as long as the embedder’s async processing loop is still running.

The Embedder API is a C header, therefore the structs don't have constructors. When initializing a FlutterPointerEvent we need to memset() the values to be 0 and then set the struct_size field which is used by the flutter engine. See ResetFlutterPointerEvent().