{% set rfcid = “RFC-0172” %} {% include “docs/contribute/governance/rfcs/_common/_rfc_header.md” %}
{# Fuchsia RFCs use templates to display various fields from _rfcs.yaml. View the #} {# fully rendered RFCs at https://fuchsia.dev/fuchsia-src/contribute/governance/rfcs #}
This RFC introduces a new UI Activity Service which replaces and downscopes the responsibilities of a pre-existing version.
The proposed service adds two new FIDL protocols:
fuchsia.input.interaction.observation.Aggregator to collect evidence of user input activity, andfuchsia.input.interaction.Notifier to notify clients of changes in user input activity state.We would like to provide a service that notifies other parts of the system whether recent user input activity has occurred. This service is useful to inform other system services such as power-conserving protocols or screensaver functionality.
There is a pre-existing “Activity Service” which establishes itself as a source of truth regarding user idle state in the system. This proposal is a departure from the current approach and introduces a new “UI Activity Service” to indicate a narrower scope of responsibility–-over user input activity alone rather than system activity altogether–-thereby enabling new features in some products while reducing technical debt in others.
Facilitator:
Reviewers:
Consulted:
Socialization:
This RFC was discussed during doc review with the Fuchsia Input team, and requested reviews for privacy and security.
Activity is user input interaction with the device that happened recently.
Recently is defined as some product-configured time threshold, e.g. 15 minutes.
When the device has not received activity recently, it is considered idle.
User input interaction is constrained to the use cases listed below for the initial implementation.
The following user interactions MUST result in the service entering or renewing an active state:
The following user interactions SHOULD result in the service entering or renewing an active state:
The following interactions are not considered active user input activities due to their corresponding rationales given inline. Therefore they MUST NOT result in the service entering an active state:
fuchsia.media.ActivityReporter should be used instead of the activity service discussed in this RFC.We introduce a new internal FIDL protocol, fuchsia.input.interaction.observation.Aggregator and a new FIDL protocol fuchsia.input.interaction.Notifier to the partner SDK. These protocols will be implemented and exposed by a new Activity class within the Input Pipeline component.
fuchsia.input.interaction.observation.AggregatorThe clients of this API report that they believe they have evidence of user input activity, and we treat their information as such. Therefore access is restricted to in-tree components via capability routing. See Security considerations for more.
library fuchsia.input.interaction.observation;
using zx;
/// The Aggregator protocol collects evidence of user activity and uses this
/// evidence to set the system's activity state.
@discoverable
protocol Aggregator {
/// Reports a discrete activity such as a keystroke.
ReportDiscreteActivity(struct {
activity DiscreteActivity;
event_time zx.time;
}) -> ();
};
Unlike its predecessor, which collected both discrete and ongoing events (i.e. activities with a start and end time), the initial version of this protocol defined by this RFC only collects “Discrete” activities. The ability to collect ongoing events was initially introduced to support media playback, which is explicitly not a use case of this RFC.
fuchsia.input.interaction.NotifierThe clients of this API should want to subscribe to changes between user interaction “active” and “idle” states as defined similarly to fuchsia.ui.activity.State. This RFC plans to create a similar enum in the fuchsia.input.interaction library that will be flexible in case of future need for expansion. Access does not need to be restricted, see Privacy considerations. Examples include:
library fuchsia.input.interaction;
/// The Notifier protocol offers a subscription interface through
/// which clients can watch for changes in the system's activity state.
@discoverable
protocol Notifier {
/// Subscribe to changes in the system's state.
/// The server will always respond immediately with the initial state,
/// and after that whenever the system's state changes.
WatchState(table {}) -> (resource struct {
state State;
});
};
The proposed protocol will differ from the existing fuchsia.ui.activity.Notifier.WatchState protocol because it will use a hanging get pattern, removing the need for a Listener protocol entirely. It also does not send a timestamp.
The UI Activity Service can use the fuchsia.hardware.input FIDL to watch for lid sensor driver reports and transition to an active state upon receiving a lid-opened report.
Implementing the UI Activity Service (A) within the Input Pipeline (IP) component instead of as its own component can save ~172 KB in disk space, depending on the product and board configurations. For space-constrained products, reducing size frees room for other improvements in the system.
| Build | A | IP | IP + two FIDLs |
|---|---|---|---|
| workstation_pro.chromebook-x64 | 196 KB | 1364 KB | 1388 KB |
| core.astro | 184 KB | 688 KB | 700 KB |
Integrating the UI Activity Service within Input Pipeline includes the following:
InputReports by marking events handled or sending them to other components, as that makes the service aware of and responsible of more information than is necessary to determine whether activity has happened recently.fuchsia.input.interaction.observation.Aggregator).InputHandler.The idle threshold is how much time has passed since the last user activity for the system to become idle. This can be set using structured configuration in the Input Pipeline or Scene Manager component at the product level and will be set to 15 minutes for the current products.
The service will be implemented as follows:
Define the new fuchsia.input.interaction.observation.Aggregator and fuchsia.input.interaction.Notifier FIDL protocols.
Implement and unit test a new Activity service that is initialized within Input Pipeline to serve both protocols.
Relevant InputHandlers report activity via fuchsia.input.interaction.observation.Aggregator. Note: Reporting activity during this phase rather than during the binding phase is preferred because InputHandlers are responsible for dispatching information from InputEvents to other components or services.
Affected handlers include:
fuchsia.input.interaction.observation.Aggregator informs Activity statefuchsia.input.interaction.Notifier is notified when system transitions to Activefuchsia.input.interaction.Notifier is notified when system transitions to Idle=== fuchsia.input.interaction.Notifier is available for use by clients at this point ===
fuchsia.ui.activity.Tracker, fuchsia.ui.activity.Provider, and fuchsia.ui.activity.control.Controlfuchsia.ui.activity.Provider to fuchsia.input.interaction.Notifier [in Cobalt, Omaha, and PowerManager]fuchsia.ui.activity.Tracker (non-functional)//src/sys/activityThe UI Activity Service MUST NOT add latency to the processing of input events in the system.
While the protocols proposed do not introduce asynchronous or computation-heavy logic, we may want to introduce throttling to high-frequency input events. For example, it's common for mice to send about 1000 events/second, so it may be desirable to rate-limit the FIDL calls from MouseInjectorHandler to the Activity Service.
We can monitor pre-existing performance tests, such as input latency tests, to discern whether changes regress performance.
The primary abuse vector for the UI Activity Service is to keep the system awake by spuriously reporting activity using fuchsia.input.interaction.observation.Aggregator. For instance, a malicious application could periodically report activity to keep the system awake. This is considered low risk given that fuchsia.input.interaction.observation.Aggregator is only available in-tree, not offered in the SDK, and will only be used by components in the UI realm.
Any service like this one poses the risk of delaying software updates, keeping the screen unlocked, depleting the battery, and possibly denying service to other applications on the device. Delegated data access is a risk not solved by this RFC. Clients relying on idle state for features like the aforementioned SHOULD implement their own maximum activity limit if relevant when using this service, e.g. a forced system OTA once a user has been active for 24 hours.
Since this capability has to be routed through CFV2, it should be noted that the platform only grants ability to report activity to Input Pipeline and potentially A11y Manager at the platform level, and that products can further route to trusted components.
While the UI Activity Service does not expose any information about the nature of user activity, it does share whether activity has occurred recently within a preset time threshold via fuchsia.input.interaction.Notifier.
Whether someone is at or using their device at a specific time is privacy impacting and sensitive, and therefore something we would not want untrusted components to have access to. Since this capability has to be routed through CFV2, it should be noted that the platform should only grant ability to observe activity that are trusted at the platform level, and that products can further route to trusted components.
Timestamps specifically will not be made available to subscribers.
Our testing strategy includes
The service will include a README overview, and new FIDL protocols will be accompanied with comments inline.
If the Input Pipeline panics or deadlocks, the UI Activity Service would also fail. In that case, subscribers to changes in idle state may want to default to active-state behavior rather than idle-state behavior, and should have other mitigating measures such as a maximum activity timeout to ensure critical processes are still able to occur. If the Input Pipeline is not able to receive or respond to Input Reports, there may be wider issues in the system.
fuchsia.ui.activity.Tracker protocolThis approach prefers creating a new fuchsia.input.interaction.observation.Aggregator protocol.
StartOngoingActivity or EndOngoingActivity, and future needs may want to re-evaluate the current pattern.fuchsia.ui.activity.Provider protocolThis approach prefers creating a new fuchsia.input.interaction.Notifier protocol.
fuchsia.ui.activity.control.Control protocol.WatchState protocol would have to be migrated to the proposed hanging-get pattern.WatchState protocol unnecessarily releases a timestamp that we do not want clients to rely on without a concrete use case or intention for exposing it.Certain signals may be considered activity in some user flows but not others. Accordingly, the correctness of how to interpret signals will vary depending on the needs of a given form factor or service.
It is recommended that the fuchsia.input.interaction.Notifier protocol is used to consume UI activity state, and can be referenced in combination with other forms of activity (e.g., from audio, microphone, or camera) to determine some final user or system state as is relevant on a case-by-case basis. For example, a screensaver feature might consult just user input activity, whereas a lockscreen feature might consult both user input activity and user authentication state.
In the future, certain services may want to follow an activity state as determined by a subset of user input modes, e.g. recent touch but not recent keypress. This would introduce exponential combinations between the possible input modes.
Given a current lack of interest in more elaborate features, we decided not to prematurely introduce this complexity.
Both System Authentication and UI Activity Service care about some notion of user behavior, but with minimal practical overlap. For instance, System Authentication cares about a more granular set of user presence states, including user distance from the device and whether a user is an account owner.
While it might be possible to track user activity in a way which is amenable to both use cases, it may not be necessary to proactively integrate these two systems without a driving use case in mind.
In the future, certain services may want to determine their own recency thresholds for user input interactions. While this RFC does not propose a solution, omitting this feature for simplicity of an initial implementation, the protocols propose may be extended to add support when concrete use cases arise.