{% set rfcid = “RFC-0174” %} {% include “docs/contribute/governance/rfcs/_common/_rfc_header.md” %}
{# Fuchsia RFCs use templates to display various fields from _rfcs.yaml. View the #} {# fully rendered RFCs at https://fuchsia.dev/fuchsia-src/contribute/governance/rfcs #}
This RFC describes how graphical scaling operations are handled in Flatland. Scenic, as Fuchsia‘s system compositor, can handle upscaling or downscaling a Flatland instance’s output through Flatland's SetScale() method. Floating-point scale factors may be used, with a guarantee that the resulting image avoids sub-pixel rendering. A Flatland instance only knows about the device pixel ratio value. In addition to the design of scale, this RFC defines some common terminology for referring to the pixel spaces.
Graphical scale operation is commonly needed for many UI features which Flatland, as a complete system compositor, should offer. Three of them are described in detail below.
Upsampling is increasing the rendered size of a Flatland instance or an Image. As an example, consider magnification. Magnification can be done by asking Scenic to upsample a Flatland client application's output.
Downsampling is decreasing the rendered size of a Flatland instance or an Image. As an example, consider GNOME's Activities Overview{: .external}. A graphical system shell may want to make a Flatland client application continue rendering at the same resolution, but shrink their content output to a smaller rectangle by asking Scenic.
Device pixel ratio is defined so that the display manufacturers can upscale the resolutions of their devices while still being able to display content, designed for lower resolutions, at the same size on the new screens. As an example, 4K and 1080p screen versions of a laptop would look identical from a distance, all buttons and layout being the same size physically. Only when you get closer you would be able to tell that 4K is sharper, because it packs 4 times the physical pixels in the same physical space as 1080p.
Facilitator: neelsa@google.com
Reviewers:
Socialization:
A detailed version of this design was reviewed internally with Scenic, Input and Accessibility teams. Alternative solutions were also discussed.
To accommodate all the use cases described in the Motivation section, we define a set of relationships that establishes what values may be set by a parent view on their child (parent-defined), what values can be set only by Flatland (system-defined), and what values may be observed by a child (child-observed).
Device pixel ratio is system-defined uniformly for all views on a single display. DPR is child-observed.
A child view‘s size is measured in LPs. This size is parent-defined and child-observed. A view has optimal resolution if the size and alignment of the view’s LPs precisely matches the display‘s DIPs for the region used to render the view. A view should act as if it has optimal resolution, where LP=DIP in size and alignment. However, the optimal resolution is not child-observed, and can be intentionally broken by a parent view (using Scale, see below) to allow features such as Magnification or GNOME’s Activities Overview.
Scale is introduced as a floating-point value. It is parent-defined but not child-observed! Parent-set scales for a child view is the multiplication of all scales introduced by all ancestor views of that child. PSS describes the ratio from a child view‘s LPs to the display’s DIPs. By hiding the PSS, Flatland retains control over how many physical pixels a child view occupies on the display, without forcing the child to re-allocate or re-present when upsampling or downsampling.
We make a further distinction with allocation pixels, which is the size of the buffer used to present content to display. This is a client implementation detail, but to remove confusion, we describe its relationship to other pixel spaces:
Flatland Mappings:
The pixel space relationships described above are illustrated in Figure 1.
Figure 1 - Flatland
The proposed changes are represented by the following FIDL snippet:
type LayoutInfo = table { /// The layout size of a View in logical pixels, defined by the parent's call to /// [`SetViewportProperties`]. Clients should re-layout their content when this value changes. 1: logical_size fuchsia.math.SizeU; /// The ratio from physical display pixels to the display's device independent pixels. /// Clients should not necessarily re-layout their content when this value changes. Clients may /// accommodate by reallocating their Image buffers that are adjusted by [`device_pixel_ratio`]. /// HiDPI-aware clients that want to avoid sampling artifacts should render onto a buffer with /// at least the size round([`logical_size`] * [`device_pixel_ratio`]). /// Note that rounding is not C-style float-to-int truncation. The floating-point product should /// be converted to the nearest integer. 2: device_pixel_ratio fuchsia.math.VecF; }; protocol Flatland { /// Sets the scale on a Transform. The order of geometric attribute application is addressed /// in the documentation for SetTranslation(). /// Note that if there is a Viewport set on this Transform, the child Flatland will not be notified /// about the changes made here. This method should only be used if the intention is to upsample or /// downsample the Viewport's output. Use [`SetViewportProperties`] if the intention is to resize or /// relayout the Viewport. SetScale(struct { transform_id TransformId; scale fuchsia.math.VecF; }); }
DPR is applied as a scale to each Image before Scenic renders or sends them to the display. If a client wants to support HiDPI, they are expected to layout their content using the given LP, but allocate larger buffers by using the reported DPR to reverse the effects of Scenic's scaling.
PSS is not communicated the children. That means upscaling through Flatland may cause blurry artifacts. However, this is preferred to forcing the client to re-allocate and re-present, for example because buffer size would grow by the scale factor and risk OOMs.
We may end up with non-integer pixels with float scales. If we have a consistent way of attaching to a nearby integer pixel value(round), we shouldn't end up with artifacts.
Flatland clients may want to go from logical pixels to physical pixels and that requires knowing PSS in addition to DPR. However, this is only necessary for input, so this design pushes this conversion information to fuchsia.ui.pointer
API.
The clients may respond to DPR changes at their own pace using this API. They will all be signaled about these changes at the same time, so we don't have a cascading delay jank pattern.
The client should always expect to work with a single DPR value. When we support multiple displays in the future, we can report a single DPR value coming multiple displays, or the DPR value of the display they are on.
Note that in Gfx(Legacy 3D API), DPR and PSS was multiplied into a single value and reported to the child. Hence, scale manipulation caused allocation side-effects in client code, causing OOMs and other unintended side effects, including large-scale architectural workarounds to achieve effects like Magnification and confusion around DPR. This design for Flatland diverges by suggesting that the child only needs to know about DPR for graphics purposes.
Gfx Mappings:
This design involves changes in fuchsia.ui.composition/Flatland
API. The implementation will be done in three main steps:
pixel_scale
field, which is being deprecated, in-tree and out-of-tree.This proposal reduces the memory usage of DPI-aware clients in comparison to Gfx API. In Gfx, DPR and PSS was multiplied into a single value and reported to the child. DPI-aware client would respond to the accumulation of all scales. As an example, if they were scaled by 5 by their parent and DPR of 2 applied, they would have received a pixel_scale
of 10 and allocated a 10 times larger buffer. This proposal distinguishes DPR and PSS, so the unnecessary allocations are gone.
This proposal reduces the expectations of re-allocate and re-layout from the clients. We can implement smoother upsample and downsample operations, because we no longer rely on the Flatland client to respond to scale changes.
This proposal does not affect the security model of Flatland API. The API guarantees around other graphical operations, such as clip limiting the input receiving area of a View, still apply to the scaled content.
This proposal reduces the extent of information passed to a Flatland view. In Gfx, DPR and PSS was multiplied into a single value and reported to the child. This proposal only reports DPR information. A child Flatland instance does not have an indication of how their parent Flatland instance decides to present their content, scaled or not, and cannot react to scaling changes.
This proposal suggests sending DPR information to Flatland clients. Device Pixel Ratio is derived from the physical and technical properties of the display unit. Although a very specific information, this property and ratio is not unique and may be common across multiple hardware, so it isn't very useful for fingerprinting. Furthermore, DPR is absolutely necessary for preparing quality graphical output.
Flatland API has been tested in a layered approach, which scale feature will follow:
Some of the Flatland documentation will be updated following this RFC to describe the HiDPI-aware client behavior and how scaling works.
There isn't a significant cost for implementing this proposal. This proposal suggests a DPR solution that will work no matter which display hardware is used. It may also be extended for multi-display use cases.
There were a couple alternatives considered for this design.