Computer systems use display devices to interact with human users's sight. The display device is connected to the computer system via a display connection.
Graphics hardware and software “figures out” what will be displayed to the user, and is ultimately responsible for creating image data, which is a low-level description of a two-dimensional image that can be processed by a display device.
The display engine is the hardware inside the computer system that drives the display connection. The display engine‘s primary responsibility is turning the image data into the electrical signals that go across the display connection. The display device at the other end of the display connection then displays the image, by turning the electrical signals into light (photons) that interacts with the human user’s sight.
Under normal operation, this process of transmitting and displaying images is repeated continuously at a high frequency. Thanks to flicker fusion, human eyes perceive a series of images displayed in quick succession as continuous motion. The sequence of images is called a “moving” image, the individual images are called frames, and the frequency of this process is called the frame rate. The display connection is said to carry video data, a low-level description of the sequence of frames.
The Fuchsia display stack only supports video displays, which are display devices that can represent arbitrary images (within a digital approximation). For a contrasting example, the display stack does not support segment displays.
The display engine's design and operation is easiest to understand in the context of the entire path from software that intends to display images to the human user that sees the images. The following subsections cover each part of the path, ordered by increasing distance from the user.
Image data is ultimately shown to the human user by converting it into light that reaches the user's eyes. In modern display hardware, the photons are produced by a panel in the display device.
The panel is a rectangular grid (lattice) of cells called pixels, where each pixel is a collection of a few subpixels that work together to produce a desired color. So, ultimately, the image displayed to the user is a raster graphic. The panel's most important characteristic is its resolution.
It can be useful to think of a panel as a write-only DRAM (Dynamic Random Access Memory). The panel‘s subpixels are similar to the DRAM’s 1-bit memory cells, in that they store information, and are arranged in a rectangular grid. Both subpixels and DRAM memory cells hold information for a short amount of time (on the order of milliseconds), and must be “refreshed” with new values before that time runs out. A panel subpixel stores the intensity of the light to be emitted, whereas a DRAM memory cell stores one bit that decides the cell's behavior during a read operation.
The subpixel refresh requirements are usually communicated indirectly, as the panel's minimum frame rate. The implication is that the panel refreshes all its subpixels in the process of displaying a new image. So, subpixels are guaranteed to hold their information for the period between two frames, which is the inverse of the minimum frame rate.
Panels differ in the materials that make up the pixels and their connections. Understanding specific panel technologies is generally not necessary for working on Fuchsia's Display stack. For curious readers, two popular panel technologies at the time of this writing are LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).
The DDIC (Display Driver IC / Integrated Circuit) manages the display device's side of the display connection. The DDIC decodes the electrical signals received from the display connection, and converts the image information into a form suitable for the display panel.
Most importantly, the DDIC is responsible for distributing the image data to the panel‘s subpixels, in a way that meets the panel’s refresh requirements. The hardware that implements this functionality is sometimes called a TCON (Timing Controller).
The DDIC sometimes supplies power to the panel, effectively integrating the functionality of a PMIC (Power Management Integrated Circuit). This entails converting the voltage provided to the DDIC‘s power rails to the voltages required by the panel’s power rails, and driving the power rails to meet the panel's power up and power down sequencing requirements.
The display connection carries image information, encoded as electrical signals, from the display engine to the DDIC.
At a high level, driving a display connection can be split into the following concerns.
In many modern use cases, such as mobile and embedded devices, the display engine and DDIC are in the same housing, also known as enclosure or “box”. The display connection may be an internal cable such as a FPC (Flexible Printed Circuit), or a collection of traces on a board. The user cannot replace the display connection while the computer is powered on. Many display connections cannot be replaced after the device leaves the factory.
In some use cases, such as desktop computers and external displays attached to mobile devices, the display engine and DDIC are in separate enclosures. The display engine may be in a computer or mobile device, while the DDIC is in a monitor or TV. In these cases, the display connection is a cable with connectors. The user may replace the cable or the connected display hardware while the computer is powered on.
The display engine manages the display connection from the computer's side, and therefore serves as the bridge between the display connection and any image producer on the computer. Image producers can be hardware components, such as the GPU and the media decoder, or software that directly produces pixel data.
Most image producers store image data in the system's main memory (DRAM). For this reason, virtually all display engines are capable of retrieving image data from DRAM.
Documentation from various vendors uses different acronyms to refer to the display engine hardware. Some of these names are below.
Most use cases on modern systems are best met by having a GPU (Graphics Processing Unit) compute the pixels that make up the images displayed to the user. The GPU is in the same computer as the display engine, either on the same SoC (System on a Chip) or on the same graphics card.
The GPU typically stores the computed image data in system DRAM. When the GPU and display engine are on the same graphics card, the GPU may store the computed image data in the GDDR (Graphics Double Data Rate) memory on the graphics card, and the display engine may read the data directly from GDDR. This optimization eliminates the performance cost of accessing the system DRAM, for both the GPU and the display engine.
While the GPU is usually colocated on the same silicon chip as the display engine, it is conceptually a different piece of hardware, and it is managed by separate driver software. Confusingly, the display driver and GPU driver are sometimes packaged together in a “graphics driver”.
Some use cases, such as playing movies or video conferencing, are best met by having dedicated media decoder hardware compute the pixels making up the images displayed to the user. The media decoder may be an independent hardware unit, or it may belong to a codec.
From a high-level system perspective, media decoders are similar to GPUs. The media decoder hardware is usually on the same SoC or graphics card as the display engine, and may use system DRAM or GDDR to store the image data. The hardware also has its own driver, which may be bundled in a “graphics driver”.
Some systems have a direct hardware path for passing pixel data between the media decoder and the display engine. This optimization saves the cost of storing image data in an intermediate memory, but requires that the media decoder can consistently produce pixel data at the rate needed by the display engine.
This section will be expanded in the future.
This section will be expanded in the future.