{% set rfcid = “RFC-0061” %} {% include “docs/contribute/governance/rfcs/_common/_rfc_header.md” %}
Note: Formerly known as FTP-015.
“Catering to Hawaii and Alaska”
To provide more ways to express payloads whose shape may need to evolve over time, we propose to replace unions as they exist today with extensible unions.
Today, unions provide no way to evolve over time, and we even warn that “in general, changing the definition of a union will break binary compatibility.”
There are a number of unions defined today where extensibility is necessary, e.g., fuchsia.modular/TriggerCondition, where fields are deprecated without being removed, or fuchsia.modular/Interaction.
As described later, there also many unions whose current representation is appropriate as they are unlikely to evolve in the near-future. However, keeping both static unions and extensible unions introduces unneeded complexity, see the pros and cons.
To introduce extensible unions, we need to modify multiple parts of FIDL: the language and fidlc, the JSON IR, the wire format and all language bindings. We'll also need to document this new feature in various places. We discuss each change one by one.
Syntactically, extensible unions look exactly the same as static unions:
union MyExtensibleUnion { Type1 field1; Type2 field2; ... TypeN fieldN; }
Behind the scenes, each field is assigned an ordinal: this is comparable to how tables have ordinals for each field, and how methods' ordinals get automatically assigned.
Specifically:
.”, the extensible union name, “/”, and finally the member name, then take the SHA256, and mask with 0x7fffffff.uint32, no two fields can claim the same ordinal, and we disallow 0. In the case of ordinal conflict, the [Selector] attribute should be used to provide an alternate name (or the member renamed).An extensible union can be used anywhere a union can currently be used in the language. Particularly:
Following tables, we will add one key in each union field declaration “ordinal.”
On the wire, an extensible union is represented by the ordinal to discriminate amongst the choices (padded to 8 bytes), followed by an envelope of the various members known to the producer. Specifically, that is:
uint32 tag which contains the ordinal of the member being encoded;uint32 padding to align to 8 bytes;uint32 num_bytes storing the number of bytes in the envelope, always a multiple of 8, and must be 0 if the envelope is null;uint32 num_handles storing the number of handles in the envelope, and must be 0 if the envelope is null;uint64 data pointer to indicate presence (or absence) of out-of-line data:0 when envelope is null;A nullable extensible union has a tag of 0, num_bytes is set to 0, num_handles is set to 0, and the data pointer is FIDL_ALLOC_ABSENT, i.e., 0. Essentially, a null extensible union is 24 bytes of 0s.
Extensible unions are similar to unions, except that one needs to also handle an “unknown” case when union is read. Ideally, most language bindings would treat
union Name { Type1 field1; ...; TypeN fieldN; };
as they would an extensible union, such that code can easily be switched from one to the other, modulo support of the unknown case, which is meaningful only in the extensible union case.
To start, we suggest no language bindings expose reserved members: while these are present in the JSON IR for completeness, we do not expect that exposing them in language bindings be useful.
Implementation will be done in two steps.
First, we will build support for extensible unions:
fidlc), by using a different keyword (xunion) to distinguish between static unions and extensible unions.Second, we will migrate all static unions to extensible unions:
Generate ordinals for static unions, and place them in the JSON IR. Backends should initially ignore those.
On read paths, have both modes of reading unions, as if they were static unions, and as if they were extensible unions (ordinals are needed for that to be possible). Choose between one and the other based on a flag in the transaction message header.
Update write paths to encode unions as extensible unions, and indicate as much by setting the flag in the transaction message header.
When all writers have been updated, deployed, and propagated, remove static union handling, and scaffolding code for the soft transition.
This would require documentation in at least these places:
An extensible union is explicitly not backwards compatible with a “static” union.
No impact on performance when not used. Negligible performance impact during build time.
No impact on security.
Unit tests in the compiler, unit tests for encoding/decoding in various language bindings, and compatibility test to check various language bindings together.
Extensible unions are less efficient than non-extensible unions. Furthermore, non-extensible unions are not expressible through other means in the language. As such, we propose both features living side by side.
However, we could decide that only extensible unions should exist, and do away with unions as currently defined. This would go against various places in Fuchsia where unions represent performance critical messages, and where there is little extension expectation, e.g. fuchsia.io/NodeInfo, fuchsia.net/IpAddress.
enum to indicate whether it is added or removed.All in all, we decided to replace static unions with extensible unions.
We use ordinal to denote the internal numeric value assigned to fields, i.e., the value calculated through hashing. We use tag to denote the representation of the variants in bindings: in Go this may be constants of a type alias, in Dart this may be an enum.
The fidlc compiler deals with ordinals only. Developers would most likely deal with tags only. And bindings provide translation from the high-level tag, to the low-level internal ordinal.
During the design phase, we considered having extensible unions be empty. However, we chose to disallow that in the end: choosing a nullable extensible union with a single variant (e.g., an empty struct) clearly models the intent. This also avoids having two “unit” values for extensible unions i.e., a null value and an empty value.