| <!-- mdformat off(templates not supported) --> |
| {% set rfcid = "RFC-0135" %} |
| {% include "docs/contribute/governance/rfcs/_common/_rfc_header.md" %} |
| # {{ rfc.name }}: {{ rfc.title }} |
| <!-- SET the `rfcid` VAR ABOVE. DO NOT EDIT ANYTHING ELSE ABOVE THIS LINE. --> |
| |
| <!-- mdformat on --> |
| |
| <!-- This should begin with an H2 element (for example, ## Summary).--> |
| |
| ## Summary |
| |
| Design for encoding the system ABI revision into packages. |
| |
| ## Motivation |
| |
| [RFC-0002] introduces the concepts of [API level] and [ABI revision]. The _API |
| level_ is a monotonically increasing human-understandable number that |
| corresponds to a set of APIs that are available when building applications. The |
| _ABI revision_ corresponds to the specific semantics exposed by the [Fuchsia |
| System Interface] the application expects the platform to provide. An API level |
| corresponds to a specific ABI revision, but multiple API levels can refer to the |
| same ABI revision. This functionality allows for the Fuchsia platform to evolve |
| over time while continuing to support older applications. |
| |
| Implementing platform versioning requires that packages encode the ABI level |
| they wish to target, so that the platform can know whether a package is |
| compatible with the running system. |
| |
| ## Stakeholders |
| |
| _Facilitator:_ abarth@google.com |
| |
| _Reviewers:_ |
| |
| The following stakeholders were chosen as this RFC touches Fuchsia Tools, |
| Software Delivery, and will eventually impact how Component Framework will |
| select the component system interface to present to applications. |
| |
| * abarth@google.com (FEC member) |
| * computerdruid@google.com (SWD) |
| * geb@google.com (CF) |
| * mkember@google.com (FIDL) |
| * raggi@google.com (Tools) |
| * wittrock@google.com (SWD) |
| |
| _Consulted:_ |
| |
| * lijiaming (PDK) |
| |
| _Socialization:_ |
| |
| This RFC went through a design review with the Software Delivery and Fuchsia |
| Tools teams. |
| |
| ## Terminology |
| |
| A Fuchsia _[package]_ is the unit of distribution for distributing and |
| organizing files for Fuchsia programs, components, and services. |
| |
| The _[meta.far]_ is the package metadata archive. It contains the mapping of |
| user-facing file names to the underlying content addressed blobs. This is used |
| by the packaging system to download package contents, and construct a namespace |
| hierarchy. It also contains custom user-provided files. |
| |
| ## Design |
| |
| This design introduces the notion of a _package ABI revision_, which is the |
| target ABI revision of the Fuchsia platform interface used by a package. This |
| value will be written as an unsigned little endian 64-bit integer written into |
| the `meta.far` in the file `meta/fuchsia.abi/abi-revision`. |
| |
| All package generation tooling will be updated to require either the API level |
| or the ABI revision to be specified during package building. When the API level |
| is specified, the tooling should encode the corresponding ABI revision found in |
| the [SDK version history], or error out. Likewise, the tooling should enforce |
| that the specified ABI revision is supported by the SDK, or error out. This |
| should avoid the risk of creating a component with an older SDK, but specifying |
| an ABI only present in a newer SDK. At best this would result in a component |
| that failed to run. At worst this could lead the component to strange or |
| dangerous bugs. |
| |
| ### Use cases for package ABI revision |
| |
| This proposal only covers how the ABI revision is encoded into a package. Users |
| of the package ABI revision will be defined in future designs. However, here |
| are some potential use cases to help illustrate how this may be used: |
| |
| * [RFC-0002 Platform] describes a mechanism where Component Manager could use |
| the package ABI revision to select which platform interface to use when |
| running a component. |
| * _System assembly_ is the process of composing a Fuchsia [system image] from a |
| set of packages and other compiled artifacts. The system image can then be |
| used to deliver Fuchsia through a different means (OTA, flashing, paving, |
| etc). The package ABI revision can be used by the the assembler to reject |
| integrating a package if the system does not support the required package ABI. |
| * Similarly, petals or application developers can use the package ABI revision |
| to understand if all of their components in a complex tree can run on their |
| target Fuchsia release. |
| * _Ephemeral packages_ are packages that are downloaded upon demand, rather than |
| being built into the system image. The Package Resolver could leverage the |
| package ABI revision to reject downloading a package that it knows is not |
| executable on the system. Note though this may not work for system updates. |
| See [System update considerations](#system-update-considerations) for more |
| details. |
| |
| ### Target API level or ABI revision selection |
| |
| This proposal indends to make the API level or ABI revision a required argument |
| when building a package, but this will be optional during the initial rollout |
| of this feature. With that in mind, end-developers should parameterize their |
| build rules for the target API level or ABI revision. This should make it easy |
| to target new releases. It may even be possible to set up automatic testing of |
| new API levels by setting up a test roller that experimentally advances the |
| target API level and sees if anything fails. This could feed back into platform |
| bug reports if these failures were unexpected. |
| |
| ### System update considerations |
| |
| In order for a Fuchsia device to update from one release to another, the |
| _system updater_ resolves a special package known as the _update package_. This |
| package describes all the base packages and other artifacts necessary for the |
| new Fuchsia version to run. These new packages are not intended to be used on |
| the current system, but instead would be used once the device is rebooted into |
| the new system. |
| |
| In light of this, if we decided to use the package ABI revision to reject |
| downloading a package, we need to be careful to avoid breaking the OTA process. |
| This can be seen through the following example: |
| |
| Consider two Fuchsia builds `A`, which only supports `ABI-1` and `C`, which |
| supports `ABI-2`. If we filter packages based off ABI, the update package for |
| `C` would need to be `ABI-1` for it to download. However, the `C` version base |
| packages would have to be `ABI-2` to use the new ABI, but these would be |
| rejected by the version `A` package resolver. |
| |
| One way to avoid this is to introduce new ABI revisions through a _stepping |
| stone release_. Fuchsia is designed to allow a device to skip multiple releases |
| in order to update to the latest version. A stepping stone release is a special |
| release that cannot be skipped. It is designed to enable graceful transitions |
| through platform interfaces. |
| |
| Redoing the prior example, instead lets have 3 sequential Fuchsia releases: |
| |
| * `A`: supports `ABI-1`, base packages at `ABI-1` |
| * `B`: supports `ABI-1` and `ABI-2`, base packages at `ABI-1`. |
| * `C`: supports `ABI-2`, base packages at `ABI-2`. |
| |
| If we mark `B` as a stepping stone release, then a device running `A` would |
| first update to `B`, then update to `C`. |
| |
| This idea can be taken even further by decoupling how packages are defined from |
| the ABI revision. Since we do not intend to frequently make backwards |
| incompatible changes to package layout, we can version our package layout, and |
| then state which package layout versions are supported by a given ABI revision. |
| This would allow us to install a greater range of update packages without |
| needing to create stepping stone releases. |
| |
| ### Avoiding namespace collisions in the meta.far |
| |
| The `meta.far` currently contains two package metadata files, `meta/package` |
| and `meta/contents`, as well as arbitrary user specified files. This means that |
| there is a possibility of collision between user files and any new metadata |
| files we introduce into a package. To avoid this possibility, the package ABI |
| revision will be written in a directory `meta/fuchsia.abi`. This directory name |
| follows the convention used elsewhere in Fuchsia, such as the platform FIDL |
| namespaces. The package building tools will be further updated to prevent users |
| from defining custom files in `meta/fuchsia.abi`. |
| |
| Since this is a directory controlled by the platform, it's possible to use this |
| location to store ABI-related files without colliding with user files. This |
| would allow us to evolve our notion of a package ABI revision. For example, if |
| we wanted to support multiple ABI revisions in a package, we could add |
| `meta/fuchsia.abi/abi-revision-set` to contain all the supported revisions. |
| We could drop `meta/fuchsia.abi/abi-revision` once all users have migrated to |
| the new file. |
| |
| ### Storage overhead |
| |
| The overhead of adding the package ABI revision to the meta adds 8KiB |
| uncompressed to the meta.far in the worst case, where the first 4KiB comes from |
| the FAR data chunk, and the second if the DIRNAMES section causes the first |
| content chunk to be pushed out to the next 4KiB aligned offset. However, this |
| shouldn't be a significant impact in practice, because [blobfs] has an 8KiB |
| block alignment. Furthermore, it compresses very well with the default |
| compressor [zstd]. According to the following example, this only adds 47 bytes |
| to the meta.far: |
| |
| ``` |
| # Create an empty package. |
| % mkdir empty && cd empty |
| % pm init && pm build |
| % ls -l meta.far |
| -rw-r--r-- 1 etryzelaar primarygroup 12288 Oct 5 10:21 meta.far |
| % fx chunked-compress c meta.far meta.far.compressed |
| Wrote 447 bytes (97% compression) |
| |
| # Then create a meta.far that contain's the abi-revision. |
| % cd .. && mkdir with-abi && cd with-abi |
| % pm init |
| % mkdir meta/fuchsia.abi |
| % python3 -c " |
| import struct; |
| abi_revision = int('0xC7003BF9', 16) |
| f = open('meta/fuchsia.abi/abi-revision', 'wb') |
| f.write(struct.pack('<Q', abi_revision)) |
| " |
| % pm build |
| % ls -l meta.far |
| -rw-r--r-- 1 etryzelaar primarygroup 16384 Oct 5 10:22 meta.far |
| % fx chunked-compress c meta.far meta.far.compressed |
| Wrote 494 bytes (97% compression) |
| ``` |
| |
| ## Implementation |
| |
| All package building tools will be extended to support specifying the API level |
| or the ABI revision on the command line. |
| |
| ### pm CLI |
| |
| The `pm` CLI will be extended to allow the ABI to be specified during package |
| initialization and package building. |
| |
| Example invocations: |
| |
| ``` |
| # Create a package with an API level. Under the covers this will look up the |
| # ABI revision from the SDK. |
| % pm init --api-level 5 |
| |
| # Create a package with an ABI revision. These commands are equivalent. |
| % pm init --abi-revision 0xC7003BF9 |
| % pm init --abi-revision 3338681337 |
| |
| # Build a package with an ABI revision. |
| % pm build --abi-revision 0xC7003BF9 |
| ``` |
| |
| Options: |
| |
| * `--api-level LEVEL`: The API level to encode into the package. This value |
| must be supported by the SDK. This option conflicts with the `--abi-revision` |
| flag. |
| * `--abi-revision REVISION`: The ABI revision which will be baked into the |
| package. This may be a decimal or hexadecimal integer, which must be |
| supported by the SDK. This option conflicts with the `--api-level` flag. |
| |
| Initially the `--api-level` and `--abi-revision` flags will be optional to |
| allow for petals and end-developers to implement support over a period of time. |
| Eventually this will be a required argument once the ecosystem has transitioned |
| over to specifying the flags. |
| |
| ## Performance |
| |
| This RFC introduces a trivial amount of additional work when building packages, |
| and a small overhead to the storage, so the performance implications should be |
| negligible. |
| |
| ## Ergonomics |
| |
| By itself this proposal does not significantly change the ergonomics of Fuchsia. |
| However, this will eventually enable product developers to target a specific |
| system interface for their components. This should provide the stability they |
| need while not blocking Fuchsia's ability to evolve. |
| |
| ## Backwards compatibility |
| |
| This change is backwards compatible, since this design introduces a new file |
| with no consumers. However, this functionality will eventually allow the system |
| to deprecate and eventually remove support for old ABI revisions. |
| |
| ## Security considerations |
| |
| The `meta/fuchsia.abi/abi-revision` file will need to be parsed. However, this |
| format is simple to parse, and should be easy to validate that parsers are |
| correct. |
| |
| See [RFC-0002 Security Considerations] for more details. |
| |
| ## Privacy considerations |
| |
| This proposal has no meaningful impact on privacy. |
| |
| ## Testing |
| |
| The packaging tooling will be extended to verify that packages are generated |
| with the expected ABI to be set. Since there will be a transition period, the |
| package resolver tests will be extended to verify that it can work with packages |
| that contain or not contain the `meta/fuchsia.abi/abi-revision` file. |
| |
| ## Documentation |
| |
| The [packaging documentation] will be updated to discuss how the ABI is |
| expressed in packages, and how users can select the API level or the ABI |
| revision when building packages. |
| |
| ## Drawbacks, alternatives, and unknowns |
| |
| ### Alternative ABI revision file formats |
| |
| Rather than encoding the ABI revision as a little endian integer, we could |
| instead use: |
| |
| * a human readable integer string |
| * JSON |
| * Persistent FIDL |
| |
| A little endian integer was selected because: |
| |
| * We don't expect humans to be conuming this value. Instead they should be |
| referring to the human-understandable API level. |
| * This value will be passed around as a little endian integer, so this avoids |
| extra transformations. |
| * It is simple to parse. |
| |
| ### Packages containing a set of ABI revisions |
| |
| [RFC-0002 Applications] states that packages can include multiple components, |
| each of which targets a different ABI revision. To support this, rather than |
| encoding a single ABI revision in the package, the package ABI could be a set of |
| all the component ABI revisions. |
| |
| However, as of writing there are no use cases that require a package that |
| contains components that target different system ABIs. A single number |
| simplifies the initial implemation for ABI support. It would not be difficult |
| to extend this design if this scenario becomes important. |
| |
| ### Encoding the ABI revision directly into the meta.far |
| |
| The meta.far could directly embed the ABI revision, by either: |
| |
| * Add the ABI into the [FAR index chunk]. |
| * Pros: |
| * Only adds 8 bytes of overhead. |
| * Cons: |
| * The index chunk does not have any reserved bytes, so adding this to the |
| index would require a breaking change to the FAR format. In order to make |
| this change, we would have to first implement support for reading the new |
| FAR format without using it, then create a stepping stone release, and |
| finally then migrate to the the format. |
| * It would be more difficult for consumers to read the ABI revision. The |
| package resolver does not directly expose the meta.far to users as a file, |
| so it would need to add an API to expose this information. |
| * Create a new ABI revision chunk type. |
| * Pros: |
| * It should be backwards compatible to add new chunk types. |
| * The overhead in bytes should be much smaller since we wouldn't need to |
| create a 4096 byte aligned content chunk. Chunks are 64-bit aligned, and |
| have an index overhead entry size of 24 bytes, so this approach should |
| only add 32 bytes to the uncompressed FAR format. |
| * Cons: |
| * We haven't added new chunk types to the FAR format, so it's possible FAR |
| libraries may error out or behave unexpectedly if they encounter a new |
| chunk type. |
| * We would have to update all the FAR libraries to understand the new chunk |
| type. Historically any changes to FAR libraries has been quite expensive |
| to perform. |
| * It would be more difficult for consumers to read the ABI revision. The |
| package resolver does not directly expose the meta.far to users as a file, |
| so it would need to add an API to expose this information. |
| |
| Since the storage overhead is minimal, especially with compression, we do not |
| think the space savings are worth defining a new chunk type. |
| |
| ### ABI revision in the repository metadata |
| |
| Rather than directly encoding the ABI into packages, we could instead add ABI |
| revision to the package metadata in the package repository. This has the |
| following cons: |
| |
| * The ABI revision is an intrinsic property of the package. It could be |
| difficult to make sure the ABI revision is properly expressed in the TUF |
| repositories, or in the base package lists. |
| * The packaging metadata would still contain the ABI revision, so the packaging |
| system would still have to carry the overhead. |
| |
| ### ABI revision in the upcoming persistent FIDL package metadata |
| |
| Software Delivery is writing an RFC that proposes a new [persistent-FIDL |
| packaging metadata]. We could instead store the ABI revision in this metadata by |
| updating the proposed schema to this: |
| |
| ``` |
| flexible union Contents { |
| 1: ContentsV1 v1; |
| }; |
| |
| table ContentsV1 { |
| 1: vector<fuchsia.io.Path>:MAX paths; |
| 2: vector<fuchsia.pkg.BlobId>:MAX hashes; |
| 3: vector<uint64>:MAX blob_sizes; |
| 4: uint64 abi_revision; |
| }; |
| ``` |
| |
| Advantages: |
| |
| * The advantages of consolidating all packaging metadata into a single FIDL file |
| is that we leverage FIDL tooling to keep the documentation and validation up |
| to date. Using multiple files would make it easier for our documentation and |
| validation to drift out of sync with each other. |
| * We would only add a handful of bytes to the uncompressed meta.far, rather |
| than ~4KiB. |
| |
| The disadvantages of using FIDL: |
| |
| * There is no guarantee that the persistent-FIDL based RFC will be accepted, |
| and how long it will take to implement. |
| * While the persistent-FIDL based approach is versioned, the internal version |
| would not help us migrate the packaging metadata to another file format. For |
| example, if we decided to switch from persistent FIDL to a JSON file. It |
| would be easier to use a new ABI revision to inform the packaging system to |
| look for a different file format than before. |
| |
| ## Prior art and references |
| |
| ### Android |
| |
| Android applications target an SDK version by specifying the |
| [uses-sdk] element in the |
| app manifest. |
| |
| ### Windows |
| |
| Windows applications target the OS version by specifying the [SupportedOS] |
| GUID listed in the application manifest. |
| |
| ### macOS |
| |
| macOS applications target the OS version by specifying the |
| [LSMinimumSystemVersion] in the application bundle's `Info.plist` file. |
| |
| ### iOS |
| |
| iOS applications target the OS version by specifying the [MinimumOSVersion] in |
| the application bundle's `Info.plist` file. |
| |
| [ABI revision]: 0002_platform_versioning.md#abi_revision |
| [API level]: 0002_platform_versioning.md#api_level |
| [FAR index chunk]: /docs/development/source_code/archive_format.md#index_chunk |
| [Fuchsia System Interface]: /docs/concepts/packages/system.md |
| [LSMinimumSystemVersion]: https://developer.apple.com/documentation/bundleresources/information_property_list/lsminimumsystemversion?language=objc |
| [MinimumOSVersion]: https://developer.apple.com/documentation/bundleresources/information_property_list/minimumosversion?language=objc |
| [RFC-0002 Applications]: 0002_platform_versioning.md#applications |
| [RFC-0002 Platform]: 0002_platform_versioning.md#platform |
| [RFC-0002 Security Considerations]: 0002_platform_versioning.md#security-considerations |
| [RFC-0002]: 0002_platform_versioning.md |
| [SDK version history]: https://fuchsia.googlesource.com/fuchsia/+/main/sdk/version_history.json |
| [SupportedOS]: https://docs.microsoft.com/en-us/windows/win32/win7appqual/compatibility---application-manifest#leveraging-feature-capabilities |
| [blobfs]: /docs/concepts/filesystems/random-access-compression.md |
| [meta.far]: /docs/concepts/packages/package.md#meta-far |
| [package]: /docs/concepts/packages/package.md |
| [packaging documentation]: /docs/concepts/packages/package.md |
| [persistent-FIDL packaging metadata]: https://fuchsia-review.googlesource.com/c/fuchsia/+/586937 |
| [system image]: 0072_standalone_image_assembly_tool.md#system_image_artifacts |
| [uses-sdk]: https://developer.android.com/guide/topics/manifest/uses-sdk-element |
| [zstd]: https://github.com/facebook/zstd |