docs/contribute/governance/rfcs/0229_fidl_2023.md - fuchsia - Git at Google

 <!-- mdformat off(templates not supported) -->
 {% set rfcid = "RFC-0229" %}
 {% include "docs/contribute/governance/rfcs/_common/_rfc_header.md" %}
 # {{ rfc.name }}: {{ rfc.title }}
 {# Fuchsia RFCs use templates to display various fields from _rfcs.yaml. View the #}
 {# fully rendered RFCs at https://fuchsia.dev/fuchsia-src/contribute/governance/rfcs #}
 <!-- SET the `rfcid` VAR ABOVE. DO NOT EDIT ANYTHING ELSE ABOVE THIS LINE. -->

 <!-- mdformat on -->

 ## Summary

 This document establishes that the present FIDL wire format and interaction
 model, to be known as **FIDL 2023**, will be supported for at least three years.

 ## Motivation

 FIDL is many things: language, compiler, bindings, wire format, and interaction
 model. This RFC is concerned with the last two, because they are what matter for
 ABI compatibility. Both have changed significantly over the years. For example,
 we recently migrated the wire format for [RFC-0113: Efficient
 envelopes][rfc-0113], and revised the interaction model for [RFC-0138: Handling
 unknown interactions][rfc-0138]. These changes were possible because we were
 aware of all code using FIDL and could update it all. In the future, as the
 platform matures, this will not be the case. We need to start thinking about
 long term ABI compatibility.

 ## Stakeholders

 _Facilitator:_ abarth@google.com

 _Reviewers:_ hjfreyer@google.com, ianloic@google.com, sethladd@google.com,
 ddorwin@google.com

 _Consulted:_ quiche@google.com, wilkinsonclay@google.com

 _Socialization:_ The plan for FIDL 2023 was discussed several times in FIDL team
 meetings before this RFC was written.

 ## Design

 The goal of this proposal is to define FIDL 2023, and ratify the plan to support
 it for at least three years from the acceptance of this RFC.

 ### Scope

 FIDL 2023 refers to the state of the FIDL **wire format** and **interaction
 model** as of Sept 2023. All other parts of FIDL are out of scope.

 The wire format specifies how FIDL messages are encoded and decoded on the wire.
 This includes transactional messages used in the interaction model, and other
 use cases via the [RFC-0120] API for standalone encoding and decoding. The wire
 format is already [documented][wire-format], but some things are worth noting:

 * The magic number is 1. Decoding must fail for any other magic number.
 * The at-rest flags include the "v2 indicator" `0x0002`. Decoding must fail if
   this bit is not set.
 * The other flag bits must not be validated. Decoding must _not_ fail due to the
   value of presently unused at-rest flags or dynamic flags.

 The interaction model specifies how FIDL clients and servers must behave. For
 example, it includes the fact that a FIDL peer must close its channel endpoint
 upon failing to decode an incoming message. The interaction model does not yet
 have comprehensive documentation (see [Documentation](#documentation)), but it
 is tested in the FIDL [dynsuite].

 FIDL 2023 includes changes to the wire format and interaction model from all
 RFCs to date, notably the following:

 * [RFC-0061: Extensible unions][rfc-0061]
 * [RFC-0113: Efficient envelopes][rfc-0113]
 * [RFC-0120: Standalone use of the FIDL wire format][rfc-0120]
 * [RFC-0137: Discard unknown data in FIDL][rfc-0137]
 * [RFC-0138: Handling unknown interactions][rfc-0138]

 Note that some RFCs have not been implemented in all bindings. In particular,
 the Go and Dart bindings have not been updated for [RFC-0137]. They are
 therefore not fully FIDL 2023 compliant.

 ### Support

 FIDL 2023 is a [long-term support][] (LTS) version of the FIDL wire format and
 interaction model. It will be supported for at least three years from the date
 this RFC is accepted.

 This does not preclude adding new features to FIDL. It just requires maintaining
 compatibility with FIDL 2023. Furthermore, if we add any new features, they will
 not automatically be part of the LTS guarantee.

 Concretely, suppose someone compiles a program that uses FIDL 2023. That binary
 should work correctly, to the extent that "working correctly" depends on FIDL,
 throughout the support period, assuming:

 * the rest of the software stack (notably, Zircon and Component Framework)
   maintains similar long term compatibility;
 * the FIDL protocols it uses are unchanged or evolve in ABI compatible ways;
 * the peers it communicates with maintain application-level compatibility.

 Here are some positive examples (allowed under this proposal) and negative
 examples (not allowed) to illustrate what FIDL 2023 support means.

 > *Positive example.* The program uses a protocol with a struct. We add a field
 > to this struct, breaking ABI. The program fails to decode a message when
 > communicating with a newer peer. This **does not** violate FIDL 2023.

 > *Negative example.* We change the wire format to use a sparse representation
 > for tables. The program fails to decode a message using this representation.
 > This **violates** FIDL 2023, even if we soft transition the change using
 > flags, because that still requires recompiling the program.

 > *Positive example.* We add a new type to FIDL, `uint128`. The program is
 > unable to decode this type. This **does not** violate FIDL 2023, because
 > `uint128` is not part of FIDL 2023. If the protocols used by the program do
 > not use `uint128`, there is no problem. If they evolve in an ABI compatible
 > way to use `uint128` _after_ the program is compiled, there is also no issue,
 > because such values would necessarily be in envelopes or in new methods.
 > Finally, if the change to use `uint128` occurs _before_ compiling the program,
 > then the program can safely decode `uint128` values.

 > *Positive example.* We change the interaction model to allow server-initiated
 > two-way methods. This **does not** violate FIDL 2023 for reasons similar to
 > the `uint128` example above.

 > *Negative example.* We change the interaction model to allow clients to send
 > epitaphs to servers. This **violates** FIDL 2023 because the program may
 > implement a server and will be unable to handle such a message correctly.

 These examples show that the wire format and interaction model can still change,
 but changes must be additive and opt-in (e.g., opting to use the `uint128` type
 in a new API). Code that does not opt in to new features must continue to work
 for the duration of FIDL 2023 support.

 ## Performance

 Committing ourselves to long term support for FIDL 2023 means we cannot improve
 performance of existing features by changing the wire format.

 ## Ergonomics

 This proposal has no impact on ergonomics.

 ## Backwards compatibility

 This proposal is all about backwards compatibility so it is discussed
 throughout rather than in this section.

 ## Security considerations

 We may break the commitment to fully supporting FIDL 2023 if we discover a
 security problem that requires making incompatible changes.

 ## Privacy considerations

 We may break the commitment to fully supporting FIDL 2023 if we discover a
 privacy problem that requires making incompatible changes.

 ## Testing

 We will continue to test the wire format with [GIDL] and the interaction model
 with the [dynsuite]. We will also invest in growing them to increase coverage of
 FIDL 2023 features and edge cases. This will help ensure that we keep our
 promise of long term support.

 ## Documentation {#documentation}

 We must update the [FIDL wire format specification][wire-format]. For example,
 it could say that everything is part of FIDL 2023 unless otherwise noted.

 The [FIDL bindings specification][bindings-spec] documents some aspects of the
 interaction model, but it is mixed with guidelines on the structure and style of
 generated bindings. We must instead create a new "FIDL interaction model
 specification". Each statement in it should be backed by a [dynsuite] test.

 ## Drawbacks, alternatives, and unknowns

 ### Drawback: Harder to improve FIDL

 This proposal will make it harder to evolve and improve FIDL. For example,
 changing the wire format will be much more costly. Not only would we have to
 execute a soft transition with flags (as before), but we would have to keep the
 old read path for much longer, even if we think nothing is relying on it.

 ### Alternative: A different length of time

 This proposal indicates we are taking long term compatibility more seriously.
 Three years feels roughly appropriate. It's longer than we've ever provided
 compatibility before. But there's not much more behind it than that. We could
 commit to a different number of years.

 ### Unknown: ABI compatibility in practice

 We are still learning what long term ABI compatibility on Fuchsia looks like in
 practice. Ideally, FIDL provides a stable foundation that developers can use to
 evolve applications and services. In the past, we've made breaking changes to
 FIDL to better support evolvability and compatibility, for example in [RFC-0061:
 Extensible unions][rfc-0061]. Does FIDL 2023 have all the features needed by
 layers above to provide long term compatibility, or is it missing some?

 ## Prior art and references

 All successful operating systems provide some degree of long term compatibility.
 There is no question that Fuchsia, and hence FIDL, needs to as well.

 The concept of [long-term support] releases is common in open source software.
 For example, the Linux kernel designates certain releases as "longterm".

 [rfc-0061]: /docs/contribute/governance/rfcs/0061_extensible_unions.md
 [rfc-0113]: /docs/contribute/governance/rfcs/0113_efficient_envelopes.md
 [rfc-0120]: /docs/contribute/governance/rfcs/0120_standalone_use_of_fidl_wire_format.md
 [rfc-0137]: /docs/contribute/governance/rfcs/0137_discard_unknown_data_in_fidl.md
 [rfc-0138]: /docs/contribute/governance/rfcs/0138_handling_unknown_interactions.md
 [at-rest flags]: /docs/contribute/governance/rfcs/0138_handling_unknown_interactions.md#transactional-message-header-v4
 [GIDL]: /tools/fidl/gidl/README.md
 [dynsuite]: /src/tests/fidl/dynsuite/README.md
 [wire-format]: /docs/reference/fidl/language/wire-format/README.md
 [bindings-spec]: /docs/reference/fidl/language/bindings-spec.md
 [long-term support]: https://en.wikipedia.org/wiki/Long-term_support
	<!-- mdformat off(templates not supported) -->
	{% set rfcid = "RFC-0229" %}
	{% include "docs/contribute/governance/rfcs/_common/_rfc_header.md" %}
	# {{ rfc.name }}: {{ rfc.title }}
	{# Fuchsia RFCs use templates to display various fields from _rfcs.yaml. View the #}
	{# fully rendered RFCs at https://fuchsia.dev/fuchsia-src/contribute/governance/rfcs #}
	<!-- SET the `rfcid` VAR ABOVE. DO NOT EDIT ANYTHING ELSE ABOVE THIS LINE. -->

	<!-- mdformat on -->

	## Summary

	This document establishes that the present FIDL wire format and interaction
	model, to be known as FIDL 2023, will be supported for at least three years.

	## Motivation

	FIDL is many things: language, compiler, bindings, wire format, and interaction
	model. This RFC is concerned with the last two, because they are what matter for
	ABI compatibility. Both have changed significantly over the years. For example,
	we recently migrated the wire format for [RFC-0113: Efficient
	envelopes][rfc-0113], and revised the interaction model for [RFC-0138: Handling
	unknown interactions][rfc-0138]. These changes were possible because we were
	aware of all code using FIDL and could update it all. In the future, as the
	platform matures, this will not be the case. We need to start thinking about
	long term ABI compatibility.

	## Stakeholders

	_Facilitator:_ abarth@google.com

	_Reviewers:_ hjfreyer@google.com, ianloic@google.com, sethladd@google.com,
	ddorwin@google.com

	_Consulted:_ quiche@google.com, wilkinsonclay@google.com

	_Socialization:_ The plan for FIDL 2023 was discussed several times in FIDL team
	meetings before this RFC was written.

	## Design

	The goal of this proposal is to define FIDL 2023, and ratify the plan to support
	it for at least three years from the acceptance of this RFC.

	### Scope

	FIDL 2023 refers to the state of the FIDL wire format and **interaction
	model** as of Sept 2023. All other parts of FIDL are out of scope.

	The wire format specifies how FIDL messages are encoded and decoded on the wire.
	This includes transactional messages used in the interaction model, and other
	use cases via the [RFC-0120] API for standalone encoding and decoding. The wire
	format is already [documented][wire-format], but some things are worth noting:

	* The magic number is 1. Decoding must fail for any other magic number.
	* The at-rest flags include the "v2 indicator" `0x0002`. Decoding must fail if
	this bit is not set.
	* The other flag bits must not be validated. Decoding must _not_ fail due to the
	value of presently unused at-rest flags or dynamic flags.

	The interaction model specifies how FIDL clients and servers must behave. For
	example, it includes the fact that a FIDL peer must close its channel endpoint
	upon failing to decode an incoming message. The interaction model does not yet
	have comprehensive documentation (see [Documentation](#documentation)), but it
	is tested in the FIDL [dynsuite].

	FIDL 2023 includes changes to the wire format and interaction model from all
	RFCs to date, notably the following:

	* [RFC-0061: Extensible unions][rfc-0061]
	* [RFC-0113: Efficient envelopes][rfc-0113]
	* [RFC-0120: Standalone use of the FIDL wire format][rfc-0120]
	* [RFC-0137: Discard unknown data in FIDL][rfc-0137]
	* [RFC-0138: Handling unknown interactions][rfc-0138]

	Note that some RFCs have not been implemented in all bindings. In particular,
	the Go and Dart bindings have not been updated for [RFC-0137]. They are
	therefore not fully FIDL 2023 compliant.

	### Support

	FIDL 2023 is a [long-term support][] (LTS) version of the FIDL wire format and
	interaction model. It will be supported for at least three years from the date
	this RFC is accepted.

	This does not preclude adding new features to FIDL. It just requires maintaining
	compatibility with FIDL 2023. Furthermore, if we add any new features, they will
	not automatically be part of the LTS guarantee.

	Concretely, suppose someone compiles a program that uses FIDL 2023. That binary
	should work correctly, to the extent that "working correctly" depends on FIDL,
	throughout the support period, assuming:

	* the rest of the software stack (notably, Zircon and Component Framework)
	maintains similar long term compatibility;
	* the FIDL protocols it uses are unchanged or evolve in ABI compatible ways;
	* the peers it communicates with maintain application-level compatibility.

	Here are some positive examples (allowed under this proposal) and negative
	examples (not allowed) to illustrate what FIDL 2023 support means.

	> Positive example. The program uses a protocol with a struct. We add a field
	> to this struct, breaking ABI. The program fails to decode a message when
	> communicating with a newer peer. This does not violate FIDL 2023.

	> Negative example. We change the wire format to use a sparse representation
	> for tables. The program fails to decode a message using this representation.
	> This violates FIDL 2023, even if we soft transition the change using
	> flags, because that still requires recompiling the program.

	> Positive example. We add a new type to FIDL, `uint128`. The program is
	> unable to decode this type. This does not violate FIDL 2023, because
	> `uint128` is not part of FIDL 2023. If the protocols used by the program do
	> not use `uint128`, there is no problem. If they evolve in an ABI compatible
	> way to use `uint128` _after_ the program is compiled, there is also no issue,
	> because such values would necessarily be in envelopes or in new methods.
	> Finally, if the change to use `uint128` occurs _before_ compiling the program,
	> then the program can safely decode `uint128` values.

	> Positive example. We change the interaction model to allow server-initiated
	> two-way methods. This does not violate FIDL 2023 for reasons similar to
	> the `uint128` example above.

	> Negative example. We change the interaction model to allow clients to send
	> epitaphs to servers. This violates FIDL 2023 because the program may
	> implement a server and will be unable to handle such a message correctly.

	These examples show that the wire format and interaction model can still change,
	but changes must be additive and opt-in (e.g., opting to use the `uint128` type
	in a new API). Code that does not opt in to new features must continue to work
	for the duration of FIDL 2023 support.

	## Performance

	Committing ourselves to long term support for FIDL 2023 means we cannot improve
	performance of existing features by changing the wire format.

	## Ergonomics

	This proposal has no impact on ergonomics.

	## Backwards compatibility

	This proposal is all about backwards compatibility so it is discussed
	throughout rather than in this section.

	## Security considerations

	We may break the commitment to fully supporting FIDL 2023 if we discover a
	security problem that requires making incompatible changes.

	## Privacy considerations

	We may break the commitment to fully supporting FIDL 2023 if we discover a
	privacy problem that requires making incompatible changes.

	## Testing

	We will continue to test the wire format with [GIDL] and the interaction model
	with the [dynsuite]. We will also invest in growing them to increase coverage of
	FIDL 2023 features and edge cases. This will help ensure that we keep our
	promise of long term support.

	## Documentation {#documentation}

	We must update the [FIDL wire format specification][wire-format]. For example,
	it could say that everything is part of FIDL 2023 unless otherwise noted.

	The [FIDL bindings specification][bindings-spec] documents some aspects of the
	interaction model, but it is mixed with guidelines on the structure and style of
	generated bindings. We must instead create a new "FIDL interaction model
	specification". Each statement in it should be backed by a [dynsuite] test.

	## Drawbacks, alternatives, and unknowns

	### Drawback: Harder to improve FIDL

	This proposal will make it harder to evolve and improve FIDL. For example,
	changing the wire format will be much more costly. Not only would we have to
	execute a soft transition with flags (as before), but we would have to keep the
	old read path for much longer, even if we think nothing is relying on it.

	### Alternative: A different length of time

	This proposal indicates we are taking long term compatibility more seriously.
	Three years feels roughly appropriate. It's longer than we've ever provided
	compatibility before. But there's not much more behind it than that. We could
	commit to a different number of years.

	### Unknown: ABI compatibility in practice

	We are still learning what long term ABI compatibility on Fuchsia looks like in
	practice. Ideally, FIDL provides a stable foundation that developers can use to
	evolve applications and services. In the past, we've made breaking changes to
	FIDL to better support evolvability and compatibility, for example in [RFC-0061:
	Extensible unions][rfc-0061]. Does FIDL 2023 have all the features needed by
	layers above to provide long term compatibility, or is it missing some?

	## Prior art and references

	All successful operating systems provide some degree of long term compatibility.
	There is no question that Fuchsia, and hence FIDL, needs to as well.

	The concept of [long-term support] releases is common in open source software.
	For example, the Linux kernel designates certain releases as "longterm".

	[rfc-0061]: /docs/contribute/governance/rfcs/0061_extensible_unions.md
	[rfc-0113]: /docs/contribute/governance/rfcs/0113_efficient_envelopes.md
	[rfc-0120]: /docs/contribute/governance/rfcs/0120_standalone_use_of_fidl_wire_format.md
	[rfc-0137]: /docs/contribute/governance/rfcs/0137_discard_unknown_data_in_fidl.md
	[rfc-0138]: /docs/contribute/governance/rfcs/0138_handling_unknown_interactions.md
	[at-rest flags]: /docs/contribute/governance/rfcs/0138_handling_unknown_interactions.md#transactional-message-header-v4
	[GIDL]: /tools/fidl/gidl/README.md
	[dynsuite]: /src/tests/fidl/dynsuite/README.md
	[wire-format]: /docs/reference/fidl/language/wire-format/README.md
	[bindings-spec]: /docs/reference/fidl/language/bindings-spec.md
	[long-term support]: https://en.wikipedia.org/wiki/Long-term_support