docs/reference/fidl/language/bindings-spec.md

# FIDL bindings specification

This document is a specification of Fuchsia Interface Definition Language
(**FIDL**) bindings. It is meant to provide guidance and best practices for
bindings authors, and recommend specific approaches for their ergonomic use.

In this document, the following keywords are to be interpreted as described in
[RFC2119][RFC2119]: **MAY**, **MUST**, **MUST NOT**, **OPTIONAL**,
**RECOMMENDED**, **REQUIRED**, **SHALL**, **SHALL NOT**, **SHOULD**, **SHOULD
NOT**.

## Generated code indication

A comment must be placed at the top of machine-generated code to indicate it is
machine generated. For languages with a standard on how to indicate generated
sources (as opposed to human-written code), that standard must be followed.

In [Go][go-generated-code-comment] for instance, generated sources must be
marked with a comment following the pattern

```go
// Code generated by <tool>; DO NOT EDIT.
```

## Scoping

It is RECOMMENDED to namespace machine-generated code to avoid clashing with
user-defined symbols. This can be implement using scoping constructs provided by
the language, like namespaces in C++, modules in Rust, or packages in Go and
Dart. If the generated scope can have a name, it SHOULD be named using
components of the name of the FIDL library that contains the definitions for the
generated code, which allows each FIDL library to exist in a unique scope. In
cases where scoping is not possible and the namespace is shared, some processing
of the generated names (see [Naming](#naming)) may be necessary.

## Naming {#naming}

In general, the names used in the generated code SHOULD match the names used in
the FIDL definition. Possible exceptions are listed in the following sections.

For inline layouts, bindings SHOULD use the names generated by `fidlc` since
they are guaranteed to be unique.

Bindings MAY generated scoped names corresponding to a FIDL name's naming context
if it is supported in the target language. For example, for some FIDL:

```
type Outer = struct {
  middle struct {
    inner struct {};
  };
};
```

The generated code would allow referring to the value corresponding to the
innermost FIDL struct using a name scoped to the parent naming context (e.g.
in C++, something like `Outer::Middle::Inner`).

### Casing

Casing changes SHOULD be made to fit the idiomatic style of the language (e.g.
using snake_case or CamelCase). `fidlc` will ensure that identifier uniqueness
is enforced taking into account potential casing differences (see [RFC-0040][rfc0040]).

### Reserved keywords and name clashes

The generated code MUST take into account the reserved keywords in the target
language to avoid unexpected when a keyword from the target language is used in
the FIDL definition. An example scheme would be to prefix conflicting names with
an underscore `_` (assuming no keywords begin with an underscore).

The generated code MUST avoid generating code that causes naming conflicts. For
example, in a function whose parameters are generated based on a FIDL
definition, it MUST be impossible for the names of the local variables in the
generated to clash with possible generated names.

## Ordinals

### Method ordinals

Ordinals used for methods are large 64-bit numbers. Bindings SHOULD emit these
ordinals in hexadecimal, i.e. `0x60e700e002995ef8`, not `6982550709377523448`.

### Union, and table ordinals

Ordinals used for `union` and `table` start at 1, and must form a dense space.
Therefore, these numbers are typically small, and bindings SHOULD emit these
ordinals in decimal notation.

## Native types

It is RECOMMENDED that bindings use the most specific and ergonomic native types
where possible when converting built-in FIDL types to native types in the target
language. For example, the Dart bindings use `Int32List` to represent a
`vector<int32>:N` and `array<int32>:N` rather than the more generic `List<int>`.

## Generated types and values

### Constant support

Generated code MUST generate variables containing matching values for each
`const` definition in the corresponding FIDL. These variables SHOULD be marked
as immutable in languages that support this (e.g. `const` in C++, Rust, and Go,
or `final` in Dart).

### Bits support

Bindings MUST provide generated values for each bits member. They MAY also
generate values representing the bits with no flags set, as well as the bits
with every flag set (the "bits mask"). These values SHOULD be scoped to each set
of bits.

It is RECOMMENDED to support the following operators over generated values:

* bitwise and, i.e `&`
* bitwise or, i.e `|`
* bitwise exclusive-or, i.e `^`
* bitwise not, i.e `~`
* bitwise difference, i.e all the bits present in one operand, except the bits
  present in the other operand. This is usually represented by the `-` operator.

An invariant of FIDL bitwise operations is that they should not introduce
unknown bits unless a corresponding unknown bit occur in the source operands.
All recommended operators naturally have this property, except `bitwise not`.
Implementations of the `bitwise not` operation MUST further mask the resulting
value with the mask of all values. In pseudo code:

```
~value1   means   mask & ~bits_of(value1)
```

This mask value is provided in the [JSON IR][jsonir] for convenience.

In languages where operator overloading is supported, such as C++, the `bitwise
not` operation MUST be implemented by overloading the built in operator in a
manner that always unsets the unknown members of the bitfield.  In languages
that do not support operator overloading, such as Go, values SHOULD provide an
`InvertBits()` method (cased in the manner most appropriate for the language)
for executing the masked inversion.

The bitwise difference operator SHOULD be preferred over the bitwise not
operator when clearing bits, because the former preserves unknown bits:

```
// Unknown bits in value1 are preserved.
value1 = value1 - value2

// Unknown bits in value1 are cleared, even if the user may only intend to
// clear just the bits in value2.
value1 = value1 & ~value2
```

Bindings SHOULD NOT support other operators which could result in invalid
bits value (or risk a non-obvious translation of their meaning), e.g.:

* bitwise shifts, i.e `<<` or `>>`
* bitwise unsigned shift, i.e `>>>`

For cases where the generated code includes a type wrapping the underlying
numeric bits value, it SHOULD be possible to convert between the raw value and
the wrapper type. It is RECOMMENDED for this conversion to be explicit.

Bindings MAY provide functions for converting a primitive value of the
underlying type of a `bits` to the `bits` type itself. These converters may be
of several flavors:

* Possibly failing (or returning null) if the input value contains any unknown
  bits.
* Truncates any unknown bits from the input value.
* For [flexible](#strict-flexible) bits only: Keeps any unknown bits from the
  input value.

#### Unknown data {#unknown-bits}

For [flexible](#strict-flexible) bits:

* Bindings MUST provide methods for checking if the value contains any unknown
  bits, and additionally MAY provide methods for retrieving those unknown bits.
* The bitwise not operator unsets all unknown members, regardless of
  their previous values (but works as expected for known members). The
  other bitwise operators retain the same semantics for unknown bits members as
  for known members.

Strict bits MAY provide the above APIs as well in order to simplify [transitions
between strict and flexible][source-compatible].

In some languages, it is difficult or impossible to prevent users from
manually creating an instance of a `bits` type from a primitive, therefore
preventing bindings designers from restricting strict bits values to having a
properly restricted domain. In this case, bindings authors SHOULD provide the
unknown data related APIs for strict bits.

In languages with compile-checked deprecation warnings such as Rust, the unknown
data related APIs SHOULD be provided for strict bits but marked as deprecated.

### Enum support

Bindings MUST provide generated values for each enum member. These values SHOULD
be scoped to each enum.

For cases where the generated code includes a type wrapping the underlying
numeric enum value, it SHOULD be possible to convert between the raw value and
the wrapper type. It is RECOMMENDED for this conversion to be explicit.

#### Unknown data {#unknown-enums}

For [flexible](#strict-flexible) enums:

* Bindings MUST provide a way for users to determine if an enum is unknown,
  including making it possible to match against the enum (for
  languages that support `switch`, `match`, or similar constructs).
* Bindings MAY expose the (possibly unknown) underlying raw value of the enum.
* Bindings MUST provide a way to obtain a valid unknown enum, without the
  user needing to provide an explicit unknown raw primitive value. If one of the
  enum members is annotated with the [`@unknown`][unknown-attr] attribute,
  then this unknown enum constructor MUST use the value of the annotated
  member. Otherwise, the value used by the unknown constructor is unspecified.
* The `@unknown` member MUST be treated as unknown in any
  function that determines whether a value is unknown.

### Struct support

Bindings MUST provide a type for each struct that supports the following
operations:

* Construction with explicit values for each member.
* Reading and writing members.

### Union support

Bindings MUST provide a type for each union that supports the following
operations:

* Construction with an explicit variant set. It is NOT RECOMMENDED for bindings
  to offer construction without a variant. This should be considered only for
  performance reasons or due to limitations of the target language.
* Reading/writing the variant of the union and the data associated with that
  variant.

For languages without union types or union value literals, it is RECOMMENDED to
support factory methods for constructing new unions given a value for one of the
possible variants. For example, in a C like language, this would allow replacing
code like:

```C
my_union_t foo;
foo.set_variant(bar);
do_stuff(foo);
```

with something like:

```C
do_stuff(my_union_with_variant(bar));
```

These factory methods SHOULD be named as "[Type]-with-[Variant]", cased properly
for the target language.

Examples of this exist for the
[HLCPP](https://fuchsia-review.googlesource.com/c/fuchsia/+/309246/) and
[Go](https://fuchsia-review.googlesource.com/c/fuchsia/+/313205/) bindings.

#### Unknown data {#unknown-unions}

For [flexible unions](#strict-flexible):

* When the ordinal is unknown, decoding MUST succeed, but re-encoding MUST fail.
* Bindings MUST provide a way to determine if the union has an unknown variant.
* Bindings MAY provide a way to access the unknown variant's ordinal.
* Bindings MAY provide a constructor to create a union with an unknown variant.
  * The constructor MUST be named to discourage use in production code, e.g.
    `unknown_variant_for_testing()`.
  * The constructor MUST NOT allow the user to choose the ordinal.
  * Having a constructor prevents end-developers from constructing unions with
    unknown variants in roundabout ways, such as by manually decoding raw bytes.

### Table support

Bindings MUST provide a type for each table that supports the following
operations:

* Construction where specifying values for each member is optional.
* Reading and writing each member, including checking whether a given member is
  set. These SHOULD follow the naming scheme: `get_[member]`, `set_[member]`,
  and `has_[member]`, cased properly for the target language.

Bindings MAY provide constructors for tables that only require specifying values
for fields that have a value. For example, in Rust this can be accomplished
using the `::EMTPY` constant along with struct update syntax. Supporting
construction this ways allows users to write code that is robust against
addition of new fields to the table.

#### Unknown data {#unknown-tables}

All tables are [flexible](#strict-flexible).

When there are unknown fields, decoding and re-encoding MUST succeed.
Re-encoding MUST omit the unknown fields.

Bindings MAY provide a way to determine whether the table included any unknown
fields during decoding. They MAY provide a way to access their ordinals.

Bindings MUST NOT provide a way to create a table with unknown fields or to set
unknown fields on an existing table.

### Strict and flexible types {#strict-flexible}

Strict types MUST fail to decode when encountering any unknown data.
Flexible types MUST succeed when decoding a value with unknown data.

Examples of flexible FIDL types and their behavior with respect to unknowns:

FIDL type      | Access to unknowns  | Re-encode fidelity
---------------|---------------------|-------------------
flexible bits  | raw integer         | lossless
flexible enum  | raw integer         | lossless
flexible union | boolean or ordinal  | _fails_
table          | boolean or ordinals | lossy

In general, the underlying unknown data can either be discarded during decoding,
or be stored within the decoded type. In either case, the type SHOULD indicate
whether it encountered unknown data or not when decoding. Refer to the [enum
support](#unknown-enums), [bits support](#unknown-bits), [union
support](#unknown-unions), and [table support](#unknown-tables) sections for
specific guidance on the design of these APIs.

Bindings authors SHOULD favor optimizing `strict` types, possibly at the expense
of `flexible` types. For example, if there is a design tradeoff between the two,
bindings authors SHOULD prefer optimizing the `strict` types.

Changing a type from strict to flexible MUST be [transitionable][soft-transitions].

### Value types and resource types {#value-resource}

Value types MUST NOT contain handles, and resource types MAY contain handles.

In the interaction between value types and flexible types, the flexible type
requirements takes priority. Specifically, decoding a flexible value type that
contains unknown handles MUST succeed.

## Protocol support

### Events

Bindings MUST provide support for handling or ignoring events in a protocol.
Bindings MAY allow the user to specify handling logic for some events, and omit
handling logic for some other events in a protocol.

When the handling logic for an event was not specified by the user, bindings
MUST carry on normal communication upon receiving the event. In other words,
it is not an error to send an event for which the user did not specify
corresponding handling logic on the client side.

Bindings SHOULD seek to minimize racy behavior between specifying an event
handler and events arriving on the endpoint.

Bindings MUST close the connection upon receiving an unknown
[strict event][strict-event].

### Domain error types

It is OPTIONAL that bindings provide some form of special support for protocol
methods with an error type matching the idiomatic way errors are handled in the
target language.

For example, languages that provide some form of a "result" type (i.e. a union
type that contains a "success" variant and an "error" variant), such as Rust's
`result::Result`, or `fpromise::result` in C++ MAY provide automatic conversions
to and from these types when receiving or sending method responses with an error
type.

Languages with exceptions can have the generated protocol method code optionally
raise an exception corresponding to the error type.

In cases where this is not possible, the generated code MAY provide convenience
functions for responding directly with a successful response or error value, or
for receiving an error type response, in order avoid boilerplate user code for
initializing result unions.

## Error handling

Protocols MAY surface transport errors back to the user. Transport errors can be
categorized as errors encountered when converting between the native type and
the wire format data, or as errors from the underlying transport mechanism (for
example, an error obtained from calling `zx_channel_write_etc`). These errors
MAY consist of the error status, as well as any other diagnostics information.

We define these transport errors as terminal. The rest of the document may
additionally specify other situations as terminal errors, such as incorrect
transaction IDs.

* Validation errors during encode, if validation is performed.
* Decode errors.
* Errors from the underlying transport mechanism.

By comparison, domain errors (in methods declared with `error` syntax) and
framework errors (in `flexible` two-way methods) are not terminal.

### Terminal error handling

Bindings MUST provide asynchronous client and server APIs that own the
underlying endpoint. When a terminal error occurs over a connection, the client
and server APIs MUST teardown the connection by closing the underlying endpoint.

Because the IPC transport model of FIDL does not contain transient errors, there
is no value in e.g. retrying sending the same reply. Triggering teardown on
error encourages this way of bindings usage and simplifies error handling.

Bindings MAY provide synchronous client and server APIs. In synchronous APIs,
closing the endpoint on terminal errors generally requires taking locks. If that
is undesirable for performance reasons, those APIs MAY leave the connection open
on terminal errors, and SHOULD be clearly documented accordingly. The
asynchronous flavors SHOULD be the recommended flavors of APIs.

Bindings MAY provide client and server APIs that do not own the underlying
endpoint, to cater to low level use cases. Those APIs cannot close the endpoint
on terminal errors, and SHOULD be clearly documented accordingly. The owning
flavors SHOULD be the recommended flavors of APIs.

### Peer closed special handling

When the underlying transport mechanism reports that the peer endpoint is
closed during message sending (e.g. getting a `ZX_ERR_PEER_CLOSED` error when
writing to the channel), the client/server MUST first read and process any
remaining messages, before surfacing the transport error to the user and closing
the connection.

When the underlying transport mechanism notifies that the peer endpoint is
closed during message waiting (e.g. observing a `ZX_CHANNEL_PEER_CLOSED` signal
when waiting for signals on the channel), the client/server MUST first read and
process any remaining message, before surfacing the transport error to the user
and closing the connection.

This is to stay coherent with the read semantics of a channel: given a pair of
endpoints `A <-> B`, suppose several messages are written into `B`, and then
`B` is closed. One can keep reading from `A` without observing peer closed
errors until all lingering messages are drained. In other words, "peer closed"
is not a fatal error until no more messages could be read from the endpoint.

## Handle type and rights checking

Bindings MUST enforce handle type and rights checking in both the incoming and
outgoing directions. This means that `zx_channel_write_etc`,
`zx_channel_read_etc` and `zx_channel_call_etc` MUST be used instead of their
non-etc equivalents.

In the outgoing direction, rights and type information must be populated based
on the FIDL definition. Concretely, this metadata should be placed in a
`zx_handle_disposition_t` in order to invoke `zx_channel_write_etc` or
`zx_channel_call_etc`. These system calls will perform type and rights checking
on behalf of the caller.

In the incoming direction, `zx_channel_read_etc` and `zx_channel_call_etc`
provide type and rights information in the form of `zx_handle_info_t` objects.
The bindings themselves must perform the appropriate checks as follows:

Suppose a handle *h* is read and its rights in the FIDL file are *R*:

* It is an error for handle *h* to be missing rights that are present in rights
  *R*. The channel MUST be closed if this condition is encountered.
* If handle *h* has more rights than rights *R*, its rights MUST be reduced to
  *R* through `zx_handle_replace`.

Additionally, it is an error for *h* to have the wrong type. The channel
MUST be closed if this condition is encountered.

See [Life of a handle] for a detailed example.

## Iovec Support

Bindings may optionally use the vectorized `zx_channel_write_etc` and
`zx_channel_call_etc` syscalls. When these are used, the first iovec entry MUST
be present and large enough to hold the FIDL
[transactional message header](/docs/reference/fidl/language/wire-format#transactional-messages)
(16 bytes).

### Attributes

Bindings MUST support the following [attributes][attributes]:

* `@transitional`

## Best practices

### Alternative output

It is OPTIONAL for bindings to provide alternative output methods to the FIDL
wire format.

One type of output could be user-friendly debug printing for the generated
types. For example, printing a value of the bits:

```fidl
{% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="examples/fidl/fuchsia.examples.docs/misc.test.fidl" region_tag="mode" %}
```

could print the string `"Mode.Read | Mode.Write"` rather than the raw value
`"0b11"`.

Similar user-friendly printing can be implemented for each of the generated FIDL
types.

### Message memory allocation

Bindings MAY provide the option for users to provide their own memory to use
when sending or receiving messages, which allows the user to control memory
allocation.

### Wire format memory layout

Bindings MAY have the in memory layout of the generated FIDL types match the
wire format of the type. Doing this can in theory avoid extra copies, as the
data can be used directly as the transactional message, or vice versa. In
practice, sending a FIDL message may still involve a copying step where the
components of a message are assembled into a contiguous chunk of memory (called
"linearization"). The downside of such an approach is that it makes the bindings
more rigid: changes to the FIDL wire format become more complex to implement.

The [LLCPP bindings][llcpp] are the only binding that take this
approach.

### Equality comparison {#equality-comparison}

For aggregate types such as structs, tables, and unions, bindings MAY provide
equality operators that perform a deep comparison on two instances of the same
type. These operators SHOULD NOT be provided for resource types (see
[RFC-0057][rfc0057] and [deep equality](#deep-equality)) as comparison of
handles is not possible. Avoiding exposing equality operators for resource types
prevents source breakages caused by an equality operation 'disappearing' when a
handle is added to the type.

### Copying

For aggregate types such as structs, tables, and unions, bindings MAY provide
functionality for copying instances of these types. Copying SHOULD NOT be
provided for resource types (see [RFC-0057][rfc0057]) as making copies of handles
is not guaranteed to succeed. Avoiding exposing copy operators for resource
types prevents source breakages caused by a copy operation 'disappearing' or
having its signature change when a handle is added to the type.

### Test utilities

It is OPTIONAL for bindings to generate additional code specifically to be used
during testing. For example, the bindings can generate stub implementations of
each protocol so that users only need too verride specific methods that are
going to be exercised in a test.

### Epitaphs

Bindings SHOULD provide support for epitaphs, i.e. generated code that allows
servers to send epitaphs and clients to receive and handle epitaphs.

### Setters and Getters

Bindings MAY provide setters and getters for fields on aggregate types (structs,
unions, and tables). Even in languages where getter/setter methods are
un-idiomatic, using these methods will allow renaming internal field names
without breaking usages of that field.

### Request "responders"

When implementing a FIDL protocol using the FIDL bindings in a target language,
the bindings provide an API to read the request parameters, and a way to write
the response parameters, if any. For example, the request parameters could be
provided as the arguments to a function, and the response parameters could be
provided as the return type of the function.

For a FIDL protocol:

```fidl
{% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="examples/fidl/fuchsia.examples.docs/misc.test.fidl" region_tag="hasher" %}
```

A binding might generate:

```typescript
// Users would implement this interface to provide an implementation of the
// Hasher FIDL protocol
interface Hasher {
  // Respond to the request by returning the desired response
  hash: (value: string): Uint8Array;
};
```

Bindings MAY provide a responder object that is used to write
responses. In the example above, this would mean passing an additional
responder object in the function arguments, and having the function return void:

```typescript
interface Hasher {
  hash: (value: string, responder: HashResponder): void;
};

interface HashResponder {
  sendResponse(value: Uint8Array);
};
```

The use of a responder object has the following benefits:

* Improved ergonomics: responders can be used to provide any type of
  interaction with the client. For example, responders can have methods that
  close the channel with an epitaph, or provide APIs for sending events. For
  two-way methods, the responder could provide the mechanism to send a response.
* Increased flexibility: encapsulating all these behaviors in a single type
  makes it possible to add or remove behavior from the bindings without
  making breaking changes to bindings users, by only changing the responder
  object, and not the protocol object.

When providing a responder object, bindings should be careful about responders
being invoked on a different thread than the one the request was processed on.
Responders may also be invoked much later than the request was processed, for
instance when implement a handing get pattern. In practice this could be
implemented by allowing users to move ownership of the responder out of the
request handler class, e.g. into a callback for an asynchronous function.

The object MAY NOT necessarily be called responder. For example, it could have
a different name depending on whether the method is fire and forget or two way:

```typescript
interface Hasher {
  // the Hash method is a two-way method, so the object is called a responder
  hash: (value: string, responder: HashResponder): void;
  // the SetSeed method is a fire and forget method, so it gets a different name
  setSeed: (seed: number, control: HasherControlHandle): void;
}
```

### Testing

#### GIDL Conformance Testing

GIDL is a test definition language and tool that is used in conjunction with
FIDL to define conformance tests. These tests are standardized across bindings
and ensure consistency in implementation of encoders and decoders and coverage
of corner cases.

#### Deep equality of decoded objects {#deep-equality}

Comparing object equality can be tricky and particularly so in the case of
decoded FIDL objects. During decode, `zx_handle_replace` may be called on
handles if they have more handle rights than needed. When this happens, the
initial input handle will be closed and a new handle will be created to
replace it with reduced rights.

Because of this, handle values cannot be directly compared. Instead,
handles can be compared by checking that their koid (kernel id), type and
ensuring their rights are the same.

## Related Documents

* [RFC-0024: Mandatory Source Compatibility][rfc0024]

<!-- xrefs -->
[jsonir]: /docs/reference/fidl/language/json-ir.md
[rfc0024]: /docs/contribute/governance/rfcs/0024_mandatory_source_compatibility.md
[rfc0040]: /docs/contribute/governance/rfcs/0040_identifier_uniqueness.md
[rfc0057]: /docs/contribute/governance/rfcs/0057_default_no_handles.md
[RFC2119]: https://tools.ietf.org/html/rfc2119
[go-generated-code-comment]: https://github.com/golang/go/issues/13560#issuecomment-288457920
[attributes]: /docs/reference/fidl/language/attributes.md
[Life of a handle]: /docs/concepts/fidl/life-of-a-handle.md
[llcpp]: /docs/reference/fidl/bindings/cpp-bindings.md
[source-compatible]: /docs/development/languages/fidl/guides/compatibility/README.md#strict-flexible
[soft-transitions]: /docs/contribute/governance/rfcs/0002_platform_versioning.md#terminology
[strict-event]: /docs/contribute/governance/rfcs/0138_handling_unknown_interactions.md#changes-to-bindings
[unknown-attr]: /docs/reference/fidl/language/attributes.md#unknown