FIDL tutorial

Audience: Beginning to intermediate FIDL developers.

About this tutorial

In this tutorial, we present a language-independent introduction to FIDL, followed by language-specific sections that develop the client and server sides of the examples.

Overview

While “FIDL” stands for “Fuchsia Interface Definition Language,” the word itself is commonly used to refer to a lot of different things:

  • the infrastructure required for the IPC calls,
  • the FIDL language,
  • the FIDL compiler, or
  • the generated bindings.

This tutorial describes the steps required to create protocols, make calls, and write server code that uses the FIDL interprocess communication (IPC) system in Fuchsia. We'll also see an overview of the entire workflow required, for both service writers and users (clients).

Note that the Fuchsia operating system has no innate knowledge of FIDL. The FIDL bindings use a standard channel communication mechanism in Fuchsia. The FIDL bindings and libraries enforce a set of semantic behavior and persistence formats on how that channel is used.

For details on the design and implementation of FIDL, see the main FIDL page

Example source code

Most examples used in this tutorial are located in Garnet at: //garnet/examples/fidl/

Language-specific examples may be found in other directories; this is indicated as appropriate.

FIDL in a nutshell

The main job of FIDL is to allow diverse clients and services to interoperate. Client diversity is aided by decoupling the implementation of the IPC mechanism from its definition, and is simplified by automatic code generation.

The FIDL language provides a familiar (though simplified) C-like declaration syntax that allows the service provider to exactly define their protocols. Basic data types, like integers, floats, and strings, can be organized into more complex aggregate structures and unions. Fixed arrays and dynamically sized vectors can be constructed from both the basic types and the aggregate types, and these can all be combined into even more complex data structures.

Due to the number of client implementations target languages (C, C++, Rust, Dart, and so on), we don't want to burden the developer of the service with providing a protocol implementation for each and every one.

This is where the FIDL toolchain comes in. The developer of the service creates just one .fidl definition file, which defines the protocol. Using this file, the FIDL compiler then generates client and server code in any of the supported target languages.

In many cases, there will only be one implementation of the server (for example, the particular service might be implemented in C++), whereas there could be any number of implementations of the client, in a multitude of languages.

FIDL architecture

From a developer's point of view, the following are the main components:

  • FIDL definition file — this is a text file (ending in .fidl by convention) that defines the values, and protocols (methods with their parameters),
  • client code — generated by the FIDL compiler (fidlc) toolchain for each specific target language, and
  • server code — also generated by the FIDL compiler toolchain.

We'll look at the client and server code generically in this chapter, and then in language specific detail in following chapters.

FIDL definition

The FIDL definition file defines the protocol. As a very simple example, consider an “echo” service — whatever the client sends to the server, the server just echoes back to the client.

Line numbers have been added for clarity and are not part of the .fidl file.

1   library fidl.examples.echo;
2
3   [Discoverable]
4   protocol Echo {
5       EchoString(string? value) -> (string? response);
6   };

Let's go through it line by line.

Line 1: The library keyword is used to define a namespace for this protocol. FIDL protocols in different libraries might have the same name, so the namespace is used to distinguish amongst them.

Line 3: The [Discoverable] attribute indicates that the protocol that follows should be made available for clients to connect to.

Line 4: The protocol keyword introduces the name of the protocol, here it's called Echo.

Line 5: The method, its parameters, and return values. There are two unusual aspects of this line:

  • Note the declaration string? (for both value and response). The string part indicates that the parameters are strings (sequences of characters), while the question mark indicates that the parameter is optional.
  • The -> part indicates the return, which appears after the method declaration, not before. Unlike C++ or Java, a method can return multiple values.

The above FIDL file, then, has declared one protocol, called Echo, with one method, called EchoString, that takes a nullable string and returns a nullable string.

Aggregate data types

The simple example above used just one data type, the string as both the input to the method as well as the output.

The data types are very flexible:

struct MyRequest {
    uint32 serial;
    string key;
    vector<uint32> options;
};

The above declares a structure called MyRequest with three members: an unsigned 32-bit integer called serial, a string called key, and a vector of unsigned 32-bit integers called options.

Messaging

In order to understand FIDL's messaging, we need to break things up into two layers, and clarify some definitions.

At the bottom (the operating system layer), there's an asynchronous communications scheme geared towards independent progress of a sender and a receiver:

  • sender — the party that originates a message,
  • receiver — the party that receives a message,

Sending a message is a non-blocking operation: the sender sends the message, and is then free to continue processing, regardless of what the receiver is doing.

A receiver can, if it wants to, block in order to wait for a message.

The top layer implements FIDL messages, and uses the bottom (asynchronous) layer. It deals with clients and servers:

  • client — the party that is making a request (of a server),
  • server — the party that is processing a request (on behalf of a client).

The terms “sender” and “receiver” make sense when we‘re discussing the messages themselves — the underlying communications scheme isn’t concerned about the roles that we've assigned to the parties, just that one is sending and one is receiving.

The terms “client” and “server” make sense when we're discussing the roles that the parties play. In particular, a client can be a sender at one time, and a receiver at a different time; same for the server.

Practically speaking, in the context of a client / server interaction, that means that there are several models:

  1. blocking call — client sends to server, waits for reply
  2. fire and forget — client sends to server, doesn't expect reply
  3. callback or async call — client sends to server, but doesn't block; a reply is delivered asynchronously some time later
  4. event — server sends to client, without the client having asked for data

The first is synchronous, the rest are asynchronous. We'll discuss these in order.

Client sends to server, waits for a reply

This model is the traditional “blocking call” or “function call” available in most programming languages, except that the invocation is done over a channel, and thus can fail due to transport level errors.

From the point of view of the client, it consists of a call that blocks, while the server performs some processing. Here's a step-by-step description:

  1. A client makes a call (optionally containing data) and blocks.
  2. The server receives the client's call (and optional data), and performs some amount of processing.
  3. At the server's discretion, it replies to the client (with optional data).
  4. The server's reply causes the client to unblock.

To implement this synchronous messaging model over an asynchronous messaging scheme is simple. Recall that both the client-to-server and server-to-client message transfers are, at the bottom layer in the protocol, asynchronous. The synchronization happens at the client end, by having the client block until the server's message arrives.

Basically, in this model, the client and server have come to an agreement:

  • data flow is initiated by the client,
  • the client shall have at most only one message outstanding,
  • the server shall send a message to the client only in response to a client's message
  • the client waits for the server's response before continuing.

This blocking model is commonly used where the client needs to get the reply to its current request before it can continue.

For example, the client may request data from the server, and not be able to do any other useful processing until that data arrives.

Or, the client may need to perform steps in a specific order, and must therefore ensure that each step completes before initiating the next one. If an error occurs, the client may need to perform corrective actions that depend on how far the operation has proceeded — another reason to be synchronized to the completion of each step.

Client sends to server, no reply

This model is also known as “fire and forget.” In it, the client sends the message to the server. and then carries on with its operation. In contrast to the blocking model, the client does not block, nor does it expect a response.

This model is used in cases where the client doesn't need to (or cannot) synchronize to the processing of its request.

The classic example is a logging system. The client sends logging information to the logging server, but has no reason to block. A lot of things can go wrong at the server end:

  1. the server is busy and can't handle the write request at this moment,
  2. the media is full and the server can't write the data,
  3. the server has encountered a fault,
  4. and so on.

However, the client isn't in a position to do anything about those problems, so blocking would just create more problems.

Client sends to server, but doesn't block

This model, and the next one (“server sends to client, without client asking for data”) are similar.

In the present model, the client sends a message to a server, but doesn‘t block. However, the client expects some kind of response from the server, but the key here is that it’s not synchronous with the request.

This allows great flexibility in the client / server interaction.

While the synchronous model forces the client to wait until the server replies, the present model frees the client to do something else while the server is processing the request.

There are actually two sub-cases here — one in which the client gets just one response, and another in which the client can get multiple responses. (The one where the client gets zero responses is the “fire and forget” model, which we discussed above.)

Single request, single response

The single response case is the closest to the synchronous model: the client sends a message, and eventually, the server replies. You‘d use this model instead of multi-threading, for example, when you know that the client could be doing useful work while waiting for the server’s reply.

Single request, multiple response

The multiple response case can be used in a “subscription” model. The client's message “primes” the server, for example, requesting notification whenever something happens.

The client then goes about its business.

Some time later, the server notices that the condition that the client is interested in has happened, and thus sends the client a message. From a client / server point of view, this message is a “reply”, with the client receiving it asynchronously to its request.

There‘s no reason why the server couldn’t send another message when another event of interest occurs; this is the “multiple response” version of the model. Note that the second (and subsequent) responses are sent without the client sending any additional messages.

Server sends to client, without client asking for data

This model is also known as the “event” model.

In it, a client prepares to receive messages from a server, but doesn‘t know when to expect them — the messages are not only asynchronous to the client, but are also (from a client / server point of view) “unsolicited”, in that the client didn’t explicitly request them (like it did in the previous model, above).

The client designates a function (the “event handling function”) to be called when messages arrive from the server, but otherwise continues about its business.

At the server‘s discretion, messages are sent asynchronously to the client, and handled by the client’s designated function.

Asynchronous messaging complexity

Breaking up asynchronous messaging into the above (somewhat arbitrary) categories is meant to show typical usage patterns, but isn't meant to be exhaustive.

In the most general case of asynchronous messaging, you have zero or more client messages loosely associated with zero or more server replies. It's this “loose association” that adds the complexity in terms of your design process.

IPC models in FIDL

Now that we have an understanding of the IPC models and how they interact with FIDL‘s asynchronous messaging, let’s see how they're defined.

We'll add the other models (fire and forget, and async call) to the protocol definition file:

1   library fidl.examples.echo;
2
3   [Discoverable]
4   protocol Echo {
5       EchoString(string? value) -> (string? response);
6       SendString(string? value);
7       -> ReceiveString (string? response);
8   };

Line 5 is the EchoString method that we discussed above — it's a traditional function call message, where the client calls EchoString with an optional string, and then blocks, waiting for the server to reply with another optional string.

Line 6 is the SendString method. It does not have the -> return declaration — that makes it into a “fire and forget” model (send only), because we've told the FIDL compiler that this particular method does not have a return associated with it.

Note that it‘s not the lack of “return parameters,” but rather the lack of “return declaration” that’s the key here — putting “-> ()” after SendString would change the meaning from declaring a fire-and-forget style method to declaring a function call style message that doesn't have any return arguments.

Line 7 is the ReceiveString method. It‘s a little different — it doesn’t have the method name in the first part, but rather it's given after the -> operator. This tells the FIDL compiler that this is an “async call” model declaration.

Client implementation

Regardless of the target language, the fidlc FIDL compiler generates client code that has the following basic structure.

The first part consists of the administration and background handling, and consists of:

  1. some means of connecting to the server is provided
  2. an asynchronous (“background”) message handling loop is started
  3. async call style and event style methods, if any, are bound to the message loop

The second part consists of implementations of the traditional function call or fire and forget style methods, as appropriate for the target language. Generally speaking, this consists of:

  1. creating a callable API and declarations
  2. generating code for each API that marshals the data from the call into a FIDL formatted buffer suitable for transmission to the server
  3. generating code to transmit the data to the server
  4. in the case of function call style calls, generating code to:
    1. wait for the response from the server
    2. unmarshal the data from the FIDL formatted buffer, and
    3. return the data via the API function.

Obviously, the exact steps may vary due to language implementation differences, but that's the basic outline.

Server implementation

The fidlc FIDL compiler can also generate server code for a given target language. Just like the client code, this code has a common structure regardless of the target language. The code:

  1. creates an object which clients can connect to,
  2. starts a main processing loop, which:
    1. waits for messages
    2. processes messages by calling out to the implementation functions
    3. if specified, issues an asynchronous call back to the client to return the output

In the next chapters, we‘ll see the details of each language’s implementation of the client and server code.

Languages

Currently, tutorials are available in the following languages: