docs/development/languages/fidl/tutorial/README.md

# FIDL tutorial

_Audience: Beginning to intermediate FIDL developers._

[TOC]

## About this tutorial

In this tutorial, we present a language-independent introduction to FIDL,
followed by language-specific sections that develop the client
and server sides of the examples.
At the end of each tutorial, we present additional language-specific reference
material.

# Overview

While "**FIDL**" stands for "Fuchsia Interface Definition Language,"
the word itself is commonly used to refer to a lot of different things:

*   the infrastructure required for the IPC calls,
*   the FIDL language,
*   the FIDL compiler, or
*   the generated bindings.

This tutorial describes the steps required to create protocols, make calls, and
write server code that uses the FIDL interprocess communication (IPC) system in Fuchsia.
We'll also see an overview of the entire workflow required, for both service writers
and users (clients).

> Note that the Fuchsia operating system has no innate knowledge of FIDL.
> The FIDL bindings use a standard channel communication mechanism in Fuchsia.
> The FIDL bindings and libraries enforce a set of semantic behavior and
> persistence formats on how that channel is used.

For details on the design and implementation of FIDL, see
the [main FIDL page](../README.md)

## Example source code

Most examples used in this tutorial are located in Garnet at:
[//garnet/examples/fidl/](/garnet/examples/fidl/)

Language-specific examples may be found in other directories;
this is indicated as appropriate.

## FIDL in a nutshell

The main job of FIDL is to allow diverse clients and services to interoperate.
Client diversity is aided by decoupling the implementation of the
IPC mechanism from its definition, and is simplified by automatic
code generation.

The FIDL language provides a familiar (though simplified) C-like declaration
syntax that allows the service provider to exactly define their protocols.
Basic data types, like integers, floats, and strings,
can be organized into more complex aggregate structures and unions.
Fixed arrays and dynamically sized vectors can be constructed from both
the basic types and the aggregate types, and these can all be combined
into even more complex data structures.

Due to the number of client implementation target languages (C,
C++, Rust, Dart, and so on), we don't want to burden the developer of
the service with providing a protocol implementation for each and every one.

This is where the FIDL toolchain comes in.
The developer of the service creates just one `.fidl` definition file,
which defines the protocol.
Using this file, the FIDL compiler then generates client and server code
in any of the supported target languages.

![Figure: server written in C++ talks to clients written in multiple
languages](fidl_architecture.png)

In many cases, there will only be one implementation of the server (for example, the
particular service might be implemented in C++), whereas there could be any number
of implementations of the client, in a multitude of languages.

# FIDL architecture

From a developer's point of view, the following are the main components:

*   FIDL definition file — this is a text file (ending in `.fidl` by convention)
    that defines the values, and protocols (methods with their parameters),
*   client code — generated by the FIDL compiler (`fidlc`) toolchain for each specific
    target language, and
*   server code — also generated by the FIDL compiler toolchain.

We'll look at the client and server code generically in this chapter, and then
in language specific detail in following chapters.

## FIDL definition

The FIDL definition file defines the protocol.
As a very simple example, consider an "echo" service — whatever the client sends
to the server, the server just echoes back to the client.

> Line numbers have been added for clarity and are not part of the `.fidl` file.

```fidl
1   library fidl.examples.echo;
2
3   [Discoverable]
4   protocol Echo {
5       EchoString(string? value) -> (string? response);
6   };
```

Let's go through it line by line.

**Line 1:** The `library` keyword is used to define a namespace for this protocol.
FIDL protocols in different libraries might have the same name, so the namespace
is used to distinguish amongst them.

**Line 3:** The `[Discoverable]` [**attribute**][attributes] indicates that
the protocol that follows should be made available for clients to connect to.

**Line 4:** The `protocol` keyword introduces the name of the protocol, here it's called `Echo`.

**Line 5:** The method, its parameters, and return values.
There are two unusual aspects of this line:

*   Note the declaration `string?` (for both `value` and `response`).
    The `string` part indicates that the parameters are strings (sequences of
    characters), while the question mark indicates that the parameter is optional.
*   The `->` part indicates the return, which appears after the method declaration, not before.
    Unlike C++ or Java, a method can return multiple values.

The above FIDL file, then, has declared one protocol, called `Echo`, with one
method, called `EchoString`, that takes a nullable string and returns a nullable string.

## Aggregate data types

The simple example above used just one data type, the `string` as both the input
to the method as well as the output.

The data types are very flexible:

```fidl
struct MyRequest {
    uint32 serial;
    string key;
    vector<uint32> options;
};
```

The above declares a structure called `MyRequest` with three members: an unsigned
32-bit integer called `serial`, a string called `key`, and a vector of unsigned 32-bit
integers called `options`.

## Messaging

In order to understand FIDL's messaging, we need to break things up into two layers,
and clarify some definitions.

At the bottom (the operating system layer), there's an asynchronous communications
scheme geared towards independent progress of a **sender** and a **receiver**:

*   **sender** &mdash; the party that originates a message,
*   **receiver** &mdash; the party that receives a message,

Sending a message is a non-blocking operation: the sender sends the message,
and is then free to continue processing, regardless of what the receiver is doing.

A receiver can, if it wants to, block in order to wait for a message.

The top layer implements FIDL messages, and uses the bottom (asynchronous) layer.
It deals with **client**s and **server**s:

*   **client** &mdash; the party that is making a request (of a server),
*   **server** &mdash; the party that is processing a request (on behalf of a client).

> The terms "sender" and "receiver" make sense when we're discussing the messages
> themselves &mdash; the underlying communications scheme isn't concerned about
> the roles that we've assigned to the parties, just that one is sending and one
> is receiving.
>
> The terms "client" and "server" make sense when we're discussing the roles that
> the parties play.
> In particular, a client can be a sender at one time, and a receiver at a
> different time; same for the server.

Practically speaking, in the context of a client / server interaction,
that means that there are several models:

1.  **blocking call** &mdash; client sends to server, waits for reply
2.  **fire and forget** &mdash; client sends to server, doesn't expect reply
3.  **callback** or **async call** &mdash; client sends to server, but doesn't block;
    a reply is delivered asynchronously some time later
4.  **event** &mdash; server sends to client, without the client having asked for data

The first is synchronous, the rest are asynchronous.
We'll discuss these in order.

### Client sends to server, waits for a reply

This model is the traditional "blocking call" or "function call" available in most
programming languages, except that the invocation is done over a channel, and thus
can fail due to transport level errors.

From the point of view of the client, it consists of a call
that blocks, while the server performs some processing.

![Figure: client and server](blocking.png)

Here's a step-by-step description:

1.  A client makes a call (optionally containing data) and blocks.
2.  The server receives the client's call (and optional data), and performs
    some amount of processing.
3.  At the server's discretion, it replies to the client (with optional data).
4.  The server's reply causes the client to unblock.

To implement this synchronous messaging model over an asynchronous messaging
scheme is simple.
Recall that both the client-to-server and server-to-client message transfers
are, at the bottom layer in the protocol, asynchronous.
The synchronization happens at the client end, by having the client block until
the server's message arrives.

Basically, in this model, the client and server have come to an agreement:

*   data flow is initiated by the client,
*   the client shall have at most only one message outstanding,
*   the server shall send a message to the client only in response to a client's message
*   the client shall wait for the server's response before continuing.

This blocking model is commonly used where the client needs to get the reply to its
current request before it can continue.

For example, the client may request data from the server, and not be able to do any
other useful processing until that data arrives.

Or, the client may need to perform steps in a specific order, and must therefore
ensure that each step completes before initiating the next one.
If an error occurs, the client may need to perform corrective actions that depend
on how far the operation has proceeded &mdash; another reason to be synchronized
to the completion of each step.

### Client sends to server, no reply

This model is also known as "fire and forget."
In it, the client sends the message to the server. and then carries on
with its operation.
In contrast to the blocking model, the client *does not* block,
*nor does it expect a response*.

This model is used in cases where the client doesn't need to (or cannot)
synchronize to the processing of its request.

![Figure: Fire and Forget; client sends to server but doesn't
expect replies](faf.png)

The classic example is a logging system.
The client sends logging information to the logging server (circles "1"
and "2" in the diagram above), but has no reason to block.
A lot of things can go wrong at the server end:

1. the server is busy and can't handle the write request at this moment,
2. the media is full and the server can't write the data,
3. the server has encountered a fault,
4. and so on.

However, the client isn't in a position to do anything about those problems,
so blocking would just create more problems.

### Client sends to server, but doesn't block

This model, and the next one ("server sends to client, without client asking
for data") are similar.

In the present model, the client sends a message to a server, but doesn't block.
However, the client expects some kind of response from the server, but the
key here is that it's not *synchronous* with the request.

This allows great flexibility in the client / server interaction.

While the synchronous model forces the client to wait until the server replies,
the present model frees the client to do something else while the server is
processing the request:

![Figure: client sends to server but doesn't block until later](async.png)

The subtle difference in this diagram vs. the similar one above is that after
circle "1" the client is *still running*.
The client chooses when to give up CPU; it's not synchronous with the message.

There are actually two sub-cases here &mdash; one in which the client
gets just one response, and another in which the client can get multiple responses.
(The one where the client gets zero responses is the "fire and forget" model,
which we discussed earlier.)

#### Single request, single response

The single response case is the closest to the synchronous model: the client sends
a message, and eventually, the server replies.
You'd use this model instead of multi-threading, for example, when you know that
the client could be doing useful work while waiting for the server's reply.

#### Single request, multiple response

The multiple response case can be used in a "subscription" model.
The client's message "primes" the server, for example, requesting notification
whenever something happens.

The client then goes about its business.

Some time later, the server notices that the condition that the client is interested
in has happened, and thus sends the client a message.
From a client / server point of view, this message is a "reply", with the client
receiving it asynchronously to its request.

![Figure: client sends to server, server replies multiple times](async-multiple.png)

There's no reason why the server couldn't send another message when another event
of interest occurs; this is the "multiple response" version of the model.
Note that the second (and subsequent) responses are sent *without* the client
sending any additional messages.

> Note that the client doesn't *need* to wait for the server to send it a message.
> In the diagram above, we showed the client in the blocked state before circle
> "3" &mdash; the client could just as well have been running.

### Server sends to client, without client asking for data

This model is also known as the "event" model.

![Figure: unsolicited messages from a server to a client](event.png)

In it, a client prepares to receive messages from a server, but doesn't know
when to expect them &mdash; the messages are not only asynchronous to the client,
but are also (from a client / server point of view) "unsolicited", in that the
client didn't explicitly request them (like it did in the previous model, above).

The client designates a function (the "event handling function") to be called when
messages arrive from the server, but otherwise continues about its business.

At the server's discretion (circles "1" and "2" in the diagram above), messages
are sent asynchronously to the client, and handled by the client's designated
function.

Note that the client may already be running when a message is sent (as in
circle "1"), or the client may have nothing to do and be waiting for a
message to be sent (as in circle "2").

> It is not a requirement that the client be waiting for a message.

### Asynchronous messaging complexity

Breaking up asynchronous messaging into the above (somewhat arbitrary) categories
is meant to show typical usage patterns, but isn't meant to be exhaustive.

In the most general case of asynchronous messaging, you have zero or more client messages
loosely associated with zero or more server replies.
It's this "loose association" that adds the complexity in terms of your design
process.

## IPC models in FIDL

Now that we have an understanding of the IPC models and how they
interact with FIDL's asynchronous messaging, let's see how they're defined.

We'll add the other models (fire and forget, and async call) to the
protocol definition file:


```fidl
1   library fidl.examples.echo;
2
3   [Discoverable]
4   protocol Echo {
5       EchoString(string? value) -> (string? response);
6       SendString(string? value);
7       -> ReceiveString (string? response);
8   };
```

**Line 5** is the `EchoString` method that we discussed above &mdash; it's
a traditional function call message, where the client calls `EchoString` with
an optional string, and then blocks, waiting for the server to reply with
another optional string.

**Line 6** is the `SendString` method.
It does not have the `->` return declaration &mdash; that makes it into a
"fire and forget" model (send only), because we've told the FIDL compiler that
this particular method does not have a return associated with it.

> Note that it's not the lack of return **parameters**, but rather the lack of
> return **declaration** that's the key here &mdash; putting "`-> ()`"
> after `SendString` would change the meaning from declaring a fire-and-forget
> style method to declaring a function call style method that doesn't have any
> return arguments.

**Line 7** is the `ReceiveString` method.
It's a little different &mdash; it doesn't have the method name in the first
part, but rather it's given after the `->` operator.
This tells the FIDL compiler that this is an "async call" model declaration.

# Client implementation

Regardless of the target language, the `fidlc` FIDL compiler generates client
code that has the following basic structure.

The first part consists of the administration and background handling, and
consists of:

1. some means of connecting to the server is provided
2. an asynchronous ("background") message handling loop is started
3. async call style and event style methods, if any, are bound to the message
   loop

The second part consists of implementations of the traditional function call or
fire and forget style methods, as appropriate for the target language.
Generally speaking, this consists of:

1. creating a callable API and declarations
2. generating code for each API that marshals the data from the call into a FIDL
   formatted buffer suitable for transmission to the server
3. generating code to transmit the data to the server
4. in the case of function call style calls, generating code to:
    1. wait for the response from the server
    2. unmarshal the data from the FIDL formatted buffer, and
    3. return the data via the API function.

Obviously, the exact steps may vary due to language implementation differences,
but that's the basic outline.

# Server implementation

The `fidlc` FIDL compiler can also generate server code for a given target
language. Just like the client code, this code has a common structure regardless
of the target language. The code:

1. creates an object which clients can connect to,
2. starts a main processing loop, which:
    1. waits for messages
    2. processes messages by calling out to the implementation functions
    3. if specified, issues an asynchronous call back to the client to return
       the output

In the next chapters, we'll see the details of each language's implementation of
the client and server code.

# Languages

Currently, tutorials are available in the following languages:

*   [C](tutorial-c.md)
*   [Low-Level C++](tutorial-llcpp.md)
*   [High-Level C++](tutorial-cpp.md)
*   [Dart](tutorial-dart.md)
*   [Rust](tutorial-rust.md)

Consult the [C Family Comparison](c-family-comparison.md) document for an
overview of the similarities and differences between the C, Low-Level C++,
and High-Level C++ bindings.

<!-- xrefs -->
[attributes]: ../reference/attributes.md