peridot/docs/ledger/api_guide.md

# API Guide

This document describes how to use Ledger for synchronized application storage
in a client application. The goal of the document is to explain the key concepts
and the most common operations, not necessarily to cover the entire API surface.

[TOC]

## Connecting to Ledger

The Ledger API is exposed through a [FIDL interface].

Access to Ledger is vended from the `fuchsia::modular::ComponentContext` of the running
application. Each application component has a separate private data store for
each user on behalf of which it runs.

|||---|||
#### C++

``` cpp
module_context->GetComponentContext(
    component_context_.NewRequest());
fuchsia::ledger::LedgerPtr ledger;
ledger.set_error_handler([](zx_status_t status) {
  // Handle ledger and connection errors.
});
component_context_->GetLedger(ledger.NewRequest());
```

#### Dart

``` dart
moduleContext.getComponentContext(
    componentContext.ctrl.request());
final LedgerProxy ledger = new LedgerProxy();
ledger.ctrl.error.then((ProxyError error) {
  // Handle ledger and connection errors.
});
componentContext.getLedger(ledger.ctrl.request());
```
|||---|||

*** aside
Note that the Ledger connection is provided through a FIDL *interface request*,
so it can be immediately used to make requests.
***

## Working with pages

Data stored in Ledger is grouped into separate independent key-value stores
called *pages*.

Pages are identified by ids of 16 bytes each. When creating a page, the
application can pick its id (for example, to create/access a page with a name
that is fixed and known in advance), or can request a new unique id to be
generated by Ledger (for example, to create pages that correspond 1-1 to
business logic objects). Ledger also offers a convenience method `GetRootPage()`
which retrieves the page with id `[0, 0 ... , 0]`.

|||---|||
#### C++

``` cpp
fuchsia::ledger::PagePtr page;
ledger->GetRootPage(page.NewRequest(),
    [] (fuchsia::ledger::Status status) {
  if (status != fuchsia::ledger::Status::OK) {
    // Handle errors.
  }
});
page.set_error_handler([] (zx_status_t status) {
  // Handle connection errors.
});
```

#### Dart

``` dart
final PageProxy page = new PageProxy();
_ledger.getRootPage(
    page.ctrl.request(), (Status status) {  
  if (status != Status.ok) {
    // Handle errors.
  }
});
page.ctrl.error.then((ProxyError error) {
  // Handle connection errors.
}
```
|||---|||

## Page API

Each page exposes a key-value store API.

The state of each page changes in commits which form a DAG. This DAG is not
directly exposed to the developer through the Ledger API, but we mention it as
it helps to understand how the page operations described below behave.

*** promo
History of a page can and does diverge when the page is modified concurrently on
multiple devices (or even locally on separate connections to the same page).
However, it's important to note that any given local page connection (/page
proxy) always tracks **one path** through the commit history and only advances
forward.
***

### Writing

Values can be written/modified using mutation methods of the Page interface.

There are two ways to write a value into a page:

 - directly, passing the value in a FIDL message (that's `Put()` and
   `PutWithPriority()` methods). This is handy, but note that the developer
   needs to ensure that the value does not exceed the maximum payload size of a
   FIDL message.
 - in two steps: first streaming the data to be written to a "reference", and
   then setting the value using the reference. (through a combination of
   `CreateReferenceFromSocket()`, `CreateReferenceFromBuffer()` and
   `PutReference()`. Note that the created reference is temporary and not
   persisted through app restarts - it must be set through `PutReference()`, or
   discarded.

### Reading

In order to ensure that reads across multiple entries (key-value pairs) are
consistent, read operations are exposed through a snapshot API. Snapshot of the
page can be obtained:

 - directly from the page interface (`GetSnapshot()`), yielding a snapshot
   corresponding to the state of the page tracked by this page connection
 - through the [watcher](#Watch) interface, which when delivering each
   notification allows to also request the snapshot corresponding to the state
   associated with the notification

Once the snapshot is obtained, data can be read using the `GetEntries()` method
which retrieves multiple entries, or the `Get()` method which retrieves the
value associated with the particular key.

#### Range queries

The `GetEntries()` method takes a `key_start` argument, allowing the app to
perform a *range query*. In order to retrieve all entries between two keys, we
need to call `GetEntries()` with the first key passed as `key_start`, and
continue reading the paginated response as long as the returned values are in
the desired range.

### Lazy values

`PutWithPriority()` and `PutReference()` methods allow the app to write a value
as a *lazy value*. Compared with the default *eager values*, lazy values are not
guaranteed to be proactively fetched to the local device when the Page syncs to
the state that contains them. The **keys** of the lazy values are always synced
eagerly, but the value itself might be missing.

To ensure predictable performance of read operations, `Get()` and `GetEntries`
methods never retrieve the missing lazy values from the network - these must be
explicitly requested through a dedicated `Fetch()` method.

### Watch

The client application can register a watcher to be notified of changes to the
state tracked by the local page connection. As typically we are interested in
retrieving the initial base state at the moment of registering the watcher,
the watchers are registered using the `GetSnapshot()` method.

[C++ watcher example], [Dart watcher example].

`GetSnapshot()` takes an optional `key_prefix` parameter, which allows the
client app to register specifically for change notifications within a particular
prefix of keys.

### Transactions

Transactions allow the application to make a set of changes that are guaranteed
to be synced and surfaced atomically. A client application would typically:

 - start a transaction
 - get a snapshot of the page state through `GetSnapshot()` or a watcher (see
   below), and read the data
 - make changes through the Page interface
 - commit the transaction

Once a transaction is started, the state of the page tracked on the page
connection is pinned and won't advance until the transaction is either committed
or aborted - this ensures that the transaction writes affect precisely the state
visible on the page snapshot.

There can be only one transaction in progress per `Page` connection, but
clients can create multiple connections to the same page. Each of these
connections can run a transaction independently from the others.

*** note
*Watch and transactions*: As starting a transaction pins the state of the page
visible on the particular page connection, the client app won't receive any
watch notifications (for watchers registered on this page connection) while the
transaction is in progress.

Conversely, on a page connection with registered watchers, a
`StartTransaction()` call will only return when the app finishes processing all
pending change notification. This ensures the app knows the base state of the
page when performing a transaction.
***

## Conflict resolution

If the page history has diverged and has more than one head, conflict resolution
will be invoked to merge the heads.

*** aside
Note that this condition is not associated with any particular local page
connection - there could be multiple local page connections on different
divergent heads. Conflict resolution is therefore configured through the Ledger
connection.
***

By default, a key-by-key last-one-wins policy is used to merge divergent
commits, based on best-effort, not-guaranteed-to-be-right, timestamps of each
commit.

The client might opt for a different merge policy by implementing the
`ConflictResolverFactory` interface and setting it through a Ledger connection.

Available conflict resolution policies:

 - last-one-wins (the default): `LAST_ONE_WINS`

   For each entry modified on both sides, `LAST_ONE_WINS` takes the entry from
   the most recent commit; for each entry modified only on one side (e.g.:
   when adding a new key, or when modifying an existing one), the modification
   is carried over. The most recent commit is determined using timestamps
   assigned to the commits by the devices that created them. Note, however that
   these timestamps are not guaranteed to be accurate.
 - custom, invoked only for conflicting (modified on both branches being
   merged) entries: `AUTOMATIC_WITH_FALLBACK`

   For all modified keys, if either they are modified only on one side, or
   changed in the same way (added/updated with the same value, or deleted) in
   both sides, `AUTOMATIC_WITH_FALLBACK` applies the change automatically. If,
   however any key is modified on both sides with a non identical change, the
   app gets a callback with all changed entries from both sides (not just the
   conflicting ones).
 - custom, invoked for all merges (even if the entries modified by each branch
   are non-conflicting): `CUSTOM`

   Conflicting changes are ignored and there are no merges performed.


See also [Conflict Resolution](conflict_resolution.md).

## Synchronization

Synchronization is configured at the Ledger level as described in the [user
guide](user_guide.md). Once configured, the Ledger will upload local changes,
and download changes made by other devices. These operations will happen
automatically in the background, and clients do not have to manage them.

## See also

 - [Data Organization](data_organization.md)
 - [Examples of client apps](examples.md)

[FIDL interface]: /public/fidl/fuchsia.ledger/ledger.fidl
[C++ watcher example]: /examples/todo_cpp/todo.h
[Dart watcher example]: https://fuchsia.googlesource.com/topaz/+/master/examples/ledger/todo_list/lib/src/models/todo_list_model.dart