Peridot

API Guide

This document describes how to use Ledger for synchronized application storage in a client application. The goal of the document is to explain the key concepts and the most common operations, not necessarily to cover the entire API surface.

Connecting to Ledger

The Ledger API is exposed through a FIDL interface.

Access to Ledger is vended from the fuchsia::modular::ComponentContext of the running application. Each application component has a separate private data store for each user on behalf of which it runs.

C++

module_context->GetComponentContext(
    component_context_.NewRequest());
fuchsia::ledger::LedgerPtr ledger;
ledger.set_error_handler([](zx_status_t status) {
  // Handle ledger and connection errors.
});
component_context_->GetLedger(ledger.NewRequest());

Dart

moduleContext.getComponentContext(
    componentContext.ctrl.request());
final LedgerProxy ledger = new LedgerProxy();
ledger.ctrl.error.then((ProxyError error) {
  // Handle ledger and connection errors.
});
componentContext.getLedger(ledger.ctrl.request());
Note that the Ledger connection is provided through a FIDL interface request, so it can be immediately used to make requests.

Working with pages

Data stored in Ledger is grouped into separate independent key-value stores called pages.

Pages are identified by ids of 16 bytes each. When creating a page, the application can pick its id (for example, to create/access a page with a name that is fixed and known in advance), or can request a new unique id to be generated by Ledger (for example, to create pages that correspond 1-1 to business logic objects). Ledger also offers a convenience method GetRootPage() which retrieves the page with id [0, 0 ... , 0].

C++

fuchsia::ledger::PagePtr page;
ledger->GetRootPage(page.NewRequest(),
    [] (fuchsia::ledger::Status status) {
  if (status != fuchsia::ledger::Status::OK) {
    // Handle errors.
  }
});
page.set_error_handler([] (zx_status_t status) {
  // Handle connection errors.
});

Dart

final PageProxy page = new PageProxy();
_ledger.getRootPage(
    page.ctrl.request(), (Status status) {  
  if (status != Status.ok) {
    // Handle errors.
  }
});
page.ctrl.error.then((ProxyError error) {
  // Handle connection errors.
}

Page API

Each page exposes a key-value store API.

The state of each page changes in commits which form a DAG. This DAG is not directly exposed to the developer through the Ledger API, but we mention it as it helps to understand how the page operations described below behave.

History of a page can and does diverge when the page is modified concurrently on multiple devices (or even locally on separate connections to the same page). However, it's important to note that any given local page connection (/page proxy) always tracks one path through the commit history and only advances forward.

Writing

Values can be written/modified using mutation methods of the Page interface.

There are two ways to write a value into a page:

  • directly, passing the value in a FIDL message (that's Put() and PutWithPriority() methods). This is handy, but note that the developer needs to ensure that the value does not exceed the maximum payload size of a FIDL message.
  • in two steps: first streaming the data to be written to a “reference”, and then setting the value using the reference. (through a combination of CreateReferenceFromSocket(), CreateReferenceFromBuffer() and PutReference(). Note that the created reference is temporary and not persisted through app restarts - it must be set through PutReference(), or discarded.

Reading

In order to ensure that reads across multiple entries (key-value pairs) are consistent, read operations are exposed through a snapshot API. Snapshot of the page can be obtained:

  • directly from the page interface (GetSnapshot()), yielding a snapshot corresponding to the state of the page tracked by this page connection
  • through the watcher interface, which when delivering each notification allows to also request the snapshot corresponding to the state associated with the notification

Once the snapshot is obtained, data can be read using the GetEntries() method which retrieves multiple entries, or the Get() method which retrieves the value associated with the particular key.

Range queries

The GetEntries() method takes a key_start argument, allowing the app to perform a range query. In order to retrieve all entries between two keys, we need to call GetEntries() with the first key passed as key_start, and continue reading the paginated response as long as the returned values are in the desired range.

Lazy values

PutWithPriority() and PutReference() methods allow the app to write a value as a lazy value. Compared with the default eager values, lazy values are not guaranteed to be proactively fetched to the local device when the Page syncs to the state that contains them. The keys of the lazy values are always synced eagerly, but the value itself might be missing.

To ensure predictable performance of read operations, Get() and GetEntries methods never retrieve the missing lazy values from the network - these must be explicitly requested through a dedicated Fetch() method.

Watch

The client application can register a watcher to be notified of changes to the state tracked by the local page connection. As typically we are interested in retrieving the initial base state at the moment of registering the watcher, the watchers are registered using the GetSnapshot() method.

C++ watcher example, Dart watcher example.

GetSnapshot() takes an optional key_prefix parameter, which allows the client app to register specifically for change notifications within a particular prefix of keys.

Transactions

Transactions allow the application to make a set of changes that are guaranteed to be synced and surfaced atomically. A client application would typically:

  • start a transaction
  • get a snapshot of the page state through GetSnapshot() or a watcher (see below), and read the data
  • make changes through the Page interface
  • commit the transaction

Once a transaction is started, the state of the page tracked on the page connection is pinned and won't advance until the transaction is either committed or aborted - this ensures that the transaction writes affect precisely the state visible on the page snapshot.

There can be only one transaction in progress per Page connection, but clients can create multiple connections to the same page. Each of these connections can run a transaction independently from the others.

Watch and transactions: As starting a transaction pins the state of the page visible on the particular page connection, the client app won't receive any watch notifications (for watchers registered on this page connection) while the transaction is in progress.

Conversely, on a page connection with registered watchers, a StartTransaction() call will only return when the app finishes processing all pending change notification. This ensures the app knows the base state of the page when performing a transaction.

Conflict resolution

If the page history has diverged and has more than one head, conflict resolution will be invoked to merge the heads.

Note that this condition is not associated with any particular local page connection - there could be multiple local page connections on different divergent heads. Conflict resolution is therefore configured through the Ledger connection.

By default, a key-by-key last-one-wins policy is used to merge divergent commits, based on best-effort, not-guaranteed-to-be-right, timestamps of each commit.

The client might opt for a different merge policy by implementing the ConflictResolverFactory interface and setting it through a Ledger connection.

Available conflict resolution policies:

  • last-one-wins (the default): LAST_ONE_WINS

    For each entry modified on both sides, LAST_ONE_WINS takes the entry from the most recent commit; for each entry modified only on one side (e.g.: when adding a new key, or when modifying an existing one), the modification is carried over. The most recent commit is determined using timestamps assigned to the commits by the devices that created them. Note, however that these timestamps are not guaranteed to be accurate.

  • custom, invoked only for conflicting (modified on both branches being merged) entries: AUTOMATIC_WITH_FALLBACK

    For all modified keys, if either they are modified only on one side, or changed in the same way (added/updated with the same value, or deleted) in both sides, AUTOMATIC_WITH_FALLBACK applies the change automatically. If, however any key is modified on both sides with a non identical change, the app gets a callback with all changed entries from both sides (not just the conflicting ones).

  • custom, invoked for all merges (even if the entries modified by each branch are non-conflicting): CUSTOM

    Conflicting changes are ignored and there are no merges performed.

See also Conflict Resolution.

Synchronization

Synchronization is configured at the Ledger level as described in the user guide. Once configured, the Ledger will upload local changes, and download changes made by other devices. These operations will happen automatically in the background, and clients do not have to manage them.

See also