Cobalt: Telemetry with built-in privacy

Overview

Cobalt is a pipeline for collecting metrics data from user-owned devices in the field and producing aggregated reports.

Cobalt includes a suite of features for preserving user privacy and anonymity while giving product owners the data they need to improve their products.

• Differential Privacy. If you enable it, Cobalt can add random noise on the client during metrics collection.
• Shuffling. Our pipeline shuffles the collected records to break linkability. This also amplifies the level of differential privacy.
• No IDs. Cobalt does not use device IDs or user IDs; not even pseudonymous IDs.
• No exact timestamps. Cobalt uses day indices only.
• Only standard metric types. Not arbitrary protocol buffers.
• No ad-hoc, post-facto querying. Whereas on other telemetry systems standard practice is to “collect now; ask questions later”, with Cobalt the analyst must think in advance about what aggregations they wish to report and predeclare reports.
• Only aggregated data is released. It is not possible to query the raw collected data.

Cobalt Core

The code in the Git repo at https://fuchsia.googlesource.com/cobalt is known as Cobalt Core. It contains:

• Client-side libraries for:
  • Logging metric events to Cobalt
  • Local aggregation
  • Privacy encoding
  • Local data storage
  • Upload orchestration
  • Protocol buffers for the registry of Cobalt's projects, metrics and reports.
  • Protocol buffers for data transport
  • The registry parser

In order to have a working Cobalt system you need to also have

• A server-side implementation of the Cobalt pipeline including a Shuffler and Analyzers (aka Report-Generators.)
  • There is a server-side pipeline running at Google with code in the internal Google repository.
  • As of this writing that is the only server-side implementation of Cobalt.
• An embedding of Cobalt Core into a client-side platform.
  • This allows the code-to-be-measured to easily invoke Cobalt's Logger interface.
  • There is an embedding into the Fuchsia operating system at //src/cobalt.
  • If you have a Fuchsia checkout, you will find a copy of Cobalt Core has been pulled in at //third_party/cobalt.
  • As of this writing, Fuchsia is the only embedding of Cobalt Core.

This document describes how to develop, build and test Cobalt Core independently of Fuchsia, as well as from in the Fuchsia tree.

Requirements

1. We have only tested the stand-alone Cobalt Core build on Linux. You may or may not have success on other systems.
2. Python version at least 2.7
3. Git

Fetch the code

To download just the Cobalt Core repo:

• git clone https://fuchsia.googlesource.com/cobalt
• cd cobalt

To download the entire Fuchsia source code, including Cobalt Core, follow the instructions. Then change into the Cobalt Core directory:

• cd third_party/cobalt

Run setup

./cobaltb.py setup

This command is required after a first checkout, before running any other cobaltb.py commands, and will take a few minutes the first time. It does the following:

• Fetches third party code into Cobalt's third_party dir via Git submodules.
• Fetches the sysroot and puts it in Cobalt's sysroot dir. This uses CIPD.

In the Fuchsia Tree

If the ./cobaltb.py setup command is run in the third_party/cobalt checkout of Cobalt Core in a Fuchsia tree, the Git submodules may interfere with the correct functioning of Jiri to update the repos in the Fuchsia tree. If problems occur, the submodules can be removed with: ./cobaltb.py deinit. Before running any other cobaltb.py commands in this checkout, ./cobaltb.py setup will need to be run again to refetch the submodules.

cobaltb.py

The Python script cobaltb.py in the root directory is used to orchestrate building and testing Cobalt. It was already used above in ./cobaltb.py setup.

• ./cobaltb.py -h for general help
• cobaltb.py <command> -h for help on a command

Build

./cobaltb.py build

The Cobalt build uses GN and ninja.

Test

./cobaltb.py test

This runs the tests from the currently built code (build must be run first).

Formatting and Linting

./cobaltb.py fmt

./cobaltb.py lint

See: clang-tidy for an explanation of clang-tidy configuration options.

Rolling Submodules

./cobaltb.py sync_with_fuchsia --make_commit

Cobalt Core‘s standalone build relies on git submodules to load several other git repositories, including the Cobalt Registry. These submodules are not rolled automatically, so developers need to occasionally manually roll them to keep the libraries and the Cobalt Registry up to date. The command above will update all Cobalt Core submodules to the latest versions used in Fuchsia’s integration repo, syncs the prebuilts into cobalt.ensure, copies over some required files from the fuchsia repo, then creates a commit with a description of which submodules were updated.

Building Cobalt in Fuchsia

If the Cobalt Core checkout is part of a Fuchsia tree, it can also be built using Fuchsia's tools. This can be done to test the Cobalt code changes build and function correctly in Fuchsia. However, the build/test/fmt/lint commands above should always be used before contributing a change to Cobalt Core.

• Build all of Fuchsia (assumes the Fuchsia build was already configured, see instructions): fx build

In this way, the Cobalt Core and Fuchsia build systems can function correctly in parallel.

Contributing

Cobalt uses the Gerrit code review tool. Although Cobalt Core is independent of Fuchsia, we do share infrastructure with Fucshia and so much of the instructions from Fuchsia's Contributing Guide apply to Cobalt Core. The main difference is that Cobalt Core does not use Jiri--it uses Git submodules instead.

Generate a cookie

Follow the instructions here.

Create a change in Gerrit

To create a change in Gerrit do the following:

• Go to your Cobalt Core directory, for example: cd ~/cobalt
• Create a new branch:

git checkout -b <branch_name>

• Create or edit files in the new branch.
• Add the updated files: git add <files>
• Commit the updated files and write a change message. git commit
• Upload the commit to Gerrit: git push origin HEAD:refs/for/main

See the Gerrit documentation for more information.

Update a change in Gerrit

After creating a change, to upload a patch to your change, do the following:

• Create or edit files in the same branch.
• Add the updated files: git add <files>
• Include the patch in the same commit using the --amend option: git commit --amend
• Upload the patch to Gerrit: git push origin HEAD:refs/for/main