commit | 931267f3493da4b99df2da2a988bc79fe9f7fa76 | [log] [tgz] |
---|---|---|
author | Nina Ilieva <ninai@google.com> | Fri Dec 13 00:35:27 2019 +0000 |
committer | Nina Ilieva <ninai@google.com> | Fri Dec 13 14:25:51 2019 -0800 |
tree | c7ea72538e50c91d8e1997b3042047bf8d0b7688 | |
parent | 0350ef57fd4a0fa2c4b26291430e528bb8d05be6 [diff] |
Reland "[cobalt][LocalAggregation] Move EventAggregatorManager code to the" This reverts commit e42ecad159ac196027c3dce1cf0334ff90f3033c. Reason for revert: https://fuchsia-review.googlesource.com/c/fuchsia/+/348613 and https://fuchsia-review.googlesource.com/c/cobalt/+/348477 should fix the issue. Original change's description: > Revert "[cobalt][LocalAggregation] Move EventAggregatorManager code to the" > > This reverts commit 894be776ac687f5475467f02894cb595ea63e94b. > > Reason for revert: breaks roller > > Original change's description: > > [cobalt][LocalAggregation] Move EventAggregatorManager code to the > > appropriate files > > > > - removes thread management and memory management from the > > EventAggregator and moves it to the EventAggregatorManager > > - Small changes to tests to match this change > > - deletes GenerateObservationsNoWorker() > > - the AggregateStoreTest is minimally changed as it will be reworked in > > a following CL. > > > > Bug: 40853 > > Change-Id: Id409279bc92edbbe50089a0eaa32bc8e5d5230aa > > TBR=ninai@google.com,pesk@google.com > > Change-Id: Icd3aea684c20b1963466b258ae782abb52f28694 > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Bug: 40853 TBR=ninai@google.com,pesk@google.com Bug: 40853 Change-Id: I48ec04c39c4abe026e10f42f964a550402718f42
An analytics pipeline with built-in user-privacy.
You may encounter this document in one of two scenarios:
https://fuchsia.googlesource.com/cobalt
, or//third_party/cobalt.
This document should be used only in the first context. It describes how to build and test Cobalt independently of Fuchsia. Stand-alone Cobalt includes server-side components that run in Linux on Google Kubernetes Engine and a generic client library that is compiled for Linux using the build described in this document..
When imported into //third_party/cobalt
within a Fuchsia checkout, the Cobalt client library is compiled for Fuchsia and accessed via a FIDL wrapper. If you are trying to use Cobalt from an application running on Fuchsia, stop reading this document and instead read Cobalt's README.md in the Fuchsia repo.
sudo apt-get install libstdc++-8-dev
For example via
git clone https://fuchsia.googlesource.com/cobalt
cd cobalt
./cobaltb.py setup
. This will take a few minutes the first time. It does the following:
third_party
dir via git submodules.sysroot
dir. This uses a tool called cipd
or Chrome Infrastructure Package Deployer.The Python script cobaltb.py in the root directory is used to orchestrate building, testing and deploying to Google Kubernetes Engine. It was already used above in ./cobaltb.py setup
.
./cobaltb.py -h
for general helpcobaltb.py <command> -h
for help on a commandcobaltb.py <command> <subcommand> -h
for help on a sub-commandIf you pass the flag --verbose
to various cobaltb.py
commands you will see more verbose output. Pass it multiple times to increase the verbosity further.
./cobaltb.py clean
./cobaltb.py build
The Cobalt build uses GN and ninja.
./cobaltb.py lint
See: clang-tidy for an explanation of clang-tidy configuration options.
./cobaltb.py test
This runs the whole suite of tests finally running the the end-to-end test. The tests should all pass.--tests=
argument../cobaltb.py test -h
for documentation about the --tests=
argument.Cobalt uses the Gerrit code review tool. See the submitting-changes section of the Fuchsia contributing doc for more info about using Gerrit. But note that the stand-alone Cobalt build currently does not use Jiri. Use the command git push origin HEAD:refs/for/master
. Also the other sections of that document are not relevant to stand-alone Cobalt.
The source layout is related to Cobalt‘s process architecture. Here we describe the source layout and process architecture together. Most of Cobalt’s code is C++. The Shuffler is written in Go.
The most interesting contents of the root directory are the .proto files observation.proto
, which contains the definitions of Observation and Envelope, and encrypted_message.proto
, which contains the definition of EncryptedMessage. Observations are the basic units of data captured by a Cobalt client application. Each Observation is encrypted and the bytes are stored in an EncryptedMessage. Multiple EncryptedMessages are stored in an ObservationBatch. Multiple ObservationBatches are stored in an Envelope. Envelopes are sent via gRPC from the Encoder to the Shuffler.
This directory contains GN files needed for building Cobalt. Most of these files should be similar if not identical to the associated file in the Fuchsia repository here.
This directory contains development keys for encrypting observations/envelopes to the backend.
This directory contains a Jiri manifest. It is used to integrate Cobalt into the rest of the Fuchsia build when Cobalt is imported into third_party/cobalt. This is not used at all in Cobalt's stand-alone build.
This directory contains cmx files for running Cobalt's tests in fuchsia.
This directory is the root directory of the Cobalt source code. All new code should be put in a subdirectory of src/
This directory contains the implementations of Cobalt's privacy- preserving algorithms. This code is linked into both the Encoder, which uses it to encode Observations, and the ReportMaster, which uses it to decode Observations.
This directory contains the code for Cobalt's Encoder, which is a client library whose job is to encode Observations using one of several privacy-preserving encodings, and send Envelopes to the Shuffler using gRPC.
This directory contains the code for Cobalt 1.0's Logger, which is a client library whose job is to encode Observation2s using one of several privacy-preserving encodings, and send Envelopes to the shuffler.
This directory contains the code for a linux test client for cobalt.
This directory contains the implementation of Cobalt's config registration system. A client that wants to use Cobalt starts by registering configurations of their Metrics, Encodings and Reports.
This directory contains build, test and deployment tooling.
This directory contains utility libraries used by the Encoder and Analyzer.