Google Git
Sign in
fuchsia / cobalt / f105c10ad9750a0e894238d41ee4a5e47dec3cec
commitf105c10ad9750a0e894238d41ee4a5e47dec3cec[log] [tgz]
authorukode <ukode@google.com>Fri Sep 30 10:29:28 2016 -0700
committerSean Klein <smklein@google.com>Thu Nov 03 15:26:48 2016 -0700
tree263a9996d65374b9ecc5138d14683d2249183290
parent40d3fc9eafe6d7bb0a278226a024a8afe5488bac [diff]
Initial skeleton code for Shuffler receiver service with an end to end
local test added for grpc testing.

The proto added is an initial version and will get updated once the
proto syntax on encoder side is finalized.

On the build infrastructure for shuffler, grpc and protobuf3 compilers
are required along with their corresponding golang plugins. Grpc and
protobuf3 dependencies are already added through setup.sh script, and
the corresponding go plugins are checked in to the thirdparty/go folder
as part of this cl.

Also added the submodule initialization for protobuf GO and grpc-go.

Change-Id: If067cf0d909fdf6d7c0b9e966ec74ea94d35f4da
18 files changed
tree: 263a9996d65374b9ecc5138d14683d2249183290
  1. algorithms/
  2. analyzer/
  3. docker/
  4. prototype/
  5. shuffler/
  6. third_party/
  7. tools/
  8. util/
  9. .gitignore
  10. .gitmodules
  11. CMakeLists.txt
  12. cobalt.proto
  13. cobaltb.py
  14. encodings.proto
  15. metrics.proto
  16. payload.proto
  17. README.md
  18. setup.sh
README.md

Cobalt

An extensible, privacy-preserving, user-data analysis pipeline.

go/cobalt-for-privacy

  • Prerequisites

    • Currently this repo has only been tested on Goobuntu.

  • One-time setup:

    • This repo uses git modules. After pulling it you must do:
      • git submodule init
      • git submodule update
    • You must install the following dependencies:
      • clang
      • cmake
      • ninja-build
      • golang
      • protobuf3
      • gRPC
      • run setup.sh as root to install the dependencies

  • The script cobaltb.py orchestrates building and testing Cobalt.

    • cobaltb.py build
    • cobaltb.py test
    • cobaltb.py -h for help

  • See the prototype subdirectory for the Cobalt prototype demo

  • Troubleshooting:

    • If setup.sh or compiling fails, try removing any previously installed versions of protobuf, grpc and golang.

Google Container Engine (GCE)

The cobaltb.py tool is also a helper to interact with GCE. The following commands are supported:

gce_build - Build the docker images for use on GCE.

gce_push - Publish the built docker images to the GCE repository.

gce_start - Start the cobalt components. To see the external IP of services run, for example: kubectl get service analyzer

gce_stop - Stops the cobalt components.

GCE Prerequisites

  • sudo apt-get install docker-engine

  • sudo usermod -aG docker

    • You'll have to login again to be added to the docker group.

  • Install gcloud: https://cloud.google.com/sdk/

  • gcloud init

  • gcloud components install kubectl

GCE Authentication

Tasks like gce_start, gce_stop or running the Analyzer outside of GCE require setting the environment variable GOOGLE_APPLICATION_CREDENTIALS to point to the JSON file containing credentials. Follow the instructions of step “1.” of “How the Application Default Credentials work” from:

https://developers.google.com/identity/protocols/application-default-credentials