| Go Protocol buffers |
| =================== |
| |
| .. _proto_library: https://docs.bazel.build/versions/master/be/protocol-buffer.html#proto_library |
| .. _default Go plugin: https://github.com/golang/protobuf |
| .. _common plugins: #predefined-plugins |
| .. _Go providers: /go/providers.rst |
| .. _GoLibrary: /go/providers.rst#golibrary |
| .. _GoSource: /go/providers.rst#gosource |
| .. _GoArchive: /go/providers.rst#goarchive |
| .. _Gazelle: https://github.com/bazelbuild/bazel-gazelle |
| .. _Make variable substitution: https://docs.bazel.build/versions/master/be/make-variables.html#make-var-substitution |
| .. _Bourne shell tokenization: https://docs.bazel.build/versions/master/be/common-definitions.html#sh-tokenization |
| .. _gogoprotobuf: https://github.com/gogo/protobuf |
| .. _compiler.bzl: compiler.bzl |
| |
| .. role:: param(kbd) |
| .. role:: type(emphasis) |
| .. role:: value(code) |
| .. |mandatory| replace:: **mandatory value** |
| |
| rules_go provides rules that generate Go packages from .proto files. These |
| packages can be imported like regular Go libraries. |
| |
| .. contents:: :depth: 2 |
| |
| ----- |
| |
| Overview |
| -------- |
| |
| Protocol buffers are built with the three rules below. ``go_proto_library`` and |
| ``go_proto_compiler`` may be loaded from ``@io_bazel_rules_go//proto:def.bzl``. |
| |
| * `proto_library`_: This is a Bazel built-in rule. It lists a set of .proto |
| files in its ``srcs`` attribute and lists other ``proto_library`` dependencies |
| in its ``deps`` attribute. ``proto_library`` rules may be referenced by |
| language-specific code generation rules like ``java_proto_library`` and |
| ``go_proto_library``. |
| * `go_proto_library`_: Generates Go code from .proto files using one or more |
| proto plugins, then builds that code into a Go library. ``go_proto_library`` |
| references ``proto_library`` sources via the ``proto`` attribute. They may |
| reference other ``go_proto_library`` and ``go_library`` dependencies via the |
| ``deps`` attributes. ``go_proto_library`` rules can be depended on or |
| embedded directly by ``go_library`` and ``go_binary``. |
| * `go_proto_compiler`_: Defines a protoc plugin. By default, |
| ``go_proto_library`` generates Go code with the `default Go plugin`_, but |
| other plugins can be used by setting the ``compilers`` attribute. A few |
| `common plugins`_ are provided in ``@io_bazel_rules_go//proto``. |
| |
| The ``go_proto_compiler`` rule produces a `GoProtoCompiler`_ provider. If you |
| need a greater degree of customization (for example, if you don't want to use |
| protoc), you can implement a compatible rule that returns one of these. |
| |
| The ``go_proto_library`` rule produces the normal set of `Go providers`_. This |
| makes it compatible with other Go rules for use in ``deps`` and ``embed`` |
| attributes. |
| |
| Avoiding conflicts |
| ------------------ |
| |
| When linking programs that depend on protos, care must be taken to ensure that |
| the same proto isn't registered by more than one package. This may happen if |
| you depend on a ``go_proto_library`` and a vendored ``go_library`` generated |
| from the same .proto files. You may see compile-time, link-time, or run-time |
| errors as a result of this. |
| |
| There are two main ways to avoid conflicts. |
| |
| Option 1: Use go_proto_library exclusively |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| You can avoid proto conflicts by using ``go_proto_library`` to generate code |
| at build time and avoiding ``go_library`` rules based on pre-generated .pb.go |
| files. |
| |
| Gazelle generates rules in this mode by default. When .proto files are present, |
| it will generate ``go_proto_library`` rules and ``go_library`` rules that embed |
| them (which are safe to use). Gazelle will automatically exclude .pb.go files |
| that correspond to .proto files. If you have .proto files belonging to multiple |
| packages in the same directory, add the following directives to your |
| root build file: |
| |
| .. code:: bzl |
| |
| # gazelle:proto package |
| # gazelle:proto_group go_package |
| |
| rules_go provides ``go_proto_library`` rules for commonly used proto libraries. |
| The Well Known Types can be found in the ``@io_bazel_rules_go//proto/wkt`` |
| package. There are implicit dependencies of ``go_proto_library`` rules |
| that use the default compiler, so they don't need to be written |
| explicitly in ``deps``. You can also find rules for Google APIs and gRPC in |
| ``@go_googleapis//``. You can list these rules with the commands: |
| |
| .. code:: bash |
| |
| $ bazel query 'kind(go_proto_library, @io_bazel_rules_go//proto/wkt:all)' |
| $ bazel query 'kind(go_proto_library, @go_googleapis//...)' |
| |
| Some commonly used Go libraries, such as ``github.com/golang/protobuf/ptypes``, |
| depend on the Well Known Types. In order to avoid conflicts when using these |
| libraries, separate versions of these libraries are provided with |
| ``go_proto_library`` dependencies. Gazelle resolves imports of these libraries |
| automatically. For example, it will resolve ``ptypes`` as |
| ``@com_github_golang_protobuf//ptypes:go_default_library_gen``. |
| |
| Option 2: Use pre-generated .pb.go files |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| You can also avoid conflicts by generating .pb.go files ahead of time and using |
| those exclusively instead of using ``go_proto_library``. This may be a better |
| option for established Go projects that also need to build with ``go build``. |
| |
| Gazelle can generate rules for projects built in this mode. Add the following |
| comment to your root build file: |
| |
| .. code:: bzl |
| |
| # gazelle:proto disable_global |
| |
| This prevents Gazelle from generating ``go_proto_library`` rules. .pb.go files |
| won't be excluded, and all special cases for imports (such as ``ptypes``) are |
| disabled. |
| |
| If you have ``go_repository`` rules in your ``WORKSPACE`` file that may |
| have protos, you'll also need to add |
| ``build_file_proto_mode = "disable_global"`` to those as well. |
| |
| .. code:: bzl |
| |
| go_repository( |
| name = "com_example_some_project", |
| importpath = "example.com/some/project", |
| tag = "v0.1.2", |
| build_file_proto_mode = "disable_global", |
| ) |
| |
| A note on vendored .proto files |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| By default, Bazel assumes imports in .proto files are relative to a repository |
| root directory. This means, for example, if you import ``"foo/bar/baz.proto"``, |
| that file must be in the directory ``foo/bar``, not |
| ``vendor/example.com/repo/foo/bar``. |
| |
| To deal with this, use the `strip_import_prefix` option in the proto_library_ |
| for the vendored file. |
| |
| API |
| --- |
| |
| go_proto_library |
| ~~~~~~~~~~~~~~~~ |
| |
| ``go_proto_library`` generates a set of .go files from a set of .proto files |
| (specified in a ``proto_library`` rule), then builds a Go library from those |
| files. ``go_proto_library`` can be imported like any ``go_library`` rule. |
| |
| Providers |
| ^^^^^^^^^ |
| |
| * GoLibrary_ |
| * GoSource_ |
| * GoArchive_ |
| |
| Attributes |
| ^^^^^^^^^^ |
| |
| +---------------------+----------------------+-------------------------------------------------+ |
| | **Name** | **Type** | **Default value** | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | :param:`name` | :type:`string` | |mandatory| | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | A unique name for this rule. | |
| | | |
| | By convention, and in order to interoperate cleanly with Gazelle_, this | |
| | should be a name like ``foo_go_proto``, where ``foo`` is the Go package name | |
| | or the last component of the proto package name (hopefully the same). The | |
| | ``proto_library`` referenced by ``proto`` should be named ``foo_proto``. | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | :param:`proto` | :type:`label` | |mandatory| | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | Points to the ``proto_library`` containing the .proto sources this rule | |
| | should generate code from. Avoid using this argument, use ``protos`` instead. | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | :param:`protos` | :type:`label` | |mandatory| | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | List of ``proto_library`` targets containing the .proto sources this rule should generate | |
| | code from. This argument should be used instead of ``proto`` argument. | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | :param:`deps` | :type:`label_list` | :value:`[]` | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | List of Go libraries this library depends on directly. Usually, this will be | |
| | a list of ``go_proto_library`` rules that correspond to the ``deps`` of the | |
| | ``proto_library`` rule referenced by ``proto``. | |
| | | |
| | Additional dependencies may be added by the proto compiler. For example, the | |
| | default compiler implicitly adds dependencies on the ``go_proto_library`` | |
| | rules for the Well Known Types. | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | :param:`importpath` | :type:`string` | |mandatory| | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | The source import path of this library. Other libraries can import this | |
| | library using this path. This must be specified in ``go_proto_library`` or | |
| | inherited from one of the targets in ``embed``. | |
| | | |
| | ``importpath`` must match the import path specified in ``.proto`` files using | |
| | ``option go_package``. The option determines how ``.pb.go`` files generated | |
| | for protos importing this proto will import this package. | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | :param:`importmap` | :type:`string` | :value:`""` | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | The Go package path of this library. This is mostly only visible to the | |
| | compiler and linker, but it may also be seen in stack traces. This may be | |
| | set to prevent a binary from linking multiple packages with the same import | |
| | path, e.g., from different vendor directories. | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | :param:`embed` | :type:`label_list` | :value:`[]` | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | List of Go libraries that should be combined with this library. The ``srcs`` | |
| | and ``deps`` from these libraries will be incorporated into this library when it | |
| | is compiled. Embedded libraries must have the same ``importpath`` and | |
| | Go package name. | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | :param:`gc_goopts` | :type:`string_list` | :value:`[]` | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | List of flags to add to the Go compilation command when using the gc | |
| | compiler. Subject to `Make variable substitution`_ and `Bourne shell tokenization`_. | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | :param:`compiler` | :type:`label` | :value:`None` | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | Equivalent to ``compilers`` with a single label. | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | :param:`compilers` | :type:`label_list` | :value:`["@io_bazel_rules_go//proto:go_proto"]` | |
| +---------------------+----------------------+-------------------------------------------------+ |
| | List of rules producing `GoProtoCompiler`_ providers (normally | |
| | `go_proto_compiler`_ rules). This is usually understood to be a list of | |
| | protoc plugins used to generate Go code. See `Predefined plugins`_ for | |
| | some options. | |
| +---------------------+----------------------+-------------------------------------------------+ |
| |
| Example: Basic proto |
| ^^^^^^^^^^^^^^^^^^^^ |
| |
| Suppose you have two .proto files in separate packages: foo/foo.proto and |
| bar/bar.proto. foo/foo.proto looks like this: |
| |
| .. code:: proto |
| |
| syntax = "proto3"; |
| |
| option go_package = "example.com/repo/foo"; |
| |
| import "google/protobuf/any.proto"; |
| import "bar/bar.proto"; |
| |
| message Foo { |
| bar.Bar x = 1; |
| google.protobuf.Any y = 2; |
| }; |
| |
| In foo/BUILD.bazel, we need to declare a ``proto_library`` rule that lists |
| foo.proto in its ``srcs`` attribute. Since we import some other protos, we |
| also need a label in ``deps`` for each imported package. We will need to |
| create another ``proto_library`` in bar/BUILD.bazel, but we can use an |
| existing library for any.proto, since it's one of the Well Known Types. |
| |
| .. code:: bzl |
| |
| proto_library( |
| name = "foo_proto", |
| srcs = ["foo.proto"], |
| deps = [ |
| "//bar:bar_proto", |
| "@com_google_protobuf//:any_proto", |
| ], |
| visibility = ["//visibility:public"], |
| ) |
| |
| In order to these this proto in Go, we need to declare a ``go_proto_library`` |
| that references to ``proto_library`` to be built via the ``proto`` attribute. |
| Like ``go_library``, an ``importpath`` attribute needs to be declared. |
| Ideally, this should match the ``option go_package`` declaration in the .proto |
| file, but this is not required. We also need to list Go packages that the |
| generated Go code imports in the ``deps`` attributes. Generally, ``deps`` |
| in ``go_proto_library`` will correspond with ``deps`` in ``proto_library``, |
| but the Well Known Types don't need to be listed (they are added automatically |
| by the compiler in use). |
| |
| .. code:: bzl |
| |
| load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library") |
| |
| go_proto_library( |
| name = "foo_go_proto", |
| importpath = "example.com/repo/foo", |
| proto = ":foo_proto", |
| visibility = ["//visibility:public"], |
| deps = ["//bar:bar_go_proto"], |
| ) |
| |
| This library can be imported like a regular Go library by other rules. |
| |
| .. code:: bzl |
| |
| load("@io_bazel_rules_go//go:def.bzl", "go_binary") |
| |
| go_binary( |
| name = "main", |
| srcs = ["main.go"], |
| deps = ["//foo:foo_go_proto"], |
| ) |
| |
| If you need to add additional source files to a package built from protos, |
| you can do so with a separate ``go_library`` that embeds the |
| ``go_proto_library``. |
| |
| .. code:: bzl |
| |
| load("@io_bazel_rules_go//go:def.bzl", "go_library") |
| |
| go_library( |
| name = "foo", |
| srcs = ["extra.go"], |
| embed = [":foo_go_proto"], |
| importpath = "example.com/repo/foo", |
| visibility = ["//visibility:public"], |
| ) |
| |
| For convenience, ``proto_library``, ``go_proto_library``, and ``go_binary`` |
| can all be generated by Gazelle_. |
| |
| Example: gRPC |
| ^^^^^^^^^^^^^ |
| |
| To compile protos that contain service definitions, just use the ``go_grpc`` |
| plugin. |
| |
| .. code:: bzl |
| |
| load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library") |
| |
| proto_library( |
| name = "foo_proto", |
| srcs = ["foo.proto"], |
| visibility = ["//visibility:public"], |
| ) |
| |
| go_proto_library( |
| name = "foo_go_proto", |
| compilers = ["@io_bazel_rules_go//proto:go_grpc"], |
| importpath = "example.com/repo/foo", |
| proto = ":foo_proto", |
| visibility = ["//visibility:public"], |
| deps = ["//bar:bar_go_proto"], |
| ) |
| |
| go_proto_compiler |
| ~~~~~~~~~~~~~~~~~ |
| |
| ``go_proto_compiler`` describes a plugin for protoc, the proto compiler. |
| Different plugins will generate different Go code from the same protos. |
| Compilers may be chosen through the ``compilers`` attribute of |
| ``go_proto_library``. |
| |
| Several instances of this rule are listed in `Predefined plugins`_. You will |
| only need to use this rule directly if you need a plugin which is not there. |
| |
| Providers |
| ^^^^^^^^^ |
| |
| * GoProtoCompiler_ |
| * GoLibrary_ |
| * GoSource_ |
| |
| Attributes |
| ^^^^^^^^^^ |
| |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | **Name** | **Type** | **Default value** | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | :param:`name` | :type:`string` | |mandatory| | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | A unique name for this rule. | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | :param:`deps` | :type:`label_list` | :value:`[]` | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | List of Go libraries that Go code *generated by* this compiler depends on | |
| | implicitly. Rules in this list must produce the `GoLibrary`_ provider. This | |
| | should contain libraries for the Well Known Types at least. | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | :param:`options` | :type:`string_list` | :value:`[]` | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | List of command line options to be passed to the compiler. Each option will | |
| | be preceded by ``--option``. | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | :param:`suffix` | :type:`string` | :value:`.pb.go` | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | File name suffix of generated Go files. ``go_proto_compiler`` assumes that | |
| | one Go file will be generated for each input .proto file. Output file names | |
| | will have the .proto suffix removed and this suffix appended. For example, | |
| | ``foo.proto`` will become ``foo.pb.go``. | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | :param:`valid_archive` | :type:`bool` | :value:`True` | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | Whether code generated by this compiler can be compiled into a standalone | |
| | archive file without additional sources. | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | :param:`import_path_option` | :type:`bool` | :value:`True` | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | When true, the ``importpath`` attribute from ``go_proto_library`` rules | |
| | using this compiler will be passed to the compiler on the command line as | |
| | ``--option import_path={}``. | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | :param:`plugin` | :type:`label` | :value:`@com_github_golang_protobuf//protoc-gen-go` | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| | The plugin to use with protoc via the ``--plugin`` option. This rule must | |
| | produce an executable file. | |
| +-----------------------------+----------------------+-----------------------------------------------------+ |
| |
| Predefined plugins |
| ------------------ |
| |
| Several ``go_proto_compiler`` rules are predefined in |
| ``@io_bazel_rules_go//proto``. |
| |
| * ``go_proto``: default plugin from github.com/golang/protobuf. |
| * ``go_grpc``: default gRPC plugin. |
| * gogoprotobuf_ plugins for the variants ``combo``, ``gofast``, ``gogo``, |
| ``gogofast``, ``gogofaster``, ``gogoslick``, ``gogotypes``, ``gostring``. |
| For each variant, there is a regular version (e.g., ``gogo_proto``) and a |
| gRPC version (e.g., ``gogo_grpc``). |
| |
| Providers |
| --------- |
| |
| Providers are objects produced by Bazel rules and consumed by other rules that |
| depend on them. See `Go providers`_ for information about Go providers, |
| specifically GoLibrary_, GoSource_, and GoArchive_. |
| |
| GoProtoCompiler |
| ~~~~~~~~~~~~~~~ |
| |
| GoProtoCompiler is the provider returned by the ``go_proto_compiler`` rule and |
| anything compatible with it. The ``go_proto_library`` rule expects any rule |
| listed in its ``compilers`` attribute to provide ``GoProtoCompiler``. If the |
| ``go_proto_compiler`` rule doesn't do what you need (e.g., you don't want to |
| use protoc), you can write a new rule that produces this. |
| |
| ``GoProtoCompiler`` is loaded from ``@io_bazel_rules_go//proto:def.bzl``. |
| |
| ``GoProtoCompiler`` has the fields described below. Additional fields may be |
| added to pass information to the ``compile`` function. This interface is |
| *not final* and may change in the future. |
| |
| +-----------------------------+-------------------------------------------------+ |
| | **Name** | **Type** | |
| +-----------------------------+-------------------------------------------------+ |
| | :param:`deps` | :type:`Target list` | |
| +-----------------------------+-------------------------------------------------+ |
| | A list of Go libraries to be added as dependencies to any | |
| | ``go_proto_library`` compiled with this compiler. Each target must provide | |
| | GoLibrary_, GoSource_, and GoArchive_. This list should include libraries | |
| | for the Well Known Types and anything else considered "standard". | |
| +-----------------------------+-------------------------------------------------+ |
| | :param:`compile` | :type:`Function` | |
| +-----------------------------+-------------------------------------------------+ |
| | A function which declares output files and actions when called. See | |
| | `compiler.bzl`_ for details. | |
| +-----------------------------+-------------------------------------------------+ |
| | :param:`valid_archive` | :type:`bool` | |
| +-----------------------------+-------------------------------------------------+ |
| | Whether the compiler produces a complete Go library. Compilers that just add | |
| | methods to structs produced by other compilers will set this to false. | |
| +-----------------------------+-------------------------------------------------+ |
| |
| Dependencies |
| ------------ |
| |
| In order to support protocol buffers, rules_go declares the external |
| repositories listed below in ``go_rules_dependencies()``. These repositories |
| will only be downloaded if proto rules are used. |
| |
| * ``@com_google_protobuf (github.com/google/protobuf)``: Well Known Types and |
| general proto support. |
| * ``@com_github_golang_protobuf (github.com/golang/protobuf)``: standard |
| Go proto plugin. |
| * ``@com_github_gogo_protobuf (github.com/gogo/protobuf)``: gogoprotobuf |
| plugins. |
| * ``@org_golang_google_grpc (github.com/grpc/grpc-go``: gRPC support. |
| * gRPC dependencies |
| |
| * ``@org_golang_x_net (golang.org/x/net)`` |
| * ``@org_golang_x_text (golang.org/x/text)`` |
| * ``@org_golang_google_genproto (google.golang.org/genproto)`` |