Configuring a custom C toolchain
================================

.. External links are here
.. _Configuring CROSSTOOL: https://docs.bazel.build/versions/0.23.0/tutorial/crosstool.html
.. _Understanding CROSSTOOL: https://docs.bazel.build/versions/0.23.0/crosstool-reference.html
.. _Configuring C++ toolchains: https://docs.bazel.build/versions/master/tutorial/cc-toolchain-config.html
.. _cc_library: https://docs.bazel.build/versions/master/be/c-cpp.html#cc_library
.. _crosstool_config.proto: https://github.com/bazelbuild/bazel/blob/master/src/main/protobuf/crosstool_config.proto
.. _go_binary: docs/go/core/rules.md#go_binary
.. _go_library: docs/go/core/rules.md#go_library
.. _toolchain: https://docs.bazel.build/versions/master/be/platform.html#toolchain
.. _#1642: https://github.com/bazelbuild/rules_go/issues/1642

References
----------

* `Configuring CROSSTOOL`_
* `Understanding CROSSTOOL`_
* `Configuring C++ toolchains`_

Introduction
------------

**WARNING:** This documentation is out of date. Some of the linked Bazel
documentation has been deleted in later versions, and there are a number of
TODOs. In particular, building and configuring a cross-compiling C++ toolchain
and testing it with cgo should be covered. `#1642`_ tracks progress on this.

The Go toolchain sometimes needs access to a working C/C++ toolchain in order to
produce binaries that contain cgo code or require external linking. rules_go
uses whatever C/C++ toolchain Bazel is configured to use. This means
`go_library`_ and `cc_library`_ rules can be linked into the same binary (via
the ``cdeps`` attribute in Go rules).

Bazel uses a CROSSTOOL file to configure the C/C++ toolchain, plus a few build
rules that declare constraints, dependencies, and file groups. By default, Bazel
will attempt to detect the toolchain installed on the host machine. This works
in most cases, but it's not hermetic (developers may have completely different
toolchains installed), and it doesn't always work with remote execution. It also
doesn't work with cross-compilation. Explicit configuration is required in these
situations.

This documented is intended to serve as a walk-through for configuring a custom
C/C++ toolchain for use with rules_go.

NOTE: The Go toolchain requires gcc, clang, or something that accepts the same
command-line arguments and produce the same error messages. MSVC is not
supported. This is a limitation of the Go toolchain, not rules_go. cgo infers
the types of C definitions based on the text of error messages.

TODO: Change the example to use a cross-compiling toolchain.

TODO: Add instructions for building a C compiler from scratch.

TODO: Build the standard C library and binutils for use with this toolchain.

Tutorial
--------

In this tutorial, we'll download a binary Clang release and install it into
a new workspace. This workspace can be uploaded into a new repository and
referenced from other Bazel workspaces.

You can find a copy of the example repository described here at
`https://github.com/jayconrod/bazel_cc_toolchains <https://github.com/jayconrod/bazel_cc_toolchains>`_.

Step 1: Create the repository
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Create a new repository and add a WORKSPACE file to the root directory. It
may be empty, but it's probably a good idea to give it a name.

.. code:: bash

  $ cat >WORKSPACE <<EOF
  workspace(name = "bazel_cc_toolchains")
  EOF

Step 2: Download a toolchain
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Download or compile a C toolchain, and install it in a subdirectory of your
workspace. I put it in ``tools``.

Note that this toolchain has Unixy subdirectories like ``bin``, ``lib``, and
``include``.

.. code:: bash

  $ curl http://releases.llvm.org/7.0.0/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz | tar xJ
  $ mv clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04 tools
  $ ls tools
  bin  include  lib  libexec  share

Step 3: Write a CROSSTOOL
~~~~~~~~~~~~~~~~~~~~~~~~~

We'll create a file named ``tools/CROSSTOOL``, which describes our toolchain
to Bazel. If you have more than one C/C++ toolchain (e.g., different tools for
debug and optimized builds, or different compilers for different platforms),
they should all be configured in the same ``CROSSTOOL`` file.

The format for this file is defined in `crosstool_config.proto`_. Specifically,
CROSSTOOL should contain a ``CrosstoolRelease`` message, formatted as text.
Each ``toolchain`` field is a ``CToolchain`` message.

Here's a short example:

.. code:: proto

  major_version: "local"
  minor_version: ""

  toolchain {
    toolchain_identifier: "clang"
    host_system_name: "linux"
    target_system_name: "linux"
    target_cpu: "x86_64"
    target_libc: "x86_64"
    compiler: "clang"
    abi_version: "unknown"
    abi_libc_version: "unknown"

    tool_path { name: "ar" path: "bin/llvm-ar" }
    tool_path { name: "cpp" path: "bin/clang-cpp" }
    tool_path { name: "dwp" path: "bin/llvm-dwp" }
    tool_path { name: "gcc" path: "bin/clang" }
    tool_path { name: "gcov" path: "bin/llvm-profdata" }
    tool_path { name: "ld" path: "bin/ld.lld" }
    tool_path { name: "nm" path: "bin/llvm-nm" }
    tool_path { name: "objcopy" path: "bin/llvm-objcopy" }
    tool_path { name: "objdump" path: "bin/llvm-objdump" }
    tool_path { name: "strip" path: "bin/llvm-strip" }

    compiler_flag: "-no-canonical-prefixes"
    linker_flag: "-no-canonical-prefixes"

    compiler_flag: "-v"
    cxx_builtin_include_directory: "/usr/include"
  }

  default_toolchain {
    cpu: "x86_64"
    toolchain_identifier: "clang"
  }

For a more complete example, build any ``cc_binary`` with Bazel without
explicitly configuring ``CROSSTOOL``, then look at the ``CROSSTOOL`` that
Bazel generates for the automatically detected host toolchain. This can
be found in ``$(bazel info
output_base)/external/bazel_tools/tools/cpp/CROSSTOOL``. (You have to build
something with the host toolchain before this will show up).

Some notes:

* ``toolchain_identifier`` is the main name for the toolchain. You'll refer to
  it using this identifier from other messages and from build files.
* Most of the other fields at the top of ``toolchain`` are descriptive and
  can have any value.
* ``tool_path`` fields describe the various tools Bazel may invoke. The paths
  are relative to the directory that contains the ``CROSSTOOL`` file.
* ``compiler_flag`` and ``linker_flag`` are passed to the compiler and linker
  on each invocation, respectively.
* ``cxx_builtin_include_directory`` is a directory with include files that
  the compiler may read. Without this declaration, these files won't be
  visible in the sandbox. (TODO: make this hermetic).

Step 4: Write a build file
~~~~~~~~~~~~~~~~~~~~~~~~~~

We'll create a set of targets that will link the CROSSTOOL into Bazel's
toolchain system. It's likely this API will change in the future. This will be
in ``tools/BUILD.bazel``.

First, we'll create some ``filegroups`` that we can reference from other rules.

.. code:: bzl

  package(default_visibility = ["//visibility:public"])

  filegroup(
      name = "empty",
      srcs = [],
  )

  filegroup(
      name = "all",
      srcs = glob([
          "bin/*",
          "lib/**",
          "libexec/**",
          "share/**",
      ]),
  )

Next, we'll create a ``cc_toolchain`` target that tells Bazel where to find some
important files. This API is undocumented and will very likely change in the
future. We need to create one of these for each ``toolchain`` in ``CROSSTOOL``.
The ``toolchain_identifier`` and ``cpu`` fields should match, and the
filegroups should cover the files referenced in ``CROSSTOOL``.

.. code:: bzl

  cc_toolchain(
      name = "cc-compiler-clang",
      all_files = ":all",
      compiler_files = ":all",
      cpu = "x86_64",
      dwp_files = ":empty",
      dynamic_runtime_libs = [":empty"],
      linker_files = ":all",
      objcopy_files = ":empty",
      static_runtime_libs = [":empty"],
      strip_files = ":empty",
      supports_param_files = 1,
      toolchain_identifier = "clang",
  )

Finally, we'll create a ``cc_toolchain_suite`` target. This should reference
``cc_toolchain`` targets for all the toolchains in ``CROSSTOOL``. This API is
also undocumented and will probably change.

.. code:: bzl

  cc_toolchain_suite(
      name = "clang-toolchain",
      toolchains = {
          "x86_64": ":cc-compiler-clang",
          "x86_64|clang": ":cc-compiler-clang",
      },
  )

Step 5: Verify your toolchain works
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

At this point, you should be able to build a simple binary by passing a bunch
of extra flags to Bazel.

.. code:: bash

  $ mkdir example
  $ cat >example/hello.c <<EOF
  #include <stdio.h>

  int main() {
    printf("Hello, world!\n");
    return 0;
  }
  EOF

  $ cat >example/BUILD.bazel <<EOF
  cc_binary(
      name = "hello",
      srcs = ["hello.c"],
  )
  EOF

  $ bazel build \
    --crosstool_top=//tools:clang-toolchain \
    --cpu=x86_64 \
    --compiler=clang \
    --host_cpu=x86_64 \
    -s \
    //example:hello

You should see an invocation of ``tools/bin/clang`` in the output.

* ``--crosstool_top`` should be the label for the ``cc_toolchain_suite`` target
  defined earlier.
* ``--cpu=x86_64`` should be the ``cpu`` attribute in ``cc_toolchain`` and in
  the ``toolchain`` message in ``CROSSTOOL``.
* ``--compiler=clang`` should be the ``toolchain_identifier`` attribute in
  ``cc_toolchain`` and in the ``toolchain`` message in ``CROSSTOOL``.
* ``--host_cpu`` should be the same as ``--cpu``. If we were cross-compiling,
  it would be the ``cpu`` value for the execution platform (where actions are
  performed), not the host platform (where Bazel is invoked).
* ``-s`` prints commands.

Step 6: Configure a Go workspace to use the toolchain
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the ``WORSKPACE`` file for your Go project, import the
``bazel_cc_toolchains`` repository. The way you do this may vary depending on
where you've put ``bazel_cc_toolchains``.

.. code:: bzl

  load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

  git_repository(
      name = "bazel_cc_toolchains",
      remote = "https://github.com/jayconrod/bazel_cc_toolchains",
      tag = "v1.0.0",
  )

Create a file named ``.bazelrc`` in the root directory of your Go project
(or add the code below to the end if already exists). Each line comprises a
Bazel command (such as ``build``), an optional configuration name (``clang``)
and a list of flags to be passed to Bazel when that configuration is used.
If the configuration is omitted, the flags will be passed by default.

.. code:: bash

  $ cat >>.bazelrc <<EOF
  build:clang --crosstool_top=@bazel_cc_toolchains//tools:clang-toolchain
  build:clang --cpu=x86_64
  build:clang --compiler=clang
  build:clang --host_cpu=x86_64
  EOF

You can build with ``bazel build --config=clang ...``.

Verify the toolchain is being used by compiling a "Hello world" cgo program.

.. code:: bash

  $ cat >hello.go <<EOF
  package main

  /*
  #include <stdio.h>

  void say_hello() {
    printf("Hello, world!\n");
  }
  */
  import "C"

  func main() {
    C.say_hello()
  }
  EOF

  $ cat >BUILD.bazel <<EOF
  load("@io_bazel_rules_go//go:def.bzl", "go_binary")

  go_binary(
      name = "hello",
      srcs = ["hello.go"],
      cgo = True,
  )

  $ bazel build --config=clang -s //:hello

You should see clang commands in Bazel's output.