sdk/lib/c/include/README.md

# Public headers for the Fuchsia C library

The C library provides a suite of public header files.  These are the files
found by `#include <something.h>` for the various `<something.h>` APIs required
by the C standard or by other standard or traditional system C library APIs,
plus some Fuchsia-specific (Zircon) extensions.

The current implementation provides not only standard C library APIs but also a
subset of POSIX plus some BSD and GNU extensions and some Linux-specific APIs.
In the future, the distinctions between standard C and other APIs will be made
more formal, and some of these things will move elsewhere.  The set of Zircon
extensions and their precise API details are also subject to change.

This directory drives the build logic for all of the public C library headers.
Only some of those headers are maintained directly in this directory.  These
and the other kinds of headers handled here are described below.

## Sources of header files

### Zircon extensions

The Fuchsia C library provides a handful of new APIs that are idiosyncratic to
Fuchsia and mesh with the Zircon system call APIs.  All of these are declared
in `<zircon/*.h>` header files.  Those are maintained [right here](zircon).

### musl headers

The Fuchsia C library began as a fork of [musl](../#musl).  Most of the public
header files are still those inherited from musl (some much modified).  These
are maintained within this repository as the sole source of truth (without
reference to upstream musl, which has long since diverged from Fuchsia's fork).
They are found in a [separate directory](/zircon/third_party/ulib/musl/include)
to reflect their lineage.

### llvm-libc headers

Some headers come from [llvm-libc], and many more will in the future as musl
code is replaced.  Those upstream headers can be updated along with the code,
triggering Fuchsia's [auto-rollers].

The llvm-libc [`include`](/third_party/llvm-libc/src/include) subdirectory
contains the public header files.  Some of these are used as public headers by
the Fuchsia C library.

#### Generated llvm-libc headers

The llvm-libc public header files comprise two categories:
 * Hand-maintained files of types and macros.
 * Generated files of function declarations.

Most of the hand-maintained files are "private" implementation headers in the
`llvm-libc-macros` and `llvm-libc-types` subdirectories of [llvm-libc's
`include`](https://github.com/llvm/llvm-project/tree/main/libc/include).  These
files are "public" in the sense that they are installed and visible to users.
But they are "private" in the sense that their names and details are not part
of the public API; instead they are included transitively by the truly public
API headers as part of how they implement the public API declarations.

Most of the public API headers do not exist directly as `.h` files in
[`include`](https://github.com/llvm/llvm-project/tree/main/libc/include).
Instead, they are generated by a build tool implemented in Python,
[`hdrgen`](https://github.com/llvm/llvm-project/tree/main/libc/utils/hdrgen).
For each generated `.h` file there is a corresponding `.yaml` file in
[`include`](https://github.com/llvm/llvm-project/tree/main/libc/include).
These files are not installed themselves, but they are the source of truth for
what goes into the public API headers.  The [llvm-libc developers'
documentation](https://libc.llvm.org/dev/header_generation.html) says more.

## Uses of header files

In "end-user" scenarios, the C library header files are always found implicitly
by the compiler without `-I` switches or build system plumbing about it.
Moreover, the compiler comes with header files of its own and is part of a
toolchain providing other libraries such as [libc++](https://libcxx.llvm.org/).
These toolchain-supplied header files can have some arcane interactions with C
library header files.  Normally, the compiler will first see header files from
the user's own build (via `-I` switches supplied through the build system);
then the toolchain's own header files; then the C library's header files.  If
instead the order is changed so that the C library's header files take
precedence over the toolchain-supplied header files, things go wrong.

### sysroot

The compiler and linker can take a `--sysroot=...` switch.  The directory this
points to is often called "the sysroot".  This is the root directory in which
header files and libraries will be found by default search paths.  When no
switch is provided, the compiler may be configured with a default sysroot
directory or may have default behavior that's roughly similar to `--sysroot=/`.
(In a traditional self-hosted Unix-style situation, that finds `/usr/include`,
`/usr/lib`, etc.)  The compiler's defaults for include paths, library paths,
and link inputs (such as the C library itself) are all relative to the sysroot
(except for those supplied by the toolchain itself, which are always relative
to the compiler's own location).

For most users, this is how both header files and link-time artifacts for the C
library (and perhaps other "system" libraries) are usually found.  To do
anything else requires special build plumbing and a potentially complex array
of compiler switches.  However, this usual way of things cannot apply to
building the C library itself, or anything else that goes into the sysroot,
without creating a circular dependency.

### Fuchsia SDK

The [Fuchsia SDK](/docs/development/sdk) includes the C library's headers along
with prebuilt binary artifacts for linking against the C library; for
redistributing the shared library binaries in Fuchsia packages; and for
debugging those binaries.  These headers and link-time artifacts are included
in a [sysroot](#sysroot) that is the foundation of public C and C++ APIs (and
public ABIs across languages) for Fuchsia SDK users.  As such, there are strict
controls on changes to the API and ABI surfaces defined by these header files
(as for the rest of the SDK).

In particular, every change to a header file in the sysroot (or any other
public header file in the SDK) requires an update to a `.api` file and must go
through [API review](/docs/contribute/governance/api_council.md#api-review).
This **cannot be fully automated, as a matter of prudently rigid policy**:
human intervention by an API reviewer is always required to provide approval
before a change can land.

### Fuchsia platform GN build

The Fuchsia GN build does not use the Fuchsia SDK per se.  Rather, it builds
many of the binary artifacts and assembles many of the source and header files
that will go into the SDK.  Among these, it builds the [sysroot](#sysroot) to
go into the SDK.  This contains copies of all the public header files for the C
library.

This sysroot is used to compile most Fuchsia userland code in the GN build.
It's automatically updated before it's used, so it automatically gets any
changes made to the header files maintained here in this repository and its
submodules (such as [llvm-libc](/third_party/llvm-libc/src)) in the local
checkout.  When this happens, those changes cannot land without necessary API
review procedures.

#### sysroot.api

For the C library, [`sysroot.api`](/zircon/public/sysroot_sdk/sysroot.api) is
the file that must be updated.  This will usually happen incidentally in a
build.  But it can always be checked explicitly with:

```shell
fx build //zircon/public/sysroot_sdk
```

This will fail for any changes by default.  To update it in the source tree,
set [`update_goldens = true`](/docs/gen/build_arguments.md#update_goldens) in
`args.gn` first.

The updated `.api` file must be included in the same commit that touches the
header files in the source tree.  Gerrit will not allow the change to land
without [API review](/docs/contribute/governance/api_council.md#api-review).
(Changing any `.api` file will trigger the API review requirement in Gerrit;
failing to change it will make the CQ bots fail.)

#### No automatic API review

As mentioned earlier, [llvm-libc] updated to a new revision in Fuchsia
[manifests] by the [auto-rollers].  No [Gerrit] change can land that
invalidates any `.api` file in the Fuchsia repository.  Manual approval is
always required, even for [auto-rollers] that usually get automatic approval.

### Building libc

When building the C library itself, no sysroot is used.  Instead, the header
files are used directly from [the source tree](#sources-of-header-files).  This
ensures that the header files always correspond to the code being compiled.
For the public C library headers that come from llvm-libc, this means they come
directly from [`//third_party/llvm-libc/src`](/third_party/llvm-libc/src).

The [generated headers from llvm-libc](#generated-llvm-libc-headers) are
emitted by the build into the `gen/third_party/llvm-libc/src/include`
subdirectory of the build directory.  This directory is used along with the
`include` source directory when building libc.

### Hermetic partial libc for special environments

A few parts of the C library code are used in the kernel, the dynamic linker,
and other special environments.  These are not like most Fuchsia userland code,
and do not use the sysroot.  A subset of the C library code is compiled
separately in these special environments (via other GN toolchains).  (Compiling
the C library itself is really just another one of these special environments.)
All these cases use the headers directly from the source tree in the same way.

### Building llvm-libc unit tests

Most C library code is built a second time for testing purposes.  This version
is used with the [llvm-libc unit tests](/third_party/llvm-libc/src/test).
Unlike when the same code is built for the C library itself or in other special
environments, this is largely treated as normal Fuchsia userland code.  It uses
the default GN toolchain used for the rest of the platform GN build, and thus
uses that same sysroot.  However, this code also uses the public C library
headers directly from the source tree instead of those in the sysroot.

The files in the source tree should preempt the sysroot copies in most cases,
though there are some arcane cases in which the sysroot headers could still be
used.  The rationale is that the library and test code should get the headers
that match, even when Fuchsia userland code generally is still getting a
different version.  During soft transitions, the source tree and the sysroot
may diverge in the contents of some header files.  If this causes problems,
temporary workarounds may be needed to to get through a particular transition.
(It may be sufficient and expedient to temporarily disable big chunks of test
code during a short transition period.)

## The conundrum of the llvm-libc roller vs API review

Two competing principles are particularly relevant to the Fuchsia C library:
 1. Automation is essential to effective software development.
 2. Careful human review is essential to effective API maintenance.

The first is exemplified by the [auto-roller](../#llvm-libc_auto_roller) from
the LLVM repository.  The second is exemplified by the strict API review
requirement for all changes to public header files.  The compromise between
these two is reached by semi-automated updates with manual API review via
[manual rolls].

**NOTE:** This means that any upstream header change will cause the auto-roller
to start failing.  This is resolved using [manual rolls], as described below.

### Landing llvm-libc header updates

Upstream llvm-libc changes can produce two kinds of differences in headers:
 1. Different contents of some existing headers.
 2. A different set of headers to install.

Changes to the contents are reflected in [the generated sysroot](#sysroot) just
like any local edits made here in this source tree.  With a the revision update
in the [manifests], follow the same [procedure described above](#sysroot_api).

Changes to the set of headers can come from an upstream change due to a
generated header changing what subordinate implementation headers it uses.
They can also come from changing the [`llvm_libc_generated_headers`](BUILD.gn)
list.  In either case, the build will notice the difference when it generates
the [`llvm-libc-generated.json`](llvm-libc-generated.json) file.  This is
generated by the build and verified against the checked-in copy.  Updating that
copy uses the [same mechanism](/docs/gen/build_arguments.md#update_goldens) as
[`sysroot.api`](#sysroot_api).  The procedure for updating them both is the
same, except that it's necessary to run the build **twice** so that the first
build can replace the JSON file that's used by the second build to re-run GN
and then regenerate the sysroot headers following the updated build rules.

---

[auto-rollers]: ../#auto_rollers
[llvm-libc]: ../#llvm_libc
[manifests]: /manifests
[manual rolls]: ../#manual-rolls