| :::{default-domain} bzl | |
| ::: | |
| (configuring-toolchains)= | |
| # Configuring Python toolchains and runtimes | |
| This document explains how to configure the Python toolchain and runtimes for different | |
| use cases. | |
| ## Bzlmod MODULE configuration | |
| How to configure `rules_python` in your `MODULE.bazel` file depends on how and why | |
| you're using Python. There are four basic use cases: | |
| 1. A root module that always uses Python. For example, you're building a | |
| Python application. | |
| 2. A library module with dev-only uses of Python. For example, a Java project | |
| that only uses Python as part of testing itself. | |
| 3. A library module without version constraints. For example, a rule set with | |
| Python build tools, but defers to the user as to what Python version is used | |
| for the tools. | |
| 4. A library module with version constraints. For example, a rule set with | |
| Python build tools, and the module requires a specific version of Python | |
| be used with its tools. | |
| ### Root modules | |
| Root modules are always the top-most module. These are special in two ways: | |
| 1. Some `rules_python` bzlmod APIs are only respected by the root module. | |
| 2. The root module can force module overrides and specific module dependency | |
| ordering. | |
| When configuring `rules_python` for a root module, you typically want to | |
| explicitly specify the Python version you want to use. This ensures that | |
| dependencies don't change the Python version out from under you. Remember that | |
| `rules_python` will set a version by default, but it will change regularly as | |
| it tracks a recent Python version. | |
| NOTE: If your root module only uses Python for development of the module itself, | |
| you should read the dev-only library module section. | |
| ``` | |
| bazel_dep(name="rules_python", version=...) | |
| python = use_extension("@rules_python//python/extensions:python.bzl", "python") | |
| python.defaults(python_version = "3.12") | |
| python.toolchain(python_version = "3.12") | |
| ``` | |
| ### Library modules | |
| A library module is a module that can show up in arbitrary locations in the | |
| Bzlmod module graph -- it's unknown where in the breadth-first search order the | |
| module will be relative to other modules. For example, `rules_python` is a | |
| library module. | |
| #### Library modules with dev-only Python usage | |
| A library module with dev-only Python usage is usually one where Python is only | |
| used as part of its tests. For example, a module for Java rules might run some | |
| Python program to generate test data, but real usage of the rules don't need | |
| Python to work. To configure this, follow the root-module setup, but remember to | |
| specify `dev_dependency = True` to the bzlmod APIs: | |
| ``` | |
| # MODULE.bazel | |
| bazel_dep(name = "rules_python", version=..., dev_dependency = True) | |
| python = use_extension( | |
| "@rules_python//python/extensions:python.bzl", | |
| "python", | |
| dev_dependency = True | |
| ) | |
| python.defaults(python_version = "3.12") | |
| python.toolchain(python_version = "3.12") | |
| ``` | |
| #### Library modules without version constraints | |
| A library module without version constraints is one where the version of Python | |
| used for the Python programs it runs isn't chosen by the module itself. Instead, | |
| it's up to the root module to pick an appropriate version of Python. | |
| For this case, configuration is simple: just depend on `rules_python` and use | |
| the normal `//python:py_binary.bzl` et al. rules. There is no need to call | |
| `python.toolchain` -- `rules_python` ensures _some_ Python version is available, | |
| but more often, the root module will specify some version. | |
| ``` | |
| # MODULE.bazel | |
| bazel_dep(name = "rules_python", version=...) | |
| ``` | |
| #### Library modules with version constraints | |
| A library module with version constraints is one where the module requires a | |
| specific Python version be used with its tools. This has some pros/cons: | |
| * It allows the library's tools to use a different version of Python than | |
| the rest of the build. For example, a user's program could use Python 3.12, | |
| while the library module's tools use Python 3.10. | |
| * It reduces the support burden for the library module because the library only needs | |
| to test for the particular Python version they intend to run as. | |
| * It raises the support burden for the library module because the version of | |
| Python being used needs to be regularly incremented. | |
| * It has higher build overhead because additional runtimes and libraries need | |
| to be downloaded, and Bazel has to keep additional configuration state. | |
| To configure this, request the Python versions needed in `MODULE.bazel` and use | |
| the version-aware rules for `py_binary`. | |
| ``` | |
| # MODULE.bazel | |
| bazel_dep(name = "rules_python", version=...) | |
| python = use_extension("@rules_python//python/extensions:python.bzl", "python") | |
| python.toolchain(python_version = "3.12") | |
| # BUILD.bazel | |
| load("@rules_python//python:py_binary.bzl", "py_binary") | |
| py_binary(..., python_version="3.12") | |
| ``` | |
| ### Pinning to a Python version | |
| Pinning to a version allows targets to force that a specific Python version is | |
| used, even if the root module configures a different version as a default. This | |
| is most useful for two cases: | |
| 1. For submodules to ensure they run with the appropriate Python version | |
| 2. To allow incremental, per-target, upgrading to newer Python versions, | |
| typically in a monorepo situation. | |
| To configure a submodule with the version-aware rules, request the particular | |
| version you need when defining the toolchain: | |
| ```starlark | |
| # MODULE.bazel | |
| python = use_extension("@rules_python//python/extensions:python.bzl", "python") | |
| python.toolchain( | |
| python_version = "3.11", | |
| ) | |
| use_repo(python) | |
| ``` | |
| Then use the `@rules_python` repo in your `BUILD` file to explicitly pin the Python version when calling the rule: | |
| ```starlark | |
| # BUILD.bazel | |
| load("@rules_python//python:py_binary.bzl", "py_binary") | |
| py_binary(..., python_version = "3.11") | |
| py_test(..., python_version = "3.11") | |
| ``` | |
| Multiple versions can be specified and used within a single build. | |
| ```starlark | |
| # MODULE.bazel | |
| python = use_extension("@rules_python//python/extensions:python.bzl", "python") | |
| python.defaults( | |
| # The environment variable takes precedence if set. | |
| python_version = "3.11", | |
| python_version_env = "BAZEL_PYTHON_VERSION", | |
| ) | |
| python.toolchain( | |
| python_version = "3.11", | |
| ) | |
| python.toolchain( | |
| python_version = "3.12", | |
| ) | |
| # BUILD.bazel | |
| load("@rules_python//python:py_binary.bzl", "py_binary") | |
| load("@rules_python//python:py_test.bzl", "py_test") | |
| # Defaults to 3.11 | |
| py_binary(...) | |
| py_test(...) | |
| # Explicitly use Python 3.11 | |
| py_binary(..., python_version = "3.11") | |
| py_test(..., python_version = "3.11") | |
| # Explicitly use Python 3.12 | |
| py_binary(..., python_version = "3.12") | |
| py_test(..., python_version = "3.12") | |
| ``` | |
| For more documentation, see the bzlmod examples under the {gh-path}`examples` | |
| folder. Look for the examples that contain a `MODULE.bazel` file. | |
| ### Other toolchain details | |
| The `python.toolchain()` call makes its contents available under a repo named | |
| `python_X_Y`, where X and Y are the major and minor versions. For example, | |
| `python.toolchain(python_version="3.14")` creates the repo `@python_3_14`. | |
| Remember to call `use_repo()` to make repos visible to your module: | |
| `use_repo(python, "python_3_14")`. | |
| :::{deprecated} 1.1.0 | |
| The toolchain-specific `py_binary` and `py_test` symbols are aliases to the regular rules. | |
| For example, `load("@python_versions//3.11:defs.bzl", "py_binary")` & `load("@python_versions//3.11:defs.bzl", "py_test")` are deprecated. | |
| Usages of them should be changed to load the regular rules directly. | |
| For example, use `load("@rules_python//python:py_binary.bzl", "py_binary")` & `load("@rules_python//python:py_test.bzl", "py_test")` and then specify the `python_version` when using the rules corresponding to the Python version you defined in your toolchain. {ref}`Library modules with version constraints` | |
| ::: | |
| #### Toolchain usage in other rules | |
| Python toolchains can be utilized in other Bazel rules, such as `genrule()`, by | |
| adding the `toolchains=["@rules_python//python:current_py_toolchain"]` | |
| attribute. You can obtain the path to the Python interpreter using the | |
| `$(PYTHON2)` and `$(PYTHON3)` ["Make" | |
| Variables](https://bazel.build/reference/be/make-variables). See the | |
| {gh-path}`test_current_py_toolchain <tests/load_from_macro/BUILD.bazel>` target | |
| for an example. We also make available `$(PYTHON2_ROOTPATH)` and `$(PYTHON3_ROOTPATH)`, | |
| which are Make Variable equivalents of `$(PYTHON2)` and `$(PYTHON3)` but for runfiles | |
| locations. These will be helpful if you need to set environment variables of binary/test rules | |
| while using [`--nolegacy_external_runfiles`](https://bazel.build/reference/command-line-reference#flag--legacy_external_runfiles). | |
| The original make variables still work in exec contexts such as genrules. | |
| (selecting-the-runtime-archive)= | |
| ### Selecting the runtime archive | |
| python-build-standalone publishes each runtime as several kinds of archive. The | |
| {obj}`--@rules_python//python/config_settings:py_pbs_distribution` | |
| flag selects which one the hermetic toolchains use: | |
| * `auto` (default): let `rules_python` choose; currently `install_only`. | |
| * `install_only`: the runtime with debug symbols. | |
| * `install_only_stripped`: the runtime without debug symbols, which is | |
| substantially smaller. This reduces the size of runtimes and | |
| {obj}`py_zipapp_binary` outputs. | |
| * `full`: the runtime plus build artifacts. | |
| Because this is a build flag, a single workspace can build with different | |
| archives, for example keeping debug symbols in development builds and using | |
| stripped runtimes in production builds: | |
| ``` | |
| # File: .bazelrc | |
| build:prod --@rules_python//python/config_settings:py_pbs_distribution=install_only_stripped | |
| ``` | |
| A toolchain is registered for each archive kind, and only the archive a build | |
| selects is downloaded. If a Python version or platform has no archive of the | |
| selected kind, the `install_only` archive is used instead. For example, | |
| stripped archives are only available for runtimes released from `20240726` | |
| onward. | |
| ### Overriding toolchain defaults and adding more versions | |
| One can perform various overrides for the registered toolchains from the root | |
| module. For example, the following use cases would be supported using the | |
| existing attributes: | |
| * Limiting the available toolchains for the entire `bzlmod` transitive graph | |
| via {attr}`python.override.available_python_versions`. | |
| * Setting particular `X.Y.Z` Python versions when modules request `X.Y` version | |
| via {attr}`python.override.minor_mapping`. | |
| * Per-version control of the coverage tool used using | |
| {attr}`python.single_version_platform_override.coverage_tool`. | |
| * Control of shared `libpython` files using the `libpython` attribute on | |
| {bzl:obj}`python.override`, {bzl:obj}`python.single_version_override`, or | |
| {bzl:obj}`python.single_version_platform_override`. | |
| * Adding additional Python versions via {bzl:obj}`python.single_version_override` or | |
| {bzl:obj}`python.single_version_platform_override`. | |
| * Adding additional Python versions dynamically from a manifest file or URL | |
| via {attr}`python.override.add_runtime_manifest_files` or | |
| {attr}`python.override.add_runtime_manifest_urls`. | |
| The `libpython` attribute accepts `auto`, `include`, or `exclude`, and defaults | |
| to `auto`. In automatic mode, shared `libpython` files are excluded only for | |
| recognized Astral Python Standalone builds from `20250517` onward, which are | |
| known to statically link `libpython` into the interpreter. Unknown, custom, and | |
| older runtimes retain the shared libraries. Use `include` or `exclude` to | |
| override this behavior when mirroring or customizing a runtime. | |
| ### Registering custom runtimes | |
| Because the python-build-standalone project has _thousands_ of prebuilt runtimes | |
| available, `rules_python` only includes popular runtimes in its built-in | |
| configurations. If you want to use a runtime that isn't already known to | |
| `rules_python`, then {obj}`single_version_platform_override()` can be used to do | |
| so. In short, it allows specifying an arbitrary URL and using custom flags | |
| to control when a runtime is used. | |
| In the example below, we register a particular python-build-standalone runtime | |
| that is activated for Linux x86 builds when the custom flag | |
| `--//:runtime=my-custom-runtime` is set. | |
| ``` | |
| # File: MODULE.bazel | |
| bazel_dep(name = "bazel_skylib", version = "1.7.1.") | |
| bazel_dep(name = "rules_python", version = "1.5.0") | |
| python = use_extension("@rules_python//python/extensions:python.bzl", "python") | |
| python.single_version_platform_override( | |
| platform = "my-platform", | |
| python_version = "3.13.3", | |
| sha256 = "01d08b9bc8a96698b9d64c2fc26da4ecc4fa9e708ce0a34fb88f11ab7e552cbd", | |
| os_name = "linux", | |
| arch = "x86_64", | |
| target_settings = [ | |
| "@@//:runtime=my-custom-runtime", | |
| ], | |
| urls = ["https://github.com/astral-sh/python-build-standalone/releases/download/20250409/cpython-3.13.3+20250409-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz"], | |
| ) | |
| # File: //:BUILD.bazel | |
| load("@bazel_skylib//rules:common_settings.bzl", "string_flag") | |
| string_flag( | |
| name = "custom_runtime", | |
| build_setting_default = "", | |
| ) | |
| config_setting( | |
| name = "is_custom_runtime_linux-x86-install-only-stripped", | |
| flag_values = { | |
| ":custom_runtime": "linux-x86-install-only-stripped", | |
| }, | |
| ) | |
| ``` | |
| Notes: | |
| - While any URL and archive can be used, it's assumed their content looks like | |
| a python-build-standalone archive. | |
| - A "version-aware" toolchain is registered, which means the Python version flag | |
| must also match (e.g., `--@rules_python//python/config_settings:python_version=3.13.3` | |
| must be set -- see `minor_mapping` and `is_default` for controls and docs | |
| about version matching and selection). | |
| - The `target_compatible_with` attribute can be used to entirely specify the | |
| argument of the same name that the toolchain uses. | |
| - The labels in `target_settings` must be absolute; `@@` refers to the main repo. | |
| - The `target_settings` are `config_setting` targets, which means you can | |
| customize how matching occurs. | |
| :::{seealso} | |
| See {obj}`//python/config_settings` for flags `rules_python` already defines | |
| that can be used with `target_settings`. Some particular ones of note are | |
| {flag}`--py_linux_libc` and {flag}`--py_freethreaded`, among others. | |
| ::: | |
| :::{versionadded} 1.5.0 | |
| Added support for custom platform names, `target_compatible_with`, and | |
| `target_settings` with `single_version_platform_override`. | |
| ::: | |
| ### Registering runtimes from a manifest | |
| If you want to register multiple custom runtimes or versions at once, you can | |
| use a python-build-standalone manifest file. This is useful if you want to | |
| adopt new versions that are not yet built into `rules_python` without having | |
| to manually define each one using `single_version_platform_override`. | |
| To do this, specify the `add_runtime_manifest_files` or | |
| `add_runtime_manifest_urls` (and `runtime_manifest_sha`) attributes in | |
| `python.override` in your `MODULE.bazel`. | |
| In the example below, we register all runtimes available in a specific local | |
| or remote PBS release manifest: | |
| ```starlark | |
| # File: MODULE.bazel | |
| python = use_extension("@rules_python//python/extensions:python.bzl", "python") | |
| python.override( | |
| add_runtime_manifest_files = [ | |
| "@//:SHA256SUMS", | |
| ], | |
| add_runtime_manifest_urls = [ | |
| "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/SHA256SUMS", | |
| ], | |
| base_url = "https://example.com/downloads", | |
| runtime_manifest_sha = "ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f", | |
| ) | |
| ``` | |
| #### Manifest file format | |
| The manifest must be a plain text file where each line contains the SHA256 hash | |
| and the location of a runtime archive, separated by whitespace: | |
| ``` | |
| <sha256> <location> | |
| ``` | |
| The `<location>` can be either: | |
| - A relative filename (e.g., | |
| `cpython-3.10.20+20260414-x86_64-unknown-linux-gnu-install_only.tar.zst`). | |
| In this case, the download URL is constructed by appending the filename to | |
| the `base_url` attribute (if using `add_runtime_manifest_files`) or to the | |
| parent directory of each URL in `add_runtime_manifest_urls` (treating them | |
| as mirrors). | |
| - An absolute URL (e.g., | |
| `https://example.com/downloads/cpython-3.10.20+20260414-x86_64-unknown-linux-gnu-install_only.tar.zst`). | |
| In this case, the URL is used directly to download the archive. | |
| In both cases, the filename or the last path segment of the URL must follow | |
| the standard python-build-standalone naming convention. `rules_python` parses | |
| this name to extract runtime metadata (such as Python version, target | |
| architecture, operating system, and libc). | |
| Notes: | |
| - `rules_python` will read or download the manifest, parse it, and | |
| automatically register toolchains for all valid Python runtimes found in it | |
| that match supported platforms. | |
| - Only runtimes matching known platforms in `rules_python` will be registered. | |
| :::{versionadded} 2.1.0 | |
| Added support for registering runtimes from a manifest using | |
| `add_runtime_manifest_files`, `add_runtime_manifest_urls`, and | |
| `runtime_manifest_sha` in `python.override`. | |
| ::: | |
| ### Using defined toolchains from WORKSPACE | |
| It is possible to use toolchains defined in `MODULE.bazel` in `WORKSPACE`. For example, | |
| the following `MODULE.bazel` and `WORKSPACE` provides a working {bzl:obj}`pip_parse` setup: | |
| ```starlark | |
| # File: WORKSPACE | |
| load("@rules_python//python:repositories.bzl", "py_repositories") | |
| py_repositories() | |
| load("@rules_python//python:pip.bzl", "pip_parse") | |
| pip_parse( | |
| name = "third_party", | |
| requirements_lock = "//:requirements.txt", | |
| python_interpreter_target = "@python_3_10_host//:python", | |
| ) | |
| load("@third_party//:requirements.bzl", "install_deps") | |
| install_deps() | |
| # File: MODULE.bazel | |
| bazel_dep(name = "rules_python", version = "0.40.0") | |
| python = use_extension("@rules_python//python/extensions:python.bzl", "python") | |
| python.defaults(python_version = "3.10") | |
| python.toolchain(python_version = "3.10") | |
| use_repo(python, "python_3_10", "python_3_10_host") | |
| ``` | |
| Note, the user has to import the `*_host` repository to use the Python interpreter in the | |
| {bzl:obj}`pip_parse` and `whl_library` repository rules, and once that is done, | |
| users should be able to ensure the setting of the default toolchain even during the | |
| transition period when some of the code is still defined in `WORKSPACE`. | |
| ## Workspace configuration | |
| To import `rules_python` in your project, you first need to add it to your | |
| `WORKSPACE` file, using the snippet provided in the | |
| [release you choose](https://github.com/bazel-contrib/rules_python/releases). | |
| To depend on a particular unreleased version, you can do the following: | |
| ```starlark | |
| load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") | |
| # Update the SHA and VERSION to the lastest version available here: | |
| # https://github.com/bazel-contrib/rules_python/releases. | |
| SHA="84aec9e21cc56fbc7f1335035a71c850d1b9b5cc6ff497306f84cced9a769841" | |
| VERSION="0.23.1" | |
| http_archive( | |
| name = "rules_python", | |
| sha256 = SHA, | |
| strip_prefix = "rules_python-{}".format(VERSION), | |
| url = "https://github.com/bazel-contrib/rules_python/releases/download/{}/rules_python-{}.tar.gz".format(VERSION,VERSION), | |
| ) | |
| load("@rules_python//python:repositories.bzl", "py_repositories") | |
| py_repositories() | |
| ``` | |
| ### Workspace toolchain registration | |
| To register a hermetic Python toolchain rather than rely on a system-installed interpreter for runtime execution, you can add to the `WORKSPACE` file: | |
| ```starlark | |
| load("@rules_python//python:repositories.bzl", "python_register_toolchains") | |
| python_register_toolchains( | |
| name = "python_3_14", | |
| # Available versions are listed in @rules_python//python:versions.bzl. | |
| # We recommend using the same version your team is already standardized on. | |
| python_version = "3.14", | |
| ) | |
| load("@rules_python//python:pip.bzl", "pip_parse") | |
| pip_parse( | |
| ... | |
| python_interpreter_target = "@python_3_14_host//:python", | |
| ... | |
| ) | |
| ``` | |
| After registration, your Python targets will use the toolchain's interpreter during execution, but a system-installed interpreter | |
| is still used to "bootstrap" Python targets (see {gh-issue}`691`). | |
| You may also find some quirks while using this toolchain. Please refer to [python-build-standalone documentation's _Quirks_ section](https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html). | |
| ## Local toolchain | |
| It's possible to use a locally installed Python runtime instead of the regular | |
| prebuilt, remotely downloaded ones. A local toolchain contains the Python | |
| runtime metadata (Python version, headers, ABI flags, etc.) that the regular | |
| remotely downloaded runtimes contain, which makes it possible to build, e.g., C | |
| extensions (unlike the autodetecting and runtime environment toolchains). | |
| For simple cases, the {obj}`local_runtime_repo` and | |
| {obj}`local_runtime_toolchains_repo` rules are provided that will introspect a | |
| Python installation and create an appropriate Bazel definition from it. To do | |
| this, three pieces need to be wired together: | |
| 1. Specify a path or command to a Python interpreter (multiple can be defined). | |
| 2. Create toolchains for the runtimes in (1). | |
| 3. Register the toolchains created by (2). | |
| The following is an example that will use `python3` from `PATH` to find the | |
| interpreter, then introspect its installation to generate a full toolchain. | |
| ```starlark | |
| # File: MODULE.bazel | |
| local_runtime_repo = use_repo_rule( | |
| "@rules_python//python/local_toolchains:repos.bzl", | |
| "local_runtime_repo", | |
| ) | |
| local_runtime_toolchains_repo = use_repo_rule( | |
| "@rules_python//python/local_toolchains:repos.bzl", | |
| "local_runtime_toolchains_repo", | |
| ) | |
| # Step 1: Define the Python runtime | |
| local_runtime_repo( | |
| name = "local_python3", | |
| interpreter_path = "python3", | |
| on_failure = "fail", | |
| dev_dependency = True | |
| ) | |
| # Step 2: Create toolchains for the runtimes | |
| local_runtime_toolchains_repo( | |
| name = "local_toolchains", | |
| runtimes = ["local_python3"], | |
| # TIP: The `target_settings` arg can be used to activate them based on | |
| # command line flags; see docs below. | |
| dev_dependency = True | |
| ) | |
| # Step 3: Register the toolchains | |
| register_toolchains("@local_toolchains//:all", dev_dependency = True) | |
| ``` | |
| In the example above, `interpreter_path` is used to find Python via `PATH` | |
| lookups. Alternatively, {obj}`interpreter_target` can be set, which can | |
| refer to a Python in an arbitrary Bazel repository. | |
| :::{important} | |
| Be sure to set `dev_dependency = True`. Using a local toolchain only makes sense | |
| for the root module. | |
| If an intermediate module does it, then the `register_toolchains()` call will | |
| take precedence over the default rules_python toolchains and cause problems for | |
| downstream modules. | |
| ::: | |
| Multiple runtimes and/or toolchains can be defined, which allows for multiple | |
| Python versions and/or platforms to be configured in a single `MODULE.bazel`. | |
| Note that `register_toolchains` will insert the local toolchain earlier in the | |
| toolchain ordering, so it will take precedence over other registered toolchains. | |
| To better control when the toolchain is used, see [Conditionally using local | |
| toolchains]. | |
| ### Conditionally using local toolchains | |
| By default, a local toolchain has few constraints and is early in the toolchain | |
| ordering, which means it will usually be used no matter what. This can be | |
| problematic for CI (where it shouldn't be used), expensive for CI (CI must | |
| initialize/download the repository to determine its Python version), and | |
| annoying for iterative development (enabling/disabling it requires modifying | |
| `MODULE.bazel`). | |
| These behaviors can be mitigated, but it requires additional configuration | |
| to avoid triggering the local toolchain repository to initialize (i.e., run | |
| local commands and perform downloads). | |
| The two settings to change are | |
| {obj}`local_runtime_toolchains_repo.target_compatible_with` and | |
| {obj}`local_runtime_toolchains_repo.target_settings`, which control how Bazel | |
| decides if a toolchain should match. By default, they point to targets *within* | |
| the local runtime repository (triggering repo initialization). We have to override | |
| them to *not* reference the local runtime repository at all. | |
| In the example below, we reconfigure the local toolchains so they are only | |
| activated if the custom flag `--//:py=local` is set and the target platform | |
| matches the Bazel host platform. The net effect is that CI won't use the local | |
| toolchain (nor initialize its repository), and developers can easily | |
| enable/disable the local toolchain with a command line flag. | |
| ``` | |
| # File: MODULE.bazel | |
| bazel_dep(name = "bazel_skylib", version = "1.7.1") | |
| local_runtime_toolchains_repo( | |
| name = "local_toolchains", | |
| runtimes = ["local_python3"], | |
| target_compatible_with = { | |
| "local_python3": ["HOST_CONSTRAINTS"], | |
| }, | |
| target_settings = { | |
| "local_python3": ["@//:is_py_local"] | |
| } | |
| ) | |
| # File: BUILD.bazel | |
| load("@bazel_skylib//rules:common_settings.bzl", "string_flag") | |
| config_setting( | |
| name = "is_py_local", | |
| flag_values = {":py": "local"}, | |
| ) | |
| string_flag( | |
| name = "py", | |
| build_setting_default = "", | |
| ) | |
| ``` | |
| :::{tip} | |
| Easily switching between *multiple* local toolchains can be accomplished by | |
| adding additional `:is_py_X` targets and setting `--//:py` to match. | |
| to easily switch between different local toolchains. | |
| ::: | |
| ## Runtime environment toolchain | |
| The runtime environment toolchain is a minimal toolchain that doesn't provide | |
| information about Python at build time. In particular, this means it is not able | |
| to build C extensions -- doing so requires knowing, at build time, what Python | |
| headers to use. | |
| In effect, all it does is generate a small wrapper script that simply calls, e.g., | |
| `/usr/bin/env python3` to run a program. This makes it easy to change what | |
| Python is used to run a program but also makes it easy to use a Python version | |
| that isn't compatible with build-time assumptions. | |
| ``` | |
| `register_toolchains`("@rules_python//python/runtime_env_toolchains:all") | |
| ``` | |
| Note that this toolchain has no constraints, i.e. it will match any platform, | |
| Python version, etc. | |
| :::{seealso} | |
| [Local `toolchain`], which creates a more full featured toolchain from a | |
| locally installed Python. | |
| ::: | |
| ### Autodetecting toolchain | |
| The autodetecting toolchain is a deprecated toolchain that is built into Bazel. | |
| **Its name is a bit misleading: it doesn't autodetect anything.** All it does is | |
| use `python3` from the environment a binary runs within. This provides extremely | |
| limited functionality to the rules (at build time, nothing is knowable about | |
| the Python runtime). | |
| Bazel itself automatically registers `@bazel_tools//tools/python:autodetecting_toolchain` | |
| as the lowest priority toolchain. For `WORKSPACE` builds, if no other toolchain | |
| is registered, that toolchain will be used. For Bzlmod builds, `rules_python` | |
| automatically registers a higher-priority toolchain; it won't be used unless | |
| there is a toolchain misconfiguration somewhere. | |
| To aid migration off the Bazel-builtin toolchain, `rules_python` provides | |
| {bzl:obj}`@rules_python//python/runtime_env_toolchains:all`. This is an equivalent | |
| toolchain but is implemented using `rules_python`'s objects. | |
| ## Custom toolchains | |
| While `rules_python` provides toolchains by default, it is not required to use | |
| them, and you can define your own toolchains to use instead. This section | |
| gives an introduction to how to define them yourself. | |
| :::{note} | |
| * Defining your own toolchains is an advanced feature. | |
| * APIs used for defining them are less stable and may change more often. | |
| ::: | |
| Under the hood, there are multiple toolchains that comprise the different | |
| information necessary to build Python targets. Each one has an | |
| associated _toolchain type_ that identifies it. We call the collection of these | |
| toolchains a "toolchain suite". | |
| One of the underlying design goals of the toolchains is to support complex and | |
| bespoke environments. Such environments may use an arbitrary combination of | |
| {bzl:obj}`RBE`, cross-platform building, multiple Python versions, | |
| building Python from source, embedding Python (as opposed to building separate | |
| interpreters), using prebuilt binaries, or using binaries built from source. To | |
| that end, many of the attributes they accept, and fields they provide, are | |
| optional. | |
| ### Target toolchain type | |
| The target toolchain type is {obj}`//python:toolchain_type`, and it | |
| is for _target configuration_ runtime information, e.g., the Python version | |
| and interpreter binary that a program will use. | |
| This is typically implemented using {obj}`py_runtime()`, which | |
| provides the {obj}`PyRuntimeInfo` provider. For historical reasons from the | |
| Python 2 transition, `py_runtime` is wrapped in {obj}`py_runtime_pair`, | |
| which provides {obj}`ToolchainInfo` with the field `py3_runtime`, which is an | |
| instance of `PyRuntimeInfo`. | |
| This toolchain type is intended to hold only _target configuration_ values. As | |
| such, when defining its associated {external:bzl:obj}`toolchain` target, only | |
| set {external:bzl:obj}`toolchain.target_compatible_with` and/or | |
| {external:bzl:obj}`toolchain.target_settings` constraints; there is no need to | |
| set {external:bzl:obj}`toolchain.exec_compatible_with`. | |
| ### Python C toolchain type | |
| The Python C toolchain type ("py cc") is {obj}`//python/cc:toolchain_type`, and | |
| it has C/C++ information for the _target configuration_, e.g., the C headers that | |
| provide `Python.h`. | |
| This is typically implemented using {obj}`py_cc_toolchain()`, which provides | |
| {obj}`ToolchainInfo` with the field `py_cc_toolchain` set, which is a | |
| {obj}`PyCcToolchainInfo` provider instance. | |
| This toolchain type is intended to hold only _target configuration_ values | |
| relating to the C/C++ information for the Python runtime. As such, when defining | |
| its associated {external:obj}`toolchain` target, only set | |
| {external:bzl:obj}`toolchain.target_compatible_with` and/or | |
| {external:bzl:obj}`toolchain.target_settings` constraints; there is no need to | |
| set {external:bzl:obj}`toolchain.exec_compatible_with`. | |
| ### Exec tools toolchain type | |
| The exec tools toolchain type is {obj}`//python:exec_tools_toolchain_type`, | |
| and it is for supporting tools for _building_ programs, e.g., the binary to | |
| precompile code at build time. | |
| This toolchain type is intended to hold only _exec configuration_ values -- | |
| usually tools (prebuilt or from-source) used to build Python targets. | |
| This is typically implemented using {obj}`py_exec_tools_toolchain`, which | |
| provides {obj}`ToolchainInfo` with the field `exec_tools` set, which is an | |
| instance of {obj}`PyExecToolsInfo`. | |
| The toolchain constraints of this toolchain type can be a bit more nuanced than | |
| the other toolchain types. Typically, you set | |
| {external:bzl:obj}`toolchain.target_settings` to the Python version the tools | |
| are for, and {external:bzl:obj}`toolchain.exec_compatible_with` to the platform | |
| they can run on. This allows the toolchain to first be considered based on the | |
| target configuration (e.g. Python version), then for one to be chosen based on | |
| finding one compatible with the available host platforms to run the tool on. | |
| However, what `target_compatible_with`/`target_settings` and | |
| `exec_compatible_with` values to use depends on the details of the tools being used. | |
| For example: | |
| * If you had a precompiler that supported any version of Python, then | |
| putting the Python version in `target_settings` is unnecessary. | |
| * If you had a prebuilt polyglot precompiler binary that could run on any | |
| platform, then setting `exec_compatible_with` is unnecessary. | |
| This can work because, when the rules invoke these build tools, they pass along | |
| all necessary information so that the tool can be entirely independent of the | |
| target configuration being built for. | |
| Alternatively, if you had a precompiler that only ran on Linux and only | |
| produced valid output for programs intended to run on Linux, then _both_ | |
| `exec_compatible_with` and `target_compatible_with` must be set to Linux. | |
| ### Custom toolchain example | |
| Here, we show an example for a semi-complicated toolchain suite, one that is: | |
| * A CPython-based interpreter | |
| * For Python version 3.12.0 | |
| * Using an in-build interpreter built from source | |
| * That only runs on Linux | |
| * Using a prebuilt precompiler that only runs on Linux and only produces | |
| bytecode valid for 3.12 | |
| * With the exec tools interpreter disabled (unnecessary with a prebuilt | |
| precompiler) | |
| * Providing C headers and libraries | |
| Defining toolchains for this might look something like this: | |
| ``` | |
| # ------------------------------------------------------- | |
| # File: toolchain_impl/BUILD | |
| # Contains the tool definitions (runtime, headers, libs). | |
| # ------------------------------------------------------- | |
| load("@rules_python//python:py_cc_toolchain.bzl", "py_cc_toolchain") | |
| load("@rules_python//python:py_exec_tools_toolchain.bzl", "py_exec_tools_toolchain") | |
| load("@rules_python//python:py_runtime.bzl", "py_runtime") | |
| load("@rules_python//python:py_runtime_pair.bzl", "py_runtime_pair") | |
| MAJOR = 3 | |
| MINOR = 12 | |
| MICRO = 0 | |
| py_runtime( | |
| name = "runtime", | |
| interpreter = ":python", | |
| interpreter_version_info = { | |
| "major": str(MAJOR), | |
| "minor": str(MINOR), | |
| "micro": str(MICRO), | |
| } | |
| implementation = "cpython" | |
| ) | |
| py_runtime_pair( | |
| name = "runtime_pair", | |
| py3_runtime = ":runtime" | |
| ) | |
| py_cc_toolchain( | |
| name = "py_cc_toolchain_impl", | |
| headers = ":headers", | |
| libs = ":libs", | |
| python_version = "{}.{}".format(MAJOR, MINOR) | |
| ) | |
| py_exec_tools_toolchain( | |
| name = "exec_tools_toolchain_impl", | |
| exec_interpreter = "@rules_python/python:none", | |
| precompiler = "precompiler-cpython-3.12" | |
| ) | |
| cc_binary(name = "python3.12", ...) | |
| cc_library(name = "headers", ...) | |
| cc_library(name = "libs", ...) | |
| # ------------------------------------------------------------------ | |
| # File: toolchains/BUILD | |
| # Putting toolchain() calls in a separate package from the toolchain | |
| # implementations minimizes Bazel loading overhead. | |
| # ------------------------------------------------------------------ | |
| toolchain( | |
| name = "runtime_toolchain", | |
| toolchain = "//toolchain_impl:runtime_pair", | |
| toolchain_type = "@rules_python//python:toolchain_type", | |
| target_compatible_with = ["@platforms/os:linux"], | |
| ) | |
| toolchain( | |
| name = "py_cc_toolchain", | |
| toolchain = "//toolchain_impl:py_cc_toolchain_impl", | |
| toolchain_type = "@rules_python//python/cc:toolchain_type", | |
| target_compatible_with = ["@platforms/os:linux"], | |
| ) | |
| toolchain( | |
| name = "exec_tools_toolchain", | |
| toolchain = "//toolchain_impl:exec_tools_toolchain_impl", | |
| toolchain_type = "@rules_python//python:exec_tools_toolchain_type", | |
| target_settings = [ | |
| "@rules_python//python/config_settings:is_python_3.12", | |
| ], | |
| exec_compatible_with = ["@platforms/os:linux"], | |
| ) | |
| # ----------------------------------------------- | |
| # File: MODULE.bazel or WORKSPACE.bazel | |
| # These toolchains will be considered before others. | |
| # ----------------------------------------------- | |
| register_toolchains("//toolchains:all") | |
| ``` | |
| When registering custom toolchains, be aware of the [toolchain registration | |
| order](https://bazel.build/extending/toolchains#toolchain-resolution). In brief, | |
| toolchain order is the BFS-order of the modules; see the Bazel docs for a more | |
| detailed description. | |
| :::{note} | |
| The toolchain() calls should be in a separate BUILD file from everything else. | |
| This avoids Bazel having to perform unnecessary work when it discovers the list | |
| of available toolchains. | |
| ::: | |
| ## Toolchain selection flags | |
| Currently the following flags are used to influence toolchain selection: | |
| * {obj}`--@rules_python//python/config_settings:py_linux_libc` for selecting the Linux libc variant. | |
| * {obj}`--@rules_python//python/config_settings:py_freethreaded` for selecting | |
| the freethreaded experimental Python builds available from `3.13.0` onwards. | |
| * {obj}`--@rules_python//python/config_settings:py_pbs_distribution` | |
| for selecting which python-build-standalone archive to use, e.g. stripped | |
| runtimes. | |
| ## Running the underlying interpreter | |
| To run the interpreter that Bazel will use, you can use the | |
| `@rules_python//python/bin:python` target. This is a binary target with | |
| the executable pointing at the `python3` binary plus its relevant runfiles. | |
| ```console | |
| $ bazel run @rules_python//python/bin:python | |
| Python 3.14.4 (main, Jan 16 2026, 22:41:20) [Clang 15.0.7 ] on linux | |
| Type "help", "copyright", "credits" or "license" for more information. | |
| >>> | |
| $ bazel run @rules_python//python/bin:python --@rules_python//python/config_settings:python_version=3.12 | |
| Python 3.12.0 (main, Oct 3 2023, 01:27:23) [Clang 17.0.1 ] on linux | |
| Type "help", "copyright", "credits" or "license" for more information. | |
| >>> | |
| ``` | |
| You can also access a specific binary's interpreter this way by using the | |
| `@rules_python//python/bin:python_src` target. In the example below, it is | |
| assumed that the `@rules_python//tools/publish:twine` binary is fixed at Python | |
| 3.14. | |
| ```console | |
| $ bazel run @rules_python//python/bin:python --@rules_python//python/bin:interpreter_src=@rules_python//tools/publish:twine | |
| Python 3.14.4 (main, Jan 16 2026, 22:41:20) [Clang 15.0.7 ] on linux | |
| Type "help", "copyright", "credits" or "license" for more information. | |
| >>> | |
| $ bazel run @rules_python//python/bin:python --@rules_python//python/bin:interpreter_src=@rules_python//tools/publish:twine --@rules_python//python/config_settings:python_version=3.12 | |
| Python 3.14.4 (main, Jan 16 2026, 22:41:20) [Clang 15.0.7 ] on linux | |
| Type "help", "copyright", "credits" or "license" for more information. | |
| >>> | |
| ``` | |
| Despite setting the Python version explicitly to 3.12 in the example above, the | |
| interpreter comes from the `@rules_python//tools/publish:twine` binary. That is | |
| a fixed version. | |
| :::{note} | |
| The `python` target does not provide access to any modules from `py_*` | |
| targets on its own. Please file a feature request if this is desired. | |
| ::: | |
| ### Differences from `//python/bin:repl` | |
| The `//python/bin:python` target provides access to the underlying interpreter | |
| without any hermeticity guarantees. | |
| The [`//python/bin:repl` target](repl) provides an environment identical to | |
| what `py_binary` provides. That means it handles things like the | |
| [`PYTHONSAFEPATH`](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSAFEPATH) | |
| environment variable automatically. The `//python/bin:python` target will not. | |
| ## Consuming Python C headers and libraries | |
| The following targets expose the headers and libraries from the | |
| currently selected Python C toolchain: | |
| - {obj}`@rules_python//python/cc:current_py_cc_headers` | |
| - {obj}`@rules_python//python/cc:current_py_cc_headers_abi3` | |
| - {obj}`@rules_python//python/cc:current_py_cc_libs` | |
| These targets behave similarly to a `cc_library`, but instead of defining | |
| their own sources, they forward providers from the underlying toolchain- | |
| selected `cc_library`. | |
| A Python C toolchain must be registered for these targets to work. | |
| Under bzlmod, a toolchain is registered automatically. In non-bzlmod | |
| setups, users must ensure that a toolchain is explicitly registered. | |
| Users should depend on these targets instead of legacy alias targets | |
| when embedding Python or building C extensions, as this ensures | |
| compatibility across different toolchain configurations. | |
| :::{seealso} | |
| The _How to get Python headres for C extensions_ how-to guide, and the | |
| {obj}`//python/cc:BUILD.bazel` package API documentation. | |
| ::: |