pipelining: add the ability to disable pipelining for a single rule. (#1499)
While testing internally we found some non-hermetic libraries have
issues with pipelining enabled, adding a per-rule switch allows us to
keep the feature generally enabled while disabling problematic targets.
diff --git a/docs/defs.md b/docs/defs.md
index 7228d32..b0c1898 100644
--- a/docs/defs.md
+++ b/docs/defs.md
@@ -214,7 +214,8 @@
<pre>
rust_library(<a href="#rust_library-name">name</a>, <a href="#rust_library-aliases">aliases</a>, <a href="#rust_library-compile_data">compile_data</a>, <a href="#rust_library-crate_features">crate_features</a>, <a href="#rust_library-crate_name">crate_name</a>, <a href="#rust_library-crate_root">crate_root</a>, <a href="#rust_library-data">data</a>, <a href="#rust_library-deps">deps</a>,
- <a href="#rust_library-edition">edition</a>, <a href="#rust_library-proc_macro_deps">proc_macro_deps</a>, <a href="#rust_library-rustc_env">rustc_env</a>, <a href="#rust_library-rustc_env_files">rustc_env_files</a>, <a href="#rust_library-rustc_flags">rustc_flags</a>, <a href="#rust_library-srcs">srcs</a>, <a href="#rust_library-stamp">stamp</a>, <a href="#rust_library-version">version</a>)
+ <a href="#rust_library-disable_pipelining">disable_pipelining</a>, <a href="#rust_library-edition">edition</a>, <a href="#rust_library-proc_macro_deps">proc_macro_deps</a>, <a href="#rust_library-rustc_env">rustc_env</a>, <a href="#rust_library-rustc_env_files">rustc_env_files</a>, <a href="#rust_library-rustc_flags">rustc_flags</a>,
+ <a href="#rust_library-srcs">srcs</a>, <a href="#rust_library-stamp">stamp</a>, <a href="#rust_library-version">version</a>)
</pre>
Builds a Rust library crate.
@@ -294,6 +295,7 @@
| <a id="rust_library-crate_root"></a>crate_root | The file that will be passed to <code>rustc</code> to be used for building this crate.<br><br>If <code>crate_root</code> is not set, then this rule will look for a <code>lib.rs</code> file (or <code>main.rs</code> for rust_binary) or the single file in <code>srcs</code> if <code>srcs</code> contains only one file. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
| <a id="rust_library-data"></a>data | List of files used by this rule at compile time and runtime.<br><br>If including data at compile time with include_str!() and similar, prefer <code>compile_data</code> over <code>data</code>, to prevent the data also being included in the runfiles. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [] |
| <a id="rust_library-deps"></a>deps | List of other libraries to be linked to this library target.<br><br>These can be either other <code>rust_library</code> targets or <code>cc_library</code> targets if linking a native library. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [] |
+| <a id="rust_library-disable_pipelining"></a>disable_pipelining | Disables pipelining for this rule if it is globally enabled. This will cause this rule to not produce a <code>.rmeta</code> file and all the dependent crates will instead use the <code>.rlib</code> file. | Boolean | optional | False |
| <a id="rust_library-edition"></a>edition | The rust edition to use for this crate. Defaults to the edition specified in the rust_toolchain. | String | optional | "" |
| <a id="rust_library-proc_macro_deps"></a>proc_macro_deps | List of <code>rust_library</code> targets with kind <code>proc-macro</code> used to help build this library target. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [] |
| <a id="rust_library-rustc_env"></a>rustc_env | Dictionary of additional <code>"key": "value"</code> environment variables to set for rustc.<br><br>rust_test()/rust_binary() rules can use $(rootpath //package:target) to pass in the location of a generated file or external tool. Cargo build scripts that wish to expand locations should use cargo_build_script()'s build_script_env argument instead, as build scripts are run in a different environment - see cargo_build_script()'s documentation for more. | <a href="https://bazel.build/docs/skylark/lib/dict.html">Dictionary: String -> String</a> | optional | {} |
diff --git a/docs/flatten.md b/docs/flatten.md
index 87af5c5..c3324b9 100644
--- a/docs/flatten.md
+++ b/docs/flatten.md
@@ -611,7 +611,8 @@
<pre>
rust_library(<a href="#rust_library-name">name</a>, <a href="#rust_library-aliases">aliases</a>, <a href="#rust_library-compile_data">compile_data</a>, <a href="#rust_library-crate_features">crate_features</a>, <a href="#rust_library-crate_name">crate_name</a>, <a href="#rust_library-crate_root">crate_root</a>, <a href="#rust_library-data">data</a>, <a href="#rust_library-deps">deps</a>,
- <a href="#rust_library-edition">edition</a>, <a href="#rust_library-proc_macro_deps">proc_macro_deps</a>, <a href="#rust_library-rustc_env">rustc_env</a>, <a href="#rust_library-rustc_env_files">rustc_env_files</a>, <a href="#rust_library-rustc_flags">rustc_flags</a>, <a href="#rust_library-srcs">srcs</a>, <a href="#rust_library-stamp">stamp</a>, <a href="#rust_library-version">version</a>)
+ <a href="#rust_library-disable_pipelining">disable_pipelining</a>, <a href="#rust_library-edition">edition</a>, <a href="#rust_library-proc_macro_deps">proc_macro_deps</a>, <a href="#rust_library-rustc_env">rustc_env</a>, <a href="#rust_library-rustc_env_files">rustc_env_files</a>, <a href="#rust_library-rustc_flags">rustc_flags</a>,
+ <a href="#rust_library-srcs">srcs</a>, <a href="#rust_library-stamp">stamp</a>, <a href="#rust_library-version">version</a>)
</pre>
Builds a Rust library crate.
@@ -691,6 +692,7 @@
| <a id="rust_library-crate_root"></a>crate_root | The file that will be passed to <code>rustc</code> to be used for building this crate.<br><br>If <code>crate_root</code> is not set, then this rule will look for a <code>lib.rs</code> file (or <code>main.rs</code> for rust_binary) or the single file in <code>srcs</code> if <code>srcs</code> contains only one file. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
| <a id="rust_library-data"></a>data | List of files used by this rule at compile time and runtime.<br><br>If including data at compile time with include_str!() and similar, prefer <code>compile_data</code> over <code>data</code>, to prevent the data also being included in the runfiles. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [] |
| <a id="rust_library-deps"></a>deps | List of other libraries to be linked to this library target.<br><br>These can be either other <code>rust_library</code> targets or <code>cc_library</code> targets if linking a native library. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [] |
+| <a id="rust_library-disable_pipelining"></a>disable_pipelining | Disables pipelining for this rule if it is globally enabled. This will cause this rule to not produce a <code>.rmeta</code> file and all the dependent crates will instead use the <code>.rlib</code> file. | Boolean | optional | False |
| <a id="rust_library-edition"></a>edition | The rust edition to use for this crate. Defaults to the edition specified in the rust_toolchain. | String | optional | "" |
| <a id="rust_library-proc_macro_deps"></a>proc_macro_deps | List of <code>rust_library</code> targets with kind <code>proc-macro</code> used to help build this library target. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [] |
| <a id="rust_library-rustc_env"></a>rustc_env | Dictionary of additional <code>"key": "value"</code> environment variables to set for rustc.<br><br>rust_test()/rust_binary() rules can use $(rootpath //package:target) to pass in the location of a generated file or external tool. Cargo build scripts that wish to expand locations should use cargo_build_script()'s build_script_env argument instead, as build scripts are run in a different environment - see cargo_build_script()'s documentation for more. | <a href="https://bazel.build/docs/skylark/lib/dict.html">Dictionary: String -> String</a> | optional | {} |
diff --git a/rust/private/rust.bzl b/rust/private/rust.bzl
index c6361bd..25e56bf 100644
--- a/rust/private/rust.bzl
+++ b/rust/private/rust.bzl
@@ -318,7 +318,7 @@
rust_lib = ctx.actions.declare_file(rust_lib_name)
rust_metadata = None
- if can_build_metadata(toolchain, ctx, crate_type):
+ if can_build_metadata(toolchain, ctx, crate_type) and not ctx.attr.disable_pipelining:
rust_metadata = ctx.actions.declare_file(
paths.replace_extension(rust_lib_name, ".rmeta"),
sibling = rust_lib,
@@ -764,7 +764,16 @@
rust_library = rule(
implementation = _rust_library_impl,
provides = _common_providers,
- attrs = dict(_common_attrs.items()),
+ attrs = dict(_common_attrs.items() + {
+ "disable_pipelining": attr.bool(
+ default = False,
+ doc = dedent("""\
+ Disables pipelining for this rule if it is globally enabled.
+ This will cause this rule to not produce a `.rmeta` file and all the dependent
+ crates will instead use the `.rlib` file.
+ """),
+ ),
+ }.items()),
fragments = ["cpp"],
host_fragments = ["cpp"],
toolchains = [
diff --git a/test/unit/pipelined_compilation/pipelined_compilation_test.bzl b/test/unit/pipelined_compilation/pipelined_compilation_test.bzl
index 93382c5..71f538c 100644
--- a/test/unit/pipelined_compilation/pipelined_compilation_test.bzl
+++ b/test/unit/pipelined_compilation/pipelined_compilation_test.bzl
@@ -175,6 +175,30 @@
rmeta_is_propagated_through_custom_rule_test = analysistest.make(_rmeta_is_propagated_through_custom_rule_test_impl, attrs = {"generate_metadata": attr.bool()}, config_settings = ENABLE_PIPELINING)
rmeta_is_used_when_building_custom_rule_test = analysistest.make(_rmeta_is_used_when_building_custom_rule_test_impl, config_settings = ENABLE_PIPELINING)
+def _rmeta_not_produced_if_pipelining_disabled_test_impl(ctx):
+ env = analysistest.begin(ctx)
+ tut = analysistest.target_under_test(env)
+
+ rust_action = [act for act in tut.actions if act.mnemonic == "RustcMetadata"]
+ asserts.true(env, len(rust_action) == 0, "expected no metadata to be produced, but found a metadata action")
+
+ return analysistest.end(env)
+
+rmeta_not_produced_if_pipelining_disabled_test = analysistest.make(_rmeta_not_produced_if_pipelining_disabled_test_impl, config_settings = ENABLE_PIPELINING)
+
+def _disable_pipelining_test():
+ rust_library(
+ name = "lib",
+ srcs = ["custom_rule_test/to_wrap.rs"],
+ edition = "2021",
+ disable_pipelining = True,
+ )
+ rmeta_not_produced_if_pipelining_disabled_test(
+ name = "rmeta_not_produced_if_pipelining_disabled_test",
+ target_compatible_with = NOT_WINDOWS,
+ target_under_test = ":lib",
+ )
+
def _custom_rule_test(generate_metadata, suffix):
rust_library(
name = "to_wrap" + suffix,
@@ -215,6 +239,7 @@
name: Name of the macro.
"""
_pipelined_compilation_test()
+ _disable_pipelining_test()
_custom_rule_test(generate_metadata = True, suffix = "_with_metadata")
_custom_rule_test(generate_metadata = False, suffix = "_without_metadata")
@@ -227,5 +252,6 @@
":rmeta_is_propagated_through_custom_rule_test_without_metadata",
":rmeta_is_used_when_building_custom_rule_test_with_metadata",
":rmeta_is_used_when_building_custom_rule_test_without_metadata",
+ ":rmeta_not_produced_if_pipelining_disabled_test",
],
)
