Maprule that runs a Bash command.
This rule is the same as cmd_maprule
except this one uses Bash to run the command, therefore the cmd
attribute must use Bash syntax. bash_maprule
rules can only be built if Bash is installed on the build machine.
Maprule runs a specific command for each of the “foreach” source files. This allows processing source files in parallel, to produce some outputs for each of them.
The name “maprule” indicates that this rule can be used to map source files to output files, and is also a reference to “genrule” that inspired this rule's design (though there are significant differences).
Below you will find an Example, Tips and Tricks, and an FAQ.
# This file is //projects/game:BUILD load("@bazel_skylib//rules:maprule.bzl", "bash_maprule") bash_maprule( name = "process_assets", foreach_srcs = [ "rust.png", "teapot.3ds", "//assets:wood.jpg", "//assets:models", ], outs_templates = { "TAGS": "{src}.tags", "DIGEST": "digests/{src_name_noext}.md5", }, tools = [ "//bin:asset-tagger", "//util:md5sum", ], add_env = { "ASSET_TAGGER": "$(location //bin:asset-tagger)", "MD5SUM": "$(location //util:md5sum)", }, # Note: this command should live in a script file, we only inline it in the `cmd` attribute # for the sake of demonstration. See Tips and Tricks section. cmd = '"$MAPRULE_ASSET_TAGGER" --input="$MAPRULE_SRC" --output="$MAPRULE_TAGS" && ' + '"$MAPRULE_MD5SUM" "$MAPRULE_SRC" > "$MAPRULE_DIGEST"', )
The “process_assets” rule above will execute the command in the cmd
to process “rust.png”, “teapot.3ds”, “//assets:wood.jpg”, and every file in the “//assets:models” rule, producing the corresponding .tags and .md5 files for each of them, under the following paths:
bazel-bin/projects/game/process_assets_out/projects/game/rust.png.tags bazel-bin/projects/game/process_assets_out/digests/rust.md5 bazel-bin/projects/game/process_assets_out/projects/game/teapot.3ds.tags bazel-bin/projects/game/process_assets_out/digests/teapot.md5 bazel-bin/projects/game/process_assets_out/assets/wood.jpg.tags bazel-bin/projects/game/process_assets_out/digests/wood.md5 ...
You can create downstream rules, for example a filegroup or genrule (or another maprule) that put this rule in their srcs
attribute and get all the .tags and .md5 files.
(The Tips and Tricks section is the same for cmd_maprule
and bash_maprule
.)
cmd
attribute.Unless the command is trivial, don't try to write it in cmd
.
Properly quoting parts of the command is challenging enough, add to that escaping for the BUILD file's syntax and the cmd
attribute quickly gets unmaintainably complex.
It's a lot easier and maintainable to create a script file such as “foo.sh” in bash_maprule
or “foo.bat” in cmd_maprule
for the commands. To do that:
tools
attributeadd_env
attribute, e.g. “TOOL: "$(location :tool.sh)"
”cmd
with just “$MAPRULE_FOO” (in bash_maprule
) or “%MAPRULE_FOO%” (in cmd_maprule
).Doing this also avoids hitting command line length limits.
Example:
cmd_maprule( ... srcs = ["//data:temperatures.txt"], tools = [ "command.bat", ":weather-stats", ], add_env = { "COMMAND": "$(location :command.bat)", "STATS_TOOL": "$(location :weather-stats-computer)", "TEMPERATURES": "$(location //data:temperatures.txt)", }, cmd = "%MAPRULE_COMMAND%", )
add_env
attribute to pass tool locations to the command.Entries in the add_env
attribute may use “$(location)” references and may also use the same placeholders as the outs_templates
attribute. For example you can use this mechanism to pass extra “$(location)” references of tools
or srcs
to the actions.
Example:
cmd_maprule( ... foreach_srcs = [...], outs_templates = {"STATS_OUT": "{src}-stats.txt"}, srcs = ["//data:temperatures.txt"], tools = [":weather-stats"], add_env = { "STATS_TOOL": "$(location :weather-stats-computer)", "TEMPERATURES": "$(location //data:temperatures.txt)", }, cmd = "%MAPRULE_STATS_TOOL% --src=%MAPRULE_SRC% --data=%MAPRULE_TEMPERATURES% > %MAPRULE_STATS_OUT%", ) cc_binary( name = "weather-stats", ... )
(The Environment Variables section is the same for cmd_maprule
and bash_maprule
.)
The rule defines several environment variables available to the command may reference. All of these envvars names start with “MAPRULE_”. You can add your own envvars with the add_env
attribute.
The command can use some envvars, all named “MAPRULE_<something>”.
The complete list of environment variables is:
foreach_srcs
srcs
attributeouts_templates
attribute, this is the path of the respective output file for the current sourceadd_env
attribute(The FAQ section is the same for cmd_maprule
and bash_maprule
.)
genrule creates a single action for all of its inputs. It requires specifying all output files. Finally, it can only create Bash commands.
Maprule supports parallel processing of its inputs and doesn't require specifying all outputs, just templates for the outputs. And not only Bash is supported (via bash_maprule
) but so are cmd.exe-style commands via cmd_maprule
.
genrule.cmd
supports “$(location)” expressions, how come *_maprule.cmd
doesn't?Maprule deliberately omits support for this feature to avoid pitfalls with quoting and escaping, and potential issues with paths containing spaces. Instead, maprule exports environment variables for the input and outputs of the action, and allows the user to define extra envvars. These extra envvars do support “$(location)” expressions, so you can pass paths of labels in srcs
and tools
.
To avoid conflicts with other envvars, whose names could also be attractive outs_templates names.
outs_templates
and add_env
keys have to be uppercase?Because they are exported as all-uppercase envvars, so requiring that they be declared as uppercase gives a visual cue of that. It also avoids clashes resulting from mixed lower-upper-case names like “foo” and “Foo”.
outs_templates
and add_env
keys have to start with “MAPRULE_” even though they are exported as such?For convenience. It seems to bring no benefit to have the user always type “MAPRULE_” in front of the name when the rule itself could as well add it.
To ensure that maprules in the same package and with the same outs_templates produce distinct output files.
outs_templates
keys available as placeholders in the values of add_env
?Because add_env
is meant to be used for passing extra “$(location)” information to the action, and the output paths are already available as envvars for the action.
Maprule that runs a Windows Command Prompt (cmd.exe
) command.
This rule is the same as bash_maprule
, but uses cmd.exe
instead of Bash, therefore the cmd
attribute must use Command Prompt syntax. cmd_maprule
rules can only be built on Windows.
Maprule runs a specific command for each of the “foreach” source files. This allows processing source files in parallel, to produce some outputs for each of them.
The name “maprule” indicates that this rule can be used to map source files to output files, and is also a reference to “genrule” that inspired this rule's design (though there are significant differences).
Below you will find an Example, Tips and Tricks, and an FAQ.
# This file is //projects/game:BUILD load("@bazel_skylib//rules:maprule.bzl", "cmd_maprule") cmd_maprule( name = "process_assets", foreach_srcs = [ "rust.png", "teapot.3ds", "//assets:wood.jpg", "//assets:models", ], outs_templates = { "TAGS": "{src}.tags", "DIGEST": "digests/{src_name_noext}.md5", }, tools = [ "//bin:asset-tagger", "//util:md5sum", ], add_env = { "ASSET_TAGGER": "$(location //bin:asset-tagger)", "MD5SUM": "$(location //util:md5sum)", }, # Note: this command should live in a script file, we only inline it in the `cmd` attribute # for the sake of demonstration. See Tips and Tricks section. cmd = "%MAPRULE_ASSET_TAGGER% --input=%MAPRULE_SRC% --output=%MAPRULE_TAGS% & " + 'IF /I "%ERRORLEVEL%" EQU "0" ( %MAPRULE_MD5SUM% %MAPRULE_SRC% > %MAPRULE_DIGEST% )', )
The “process_assets” rule above will execute the command in the cmd
to process “rust.png”, “teapot.3ds”, “//assets:wood.jpg”, and every file in the “//assets:models” rule, producing the corresponding .tags and .md5 files for each of them, under the following paths:
bazel-bin/projects/game/process_assets_out/projects/game/rust.png.tags bazel-bin/projects/game/process_assets_out/digests/rust.md5 bazel-bin/projects/game/process_assets_out/projects/game/teapot.3ds.tags bazel-bin/projects/game/process_assets_out/digests/teapot.md5 bazel-bin/projects/game/process_assets_out/assets/wood.jpg.tags bazel-bin/projects/game/process_assets_out/digests/wood.md5 ...
You can create downstream rules, for example a filegroup or genrule (or another maprule) that put this rule in their srcs
attribute and get all the .tags and .md5 files.
(The Tips and Tricks section is the same for cmd_maprule
and bash_maprule
.)
cmd
attribute.Unless the command is trivial, don't try to write it in cmd
.
Properly quoting parts of the command is challenging enough, add to that escaping for the BUILD file's syntax and the cmd
attribute quickly gets unmaintainably complex.
It's a lot easier and maintainable to create a script file such as “foo.sh” in bash_maprule
or “foo.bat” in cmd_maprule
for the commands. To do that:
tools
attributeadd_env
attribute, e.g. “TOOL: "$(location :tool.sh)"
”cmd
with just “$MAPRULE_FOO” (in bash_maprule
) or “%MAPRULE_FOO%” (in cmd_maprule
).Doing this also avoids hitting command line length limits.
Example:
cmd_maprule( ... srcs = ["//data:temperatures.txt"], tools = [ "command.bat", ":weather-stats", ], add_env = { "COMMAND": "$(location :command.bat)", "STATS_TOOL": "$(location :weather-stats-computer)", "TEMPERATURES": "$(location //data:temperatures.txt)", }, cmd = "%MAPRULE_COMMAND%", )
add_env
attribute to pass tool locations to the command.Entries in the add_env
attribute may use “$(location)” references and may also use the same placeholders as the outs_templates
attribute. For example you can use this mechanism to pass extra “$(location)” references of tools
or srcs
to the actions.
Example:
cmd_maprule( ... foreach_srcs = [...], outs_templates = {"STATS_OUT": "{src}-stats.txt"}, srcs = ["//data:temperatures.txt"], tools = [":weather-stats"], add_env = { "STATS_TOOL": "$(location :weather-stats-computer)", "TEMPERATURES": "$(location //data:temperatures.txt)", }, cmd = "%MAPRULE_STATS_TOOL% --src=%MAPRULE_SRC% --data=%MAPRULE_TEMPERATURES% > %MAPRULE_STATS_OUT%", ) cc_binary( name = "weather-stats", ... )
(The Environment Variables section is the same for cmd_maprule
and bash_maprule
.)
The rule defines several environment variables available to the command may reference. All of these envvars names start with “MAPRULE_”. You can add your own envvars with the add_env
attribute.
The command can use some envvars, all named “MAPRULE_<something>”.
The complete list of environment variables is:
foreach_srcs
srcs
attributeouts_templates
attribute, this is the path of the respective output file for the current sourceadd_env
attribute(The FAQ section is the same for cmd_maprule
and bash_maprule
.)
genrule creates a single action for all of its inputs. It requires specifying all output files. Finally, it can only create Bash commands.
Maprule supports parallel processing of its inputs and doesn't require specifying all outputs, just templates for the outputs. And not only Bash is supported (via bash_maprule
) but so are cmd.exe-style commands via cmd_maprule
.
genrule.cmd
supports “$(location)” expressions, how come *_maprule.cmd
doesn't?Maprule deliberately omits support for this feature to avoid pitfalls with quoting and escaping, and potential issues with paths containing spaces. Instead, maprule exports environment variables for the input and outputs of the action, and allows the user to define extra envvars. These extra envvars do support “$(location)” expressions, so you can pass paths of labels in srcs
and tools
.
To avoid conflicts with other envvars, whose names could also be attractive outs_templates names.
outs_templates
and add_env
keys have to be uppercase?Because they are exported as all-uppercase envvars, so requiring that they be declared as uppercase gives a visual cue of that. It also avoids clashes resulting from mixed lower-upper-case names like “foo” and “Foo”.
outs_templates
and add_env
keys have to start with “MAPRULE_” even though they are exported as such?For convenience. It seems to bring no benefit to have the user always type “MAPRULE_” in front of the name when the rule itself could as well add it.
To ensure that maprules in the same package and with the same outs_templates produce distinct output files.
outs_templates
keys available as placeholders in the values of add_env
?Because add_env
is meant to be used for passing extra “$(location)” information to the action, and the output paths are already available as envvars for the action.