Target Archive Tool

For a specific Vulkan device, the target-archive compile-time tool packages run-time discoverable configuration data and SPIR-V modules into a single read-only binary blob (a “target archive”) that can be loaded at run time and passed to a Vulkan library at library instantiation time.

The benefit of a “target archive” is that the Vulkan library is decoupled from any device-specific details, shader source changes, or GLSL toolchain changes.

As a result, there is no runtime compilation of GLSL or introspection of SPIR-V modules.

Background

Certain Fuchsia Vulkan compute libraries require creation of device-specific SPIR-V modules because architectural differences like subgroup width, register file depth, shared memory size, and vendor extensions determine the “shape” and composition of the GLSL source.

Instead of writing custom shaders for each device, the library has a single set of shaders that are compiled with device-specific settings and switches.

Some of the compile-time configuration information must also be discoverable at library instantiation time.

Implementation

There is a generic “base” GN target for building C sources and GLSL shaders and bundling them into a binary “archive”. See: compute/tools/target_archive/gn/target_archive.gni.

A library-specific GN target should be created that extends the base GN target and possibly hides any implicit steps, sources or configuration data. For example, see: compute/radix_sort/platforms/vk/targets/gn/radix_sort_vk_target.gni.

All runtime configuration data required by a Vulkan target archive is packaged into a C header.

The header, any other binaries and all SPIR-V modules are then archived into an easily decodable format.

The base GN target then performs the following steps:

StepInputsOutputs
Compile “structified” runtime header.c source file.o object file
Dump the object file's .rodata.o object file.bin binary data
Compile all shaders with the configuration dataGLSL shaders.spv SPIR-V modules
Package binary and SPIR-V modulesbinary, modules.ar target archive file,
.S linkable object,
.h linkable symbol

The result is a device and configuration-specific pure data payload that can be included in a component as a file and opened at runtime or ephemerally loaded through some other mechanism.

Additionally, a linkable object and header file are available for applications that wish to link the target archive as read-only data.

Usage

To apply this to your own library take the following steps:

  • Determine what shaders are in your library as well as their arch-specific configuration data.

  • Determine what configuration data needs to be known at runtime when the selected target archive is handed to your library.

  • Consider creating your own GN target that extends the base target_archive.gni and hides implicit dependencies.

    An example of extending target_archive.gni is here: compute/radix_sort/platforms/vk/targets/gn/radix_sort_vk_target.gni.

  • Come up with a directory hierarchy for each device-specific target archive and drop a BUILD.gn in each.

  • Determine whether you want to load or link the target archive(s).

  • Add all required device-specific GN targets[^1] to your Fuchsia application.

  • Fuchsia loadable target archive files will be found in the component's pkg/data/targets directory.

  • A linkable target archive's include file looks similar to this:

#pragma once

#include "target_archive/target_archive.h"

#ifdef __cplusplus
extern "C" {
#endif

extern struct target_archive_header const my_gn_target_name[];

#ifdef __cplusplus
}
#endif

[^1] GN also uses the word “target” so don't get confused!

Internals

The target-archive executable tool concatenates one or more binaries prefixed by a table containing the number of binaries and the offset and size of each binary.

  • Offsets are relative to the end of the entries[] table.
  • Offsets and sizes are in bytes.
  • Offsets and sizes are 64-bit.
  • Binaries and their offsets are 4-byte aligned.

This format enables type-safe loading of SPIR-V modules and creation of Vulkan pipelines.

The target archive memory map is:

   +-----------------------------------------+ 0
   | alignas(8) struct target_archive_header |
   +-----------------------------------------+ 8
   | struct target_archive_entry[0]          |
   | struct target_archive_entry[1]          |
   | ...                                     |
   | struct target_archive_entry[count-1]    |
   +-----------------------------------------+ 8 + 16 * count
   | alignas(4) data_(0)                     |
   | alignas(4) data_(1)                     |
   | ...                                     |
   | alignas(4) data_(count-1)               |
   +-----------------------------------------+