This document describes how to update or extend the Component Inspection File Format.
While extending the format, it is imperative that you don't break any existing functionality, in particular, the Inspect readers and validation tests. However, packing all changes into a single change can be overwhelming to review. In general, there should be 5-9 changes or steps in a chain to alter the VMO format:
View the type table in the Inspect file format and choose an available type number. There are a total of 256 possible types in the current specification.
In order to reserve the new type, update the Inspect file format.
It is cumbersome to test a reader or writer without the other and difficult to get the proper block-level API without both a reader and writer.
To test a reader and writer:
Pick a language and design and implement the feature entirely in that language, using unit tests to model actual usage of the API.
Split the changes into separate reader and writer changes, stacking the writer on top of the reader.
At this point, tests in the reader that rely on the writer API are probably broken in the reader change.
Rebase into that change and rewrite the tests (keeping around the original version of the tests) using lower level functionality.
Typically, you can put all changes to block-code in the reader change, making it possible but messy to write tests for the reader API.
The test will be ugly. Rebase into the writer change and remove the modified tests, replacing them with the original tests that are written in terms of the high level API.
Use the two changes for reference and duplicate them in the second language.
It‘s much easier now that you aren’t designing an API without actually using it. Ideally, you would also update the Dart library at this point. There is no Dart reader, so changes in the Rust/C++ writers won‘t break the Dart library; however, the ability to read a feature can’t be removed from the Rust and C++ libraries unless the Dart one is updated.
(Optional) Depending on the contents of your change, you may have to update the validator tests as you go, since changes to the existing format of existing blocks will likely break them.
The following sections outline how to put together each change, but in practice, take these as hints for designing the whole system cohesively before splitting the design as described above.
The examples in this section create a new type called MyFoo
.
fx set core.x64 --with //src/lib/diagnostics:tests
Run tests:
Note: The inspect-format-tests
are the VMO block format tests, and fuchsia-inspect-tests
are the core library tests.
fx test inspect-format-tests fuchsia-inspect-tests
If defining a new Block Type, update the BlockType
enum.
Note: This must be the same as the VMO format docs and the C++ implementation.
Update the methods and functions defined for BlockType
.
If changing the fields in an existing block or creating a new BlockType
, update the bitfield layout.
Run inspect-format-tests
to verify changes compile:
fx test inspect-format-tests
Update the block definition to include methods for reading and writing the new fields. It is appropriate to include write-functionality at this time in the block library; it is next to impossible to write a reader test without it.
Note: Use #[cfg(test)]
if the function is currently test-only and only used within the current crate. Otherwise, use #[doc(hidden)]
to elide the documentation from public view for now. This will keep it out of the crate public API.
Write block tests that exercise the new functionality.
Write tests that make assertions on the first 8-16 bytes of the block.
Note: The slice should be formatted as little endian.
Typically this means writing the expected contents as a &[u8]
that contains hex values and asserting its equivalence to the buffer the block is using as a container.
You can find the reader code in mod.rs. The tests in this change will probably be tricky, because the high-level API writer doesn't exist yet.
The primary changes here will be in the State
functionality. This is where blocks can be allocated and converted into the new type. If all you're doing is modifying an existing block, this is likely the only place you need to make changes.
There is a types directory where you can add a new file for your type.
Create the new type in the file created. Use an existing type as an example. Types always have access to internal State
. Use this to create the necessary methods on your new type, calling into the methods created in State
.
Add a method to Node
for creating the new type.
Ensure that your type has RAII semantics in the VMO. If your type is a value, this is probably done automatically by the boilerplate copied from an existing type in step 1.
Finally, go back and update the tests from the Reader change to use the new API!
The examples in this section create a new type called MyFoo
.
As noted above, this section should be two changes in Gerrit.
Note: You can also use Fuchsia products other than core.x64
.
fx set core.x64 --with //zircon/system/ulib/inspect:tests
fx test inspect-cpp-unittest
This section describes how to define the bitfields for your new type.
Update the block definition.
Change BlockType
to include your new type. For example: kMyFoo = #;
Note: This must be the same as the VMO format docs and the Rust implementation.
If your type needs a new header (typically if it is not a VALUE
), define the header bitfields for your type with a struct. For example: struct MyFooBlockFields final : public BlockFields
.
If your type needs a new payload (it requires using the second 8 bytes of the block), define the payload bitfields for your type with a struct. For example: struct MyFooBlockPayload final
.
If your type contains enums (such as format), define a new enum at the top of block.h. For example: enum class MyFooBlockFormat : uint8_t
.
This section describes how to make your new type readable.
Update the Inspect hierarchy based on your type:
{A value (child of Node
)}
Update PropertyFormat
enum with a new number for your type. This must be sequential in this specific enum and does not need to match the format type ordinal you chose.
Create a new value type. For example, using MyFooValue = internal::Value<T, static_cast<size_t>(PropertyFormat::kMyFoo)>;
Update PropertyValue
variant with the new value. Note: The index in fit::internal::variant
must match the value of PropertyFormat
.
{Not a value}
Update the actual reader.
Update InnerScanBlocks
to dispatch your type. If you are creating a new Property
, you may only have to add your BlockType
.
If you need a custom parser, implement InnerParseMyFoo
, which takes a parent (if needed) and the pointer to the scanned block.
This section describes how to declare a C++ RAII-style wrapper for your new type.
Type wrappers contain indices of blocks that are owned by the type. You are responsible for implementing operations on those blocks, including creation and deletion, in State action updates.
Update the writer types definition.
Determine if you can reuse an existing wrapper or if you need a bespoke type:
{Reuse}
If you need to support Add, Subtract, and Set: using MyFoo = internal::NumericProperty<T>
, where T
is the argument type to those operations.
If you need to support Set: using MyFoo = internal::Property<T>
, where T
is the argument type to Set.
If you need to support numeric operations on an array: using MyFood = internal::ArrayProperty<T>
, where T
is the argument type for slots in the array.
If you need to support inserting to a histogram: using MyFoo = internal::{Linear,Exponential}Histogram<T>
, where T
is the argument to Insert.
{Bespoke}
Create a new type wrapper. For example class MyFoo final
.
Ensure your class has internal::State
as a friend class. Note: See class Link
for a copyable starting point.
The State
class is the actual implementation for all operations on all types. This section describes how to implement the operations you will need to complete your wrapper implementation.
Update State
header:
Add Create and Free methods. For example: MyFoo CreateMyFoo(<args>); void FreeMyFoo(MyFoo* property);
where args
typically includes name, parent, and some initial value.
Add methods for each operation you need to support on your type. For example, if your type can be Set, void SetMyFoo(MyFoo* property, T)
, where T
is the same type from your update to types.h.
Update State
:
Note: Always lock the state before accessing any internal data, using std::lock_guard<std::mutex> lock(mutex_);
.
Note: Always lock the buffer before making any modifications to blocks, using AutoGenerationIncrement gen(header_, heap_.get());
.
Implement your new type's methods. The implementation varies between the different types. This section provides a high-level overview of what each method must do:
MyFoo CreateMyFoo(Args...)
is responsible for allocating a number of blocks, setting their values, and returning them wrapped in a MyFoo
. You may use a private constructor to create MyFoo
from the BlockIndex
objects it wraps. Various internal helpers exist to simplify this operation. See CreateIntProperty
for an example.
void FreeMyFoo(MyFoo* property)
is responsible for freeing all blocks wrapped by the MyFoo
. There are sometimes particular ordering requirements or updates necessary for freeing blocks. See InnerFreeValue
for an example of how values are freed.
Operations, such as void SetMyFoo(MyFoo* property, T value)
change the value of blocks allocated to MyFoo
to implement the operation. See SetIntProperty
for an example.
This section describes how to implement the wrapper methods declared previously.
If you used an existing templated type, you need to override each method for your new base type T
. For example, if you typed using MyFoo = internal::Property<T>
, you will write: template<> void internal::Property<T>::OPERATION(...) { ... }
If you created your own type, simply create definitions for the methods you declared. You need to do the following:
Make your constructor call state_->CreateMyFoo(...);
Make your destructor call state_->FreeMyFoo(...);
Make your other methods call the corresponding implementation on State.
Have all your constructors and methods check that state_
is not null before calling.
Update state unit tests with tests for your low-level operations.
Update reader unit tests with tests for your high-level reader implementation.
{% comment %} TODO(fxbug.dev/43131) {% endcomment %}
{% comment %} TODO(fxbug.dev/43131) {% endcomment %}
{% comment %}