docs/mdi.md - zircon - Git at Google

 # Zircon Device Index (MDI)

 ## Introduction

 The Zircon Device Index (MDI) is a read-only binary data structure passed from the bootloader
 that contains configuration information for the kernel and various drivers in zircon.
 For platforms that use it, the MDI binary data is included as a section in the bootdata.bin file.
 The MDI binary is generated by the mdigen tool from one or more MDI source files.
 The mdigen tool also generates a header file containing C preprocessor defines for the MDI ids
 defined in the MDI sources.

 The MDI data is structured as a tree of nodes, can be thought of as ID/value pairs.
 IDs are 32 bit integers that encode both the name and type of the node.
 MDI node values can be one of the following:

  * `uint8` - an 8-bit unsigned integer
  * `int32` - a 32-bit signed integer
  * `uint32` - a 32-bit unsigned integer
  * `uint64` - a 64-bit unsigned integer
  * `boolean` -  uint8 that is either `true` or `false`
  * `string` - a zero-terminated array of uint8s representing an ASCII string
  * `list` - an ordered list of MDI nodes
  * `array` - an ordered list of directly indexable values of the same type.
    Allowed array types are uint8, int32, uint32, uint64 and boolean.

 ## Syntax

 MDI has a very simple syntax with a small number of tokens and reserved words.
 The syntax is so simple that no extra symbols are needed to determine the boundary between
 top-level statements or elements within lists.
 The only exception is that commas are needed between elements in arrays.

 MDI supports C/C++ style comments. Whitespace is not significant,
 other than the fact that comments beginning with `//` are terminated by the end of the line.

 Integers can be expressed as literal values, either in decimal, octal or hexadecimal form.
 Octal integer literals start with a leading `0` and hexadecimal literals start with a leading
 `0x` or `0X`

 MDI also supports integer expressions. The arithmetic operators `+`, `-`, `*`, `/`, `%` are supported,
 as well as the bitwise operators `~`, `|`, `&`, `^`, `<<` and `>>`.
 The syntax and precedence rules are the same as in C.

 MDI source files can contain three types of top-level statements:
  * Includes
  * ID Definitions
  * Constant Definitions
  * Node Definitions

 ### Includes

 Includes are statements that include other MDI source files, similar to `#include` in C or C++.
 Include statements begin with the `include` keyword, followed by a double quoted path to another
 MDI file to include. The path is relative to the current working directory of the mdigen tool.
 For example:

 `include "kernel/include/mdi/kernel-defs.mdi"`

 At this point the mdigen will process the MDI source code in the file
 `kernel/include/mdi/kernel-defs.mdi` before continuing to process the current source file.

 ### ID Definitions

 ID definitions define the IDs that can be used for nodes in the tree.
 ID definitions consist of a type, one or more dot-separated identifiers, a C symbol name and an integer value.
 When the MDI binary is generated, the IDs are written as 32-bit integers containing both the
 ID definition's type and integer value.
 The dot-separated list of identifiers are used to match a paths to nodes in the MDI source file.

 For example, the following are ID definitions:

 ```
 int32  foo      MDI_FOO     1
 list   bar      MDI_BAR     2
 string bar.baz  MDI_BAR_BAZ 3
 ```

 IDs of type array must also specify the array element type for the ID:

 ```
 array[boolean]  boolean-array   4
 ```

 Identifiers can contain alphanumeric characters and the `'-'` and `'_'` characters,
 and must start with a letter.

 The mdigen uses the C symbol names to generate a C header file to be included by C/C++ code that
 uses the MDI. The example above will result in the following code in the C header file:

 ```
 #define MDI_FOO                                            0x00800001
 #define MDI_BAR                                            0x08000002
 #define MDI_BAR_BAZ                                        0x09000003
 ```

 The purpose of using integer IDs rather than arbitrary string names for nodes is to:
  1. provide build time error checking when compiling MDI files, and
  2. provide better efficiency when traversing the MDI binary at runtime

 ### Constant Definitions

 Integer constants can be defined for integer literals or expressions. For example:

 ```
 const ONE = 1
 const THREE = 1 + 2
 ```

 ### Node Definitions

 Node definitions define a top-level node in the MDI node tree.
 All top-level nodes are implicitly added to the MDI root node,
 which is an unnamed list that does not actually appear in the MDI source code.
 Node definitions are of the form `<identifier> = <value>`,
 where value can be an integer literal, expression, constant, a boolean or string literal, a list or an array.
 List values begin and end with `'{' and '}'`, while array values begin and end with `'[' and ']'`

 For example, the following MDI node definitions can be written using the ID definitions
 in the examples above:

 ```
 foo = 1

 bar = {
     baz = "Hi there"
 }

 boolean-array = [ true, false, true ]
 ```

 Compiling this code will generate an MDI binary a root node with three children: `foo`, `bar`
 and `bool-array`.

 ## Binary Format

 An MDI binary begins with a `bootdata_t` header
 (see [system/public/zircon/boot/bootdata.h](../system/public/zircon/boot/bootdata.h))
 The `bootdata_t.type` field is set to `BOOTDATA_TYPE_MDI`.

 Following the `bootdata_t` header are a sequence of `mdi_node_t` structs.
 These structs are aligned to 8 byte boundaries.
 The `mdi_node_t` struct contains:

  * the node's ID
  * total length of the node, including subnodes
  * the node's value (for nodes of type `uint8`, `int32`, `uint32`, `uint64` and `boolean`
  * for strings, the string's length
  * for lists, the number of immediate child nodes
  * for arrays, the number of array elements

 List nodes are immediately followed by the children of the list. Strings and arrays are
 immediately followed by the string value or array elements.

 IDs contain the node type in the high bits and the ID's integer value in the low bits.
 See the header file [system/public/zircon/mdi.h](../system/public/zircon/mdi.h) for more details.

 ## mdigen

 The mdigen tool generates MDI binaries from one or more MDI source files,
 with paths to the source files provided via mdigen's command line.
 mdigen can also take the following additional options:

  * `-o <path to output MDI binary>`
  * `-h <path to output C header file containing MDI ID definitions>`

 When invoking mdigen, you must specify either a `-o` or `-h` option, or both.

 ## MDI Library

 Zircon contains a library for reading the MDI binary format
  The library can be used both in the kernel and userspace.
 The header file for the MDI library is at
 [system/ulib/mdi/include/mdi/mdi.h](../system/ulib/mdi/include/mdi/mdi.h).

 When using the MDI library, one must first call `mdi_init()` to get a reference
 to the root node in the MDI tree. The remaining library functions can be used to read
 the value of a node, access the ith element of an array node,
 or traverse the children of list nodes.

 The MDI library does not allocate memory or depend on any other libraries,
 so it can be used within any context in the zircon kernel or userspace.
	# Zircon Device Index (MDI)

	## Introduction

	The Zircon Device Index (MDI) is a read-only binary data structure passed from the bootloader
	that contains configuration information for the kernel and various drivers in zircon.
	For platforms that use it, the MDI binary data is included as a section in the bootdata.bin file.
	The MDI binary is generated by the mdigen tool from one or more MDI source files.
	The mdigen tool also generates a header file containing C preprocessor defines for the MDI ids
	defined in the MDI sources.

	The MDI data is structured as a tree of nodes, can be thought of as ID/value pairs.
	IDs are 32 bit integers that encode both the name and type of the node.
	MDI node values can be one of the following:

	* `uint8` - an 8-bit unsigned integer
	* `int32` - a 32-bit signed integer
	* `uint32` - a 32-bit unsigned integer
	* `uint64` - a 64-bit unsigned integer
	* `boolean` - uint8 that is either `true` or `false`
	* `string` - a zero-terminated array of uint8s representing an ASCII string
	* `list` - an ordered list of MDI nodes
	* `array` - an ordered list of directly indexable values of the same type.
	Allowed array types are uint8, int32, uint32, uint64 and boolean.

	## Syntax

	MDI has a very simple syntax with a small number of tokens and reserved words.
	The syntax is so simple that no extra symbols are needed to determine the boundary between
	top-level statements or elements within lists.
	The only exception is that commas are needed between elements in arrays.

	MDI supports C/C++ style comments. Whitespace is not significant,
	other than the fact that comments beginning with `//` are terminated by the end of the line.

	Integers can be expressed as literal values, either in decimal, octal or hexadecimal form.
	Octal integer literals start with a leading `0` and hexadecimal literals start with a leading
	`0x` or `0X`

	MDI also supports integer expressions. The arithmetic operators `+`, `-`, `*`, `/`, `%` are supported,
	as well as the bitwise operators `~`, `\|`, `&`, `^`, `<<` and `>>`.
	The syntax and precedence rules are the same as in C.

	MDI source files can contain three types of top-level statements:
	* Includes
	* ID Definitions
	* Constant Definitions
	* Node Definitions

	### Includes

	Includes are statements that include other MDI source files, similar to `#include` in C or C++.
	Include statements begin with the `include` keyword, followed by a double quoted path to another
	MDI file to include. The path is relative to the current working directory of the mdigen tool.
	For example:

	`include "kernel/include/mdi/kernel-defs.mdi"`

	At this point the mdigen will process the MDI source code in the file
	`kernel/include/mdi/kernel-defs.mdi` before continuing to process the current source file.

	### ID Definitions

	ID definitions define the IDs that can be used for nodes in the tree.
	ID definitions consist of a type, one or more dot-separated identifiers, a C symbol name and an integer value.
	When the MDI binary is generated, the IDs are written as 32-bit integers containing both the
	ID definition's type and integer value.
	The dot-separated list of identifiers are used to match a paths to nodes in the MDI source file.

	For example, the following are ID definitions:

	```
	int32 foo MDI_FOO 1
	list bar MDI_BAR 2
	string bar.baz MDI_BAR_BAZ 3
	```

	IDs of type array must also specify the array element type for the ID:

	```
	array[boolean] boolean-array 4
	```

	Identifiers can contain alphanumeric characters and the `'-'` and `'_'` characters,
	and must start with a letter.

	The mdigen uses the C symbol names to generate a C header file to be included by C/C++ code that
	uses the MDI. The example above will result in the following code in the C header file:

	```
	#define MDI_FOO 0x00800001
	#define MDI_BAR 0x08000002
	#define MDI_BAR_BAZ 0x09000003
	```

	The purpose of using integer IDs rather than arbitrary string names for nodes is to:
	1. provide build time error checking when compiling MDI files, and
	2. provide better efficiency when traversing the MDI binary at runtime

	### Constant Definitions

	Integer constants can be defined for integer literals or expressions. For example:

	```
	const ONE = 1
	const THREE = 1 + 2
	```

	### Node Definitions

	Node definitions define a top-level node in the MDI node tree.
	All top-level nodes are implicitly added to the MDI root node,
	which is an unnamed list that does not actually appear in the MDI source code.
	Node definitions are of the form `<identifier> = <value>`,
	where value can be an integer literal, expression, constant, a boolean or string literal, a list or an array.
	List values begin and end with `'{' and '}'`, while array values begin and end with `'[' and ']'`

	For example, the following MDI node definitions can be written using the ID definitions
	in the examples above:

	```
	foo = 1

	bar = {
	baz = "Hi there"
	}

	boolean-array = [ true, false, true ]
	```

	Compiling this code will generate an MDI binary a root node with three children: `foo`, `bar`
	and `bool-array`.

	## Binary Format

	An MDI binary begins with a `bootdata_t` header
	(see [system/public/zircon/boot/bootdata.h](../system/public/zircon/boot/bootdata.h))
	The `bootdata_t.type` field is set to `BOOTDATA_TYPE_MDI`.

	Following the `bootdata_t` header are a sequence of `mdi_node_t` structs.
	These structs are aligned to 8 byte boundaries.
	The `mdi_node_t` struct contains:

	* the node's ID
	* total length of the node, including subnodes
	* the node's value (for nodes of type `uint8`, `int32`, `uint32`, `uint64` and `boolean`
	* for strings, the string's length
	* for lists, the number of immediate child nodes
	* for arrays, the number of array elements

	List nodes are immediately followed by the children of the list. Strings and arrays are
	immediately followed by the string value or array elements.

	IDs contain the node type in the high bits and the ID's integer value in the low bits.
	See the header file [system/public/zircon/mdi.h](../system/public/zircon/mdi.h) for more details.

	## mdigen

	The mdigen tool generates MDI binaries from one or more MDI source files,
	with paths to the source files provided via mdigen's command line.
	mdigen can also take the following additional options:

	* `-o <path to output MDI binary>`
	* `-h <path to output C header file containing MDI ID definitions>`

	When invoking mdigen, you must specify either a `-o` or `-h` option, or both.

	## MDI Library

	Zircon contains a library for reading the MDI binary format
	The library can be used both in the kernel and userspace.
	The header file for the MDI library is at
	[system/ulib/mdi/include/mdi/mdi.h](../system/ulib/mdi/include/mdi/mdi.h).

	When using the MDI library, one must first call `mdi_init()` to get a reference
	to the root node in the MDI tree. The remaining library functions can be used to read
	the value of a node, access the ith element of an array node,
	or traverse the children of list nodes.

	The MDI library does not allocate memory or depend on any other libraries,
	so it can be used within any context in the zircon kernel or userspace.