This document describes the practices adopted by the local storage team but implementing these rules has not yet been completed for all storage formats.
Our on-disk storage systems need to persist across updates and multiple versions of the system may need to interact with this data. There will be some changes that will not be be backwards-compatible and software must avoid changing data it can not full understand. There can also be minor updates to the storage format that are backwards-compatible but may need special handling.
As an example for special handling of different revisions of backwards-compatible formats, say there is a minor bug that was corrected that made some previously valid data layout no longer permitted:
Old code should still be allowed to use the device since it can still read all of the data and can still write a format understood by newer revisions.
New code would like to correct the no-longer-valid format to a newer format when it occurs and it should be able to tell when this migration is required.
Utilities such as fsck need to know exactly what format to expect. If it sees the invalid structure in a device written only by the newer version of the code, it knows there is a serious error. But if the device was written to by older code, it knows that this is expected and to continue.
Format version: The version of the format on device. Different format versions are not compatible. Any non-backwards-compatible changes should increment the format version.
Revision: Any change to how data is stored in any way should update the revision. It may or may not be backwards-compatible.
Oldest revision: The oldest revision of the software (compatible with the format version) that has written to the device.
We do not maintain minor version numbers. The “revision” covers most of the uses of a minor version number in other systems (differentiating different but compatible formats). But because a device may be potentially written to by a range of software revisions (as long as they all understand the format version), there is no single “revision number” of the data.
Persistent storage systems should maintain two numbers in the header of their data:
Systems encode data in a way that allows formats to be added without invalidating older versions when possible. For example, if compression is supported in a filesystem, the compression algorithm should be stored on the file. This allows adding additional compression algorithms without invalidating older data.
To support future updates, all reserved regions in metadata should be zero initialized at format time. Unit tests should verify this. Verification tools should be lenient about checking reserved sections i.e. they should not check that reserved sections are zeroed. Similarly, length fields that allow structures to expand should be loosely checked. If it makes sense to do so, consider adding a “strict” option (disabled by default) that performs these checks.
Metrics for the format version, oldest revision, and current revision should be available via Cobalt.
When a device is initially created, the current format version and revision should be written to the header.
Software that opens a device should first check the format version. If the format version is larger than expected, the operation should fail and no operations should be attempted with the device.
Software that opens a device for writing should next check the device‘s current oldest revision. If the software revision is less than the oldest revision stored on the device, it should update the device’s oldest revision to the current software revision and continue. Software with newer revisions should not increase the oldest revision without performing an update.
Sometimes a migration may need to be done. For the example given in the “Background” section, we may want to check for and correct the newly-invalid format. In this case, the software can check whether the update is required by looking at the oldest revision of the device. If it is before the revision with the fix, the software knows that there may be data on the device that has the older format and should perform an update.
If the persistent data is updated so that none of it can be considered to have been written by an older revision, the oldest revision value should be update to the current value. This will prevent performing the migration in the future so long as no older revisions of the software write to the device.