This document describes the design of parsing and serialization in the netstack.
The design of parsing and serialization is organized around the principle of zero copy. Whenever a packet is parsed, the resulting packet object (such as wire::ipv4::Ipv4Packet) is simply a reference to the buffer it was parsed from, and so parsing involves no copying of memory. This allows for a number of desirable design patterns.
Most of the design patterns relating to buffers are enabled by utilities provided by the packet crate. See its documentation here.
Since packet objects are merely views into an existing buffer, when they are dropped, the original buffer can be modified directly again. This allows buffers to be reused once the data inside of them is no longer needed. To accomplish this, we use the packet crate's ParseBuffer trait, which is a buffer that can keep track of which bytes have already been consumed by parsing, and which are yet to be parsed. The bytes which have not yet been parsed are called the “body”, and the bytes preceding and following the body are called the “prefix” and the “suffix.”
In order to demonstrate how we use buffers in the netstack, consider this hypothetical control flow in response to receiving an Ethernet frame:
TcpSegment object, which itself borrowed the buffer), regaining direct mutable access to the original buffer. Since the buffer has access to the now-parsed prefix and suffix in addition to the body, the TCP stack can now use the entire buffer for serializing. It figures out how much space must be left at the beginning of the buffer for all lower-layer headers (in this case, IPv4 and Ethernet), and serializes a TCP Ack segment at the appropriate offset into the buffer. Now that the buffer is being used for serialization (rather than parsing), the body indicates the range of bytes which have been serialized so far, and the prefix and suffix represent empty space which can be used by lower layers to serialize their headers and footers.Note that, in this entire control flow, only a single buffer is ever used, although it is at times used for different purposes. If, in step 3, the TCP layer finds that the buffer is too small for the TCP segment that it wishes to serialize, it can still allocate a larger buffer. However, so long as the existing buffer is sufficient, it may be reused.
When using a single buffer to serialize a packet - including any encapsulating headers of lower layers of the stack - it is important to satisfy some constraints:
Consider, for example, Ethernet. When serializing an IPv4 packet inside of an Ethernet frame, the IPv4 packet must leave enough room for the Ethernet header. Ethernet has a minimum body requirement, and so there must also be enough bytes following the IPv4 packet to be used as padding in order to meet this minimum in case the IPv4 packet itself is not sufficiently large.
To accomplish this, we use the packet crate's Serializer trait. A Serializer represents the metadata needed to serialize a request in the future. Serializers may be nested, which results in a Serializer describing a sequence of encapsulated packets to be serialized, each being used as the body of an encapsulating packet. When a sequence of nested Serializers is processed, the header, footer, and minimum body size requirements are computed starting with the outermost packet and working in. Once the innermost Serializer is reached, it is that Serializer's responsibility to provide a buffer:
BufferMut trait (a superset of the functionality required by the ParseBuffer trait mentioned above), and the buffer's body indicates the bytes to be encapsulated by the next layer.Once the innermost Serializer has produced its buffer, each subsequent packet serializes its headers and footers, expands the buffer's body to contain the whole packet as the body to be encapsulated in the next layer, and returns the buffer to be handled by the next layer.
When control makes its way to a layer of the stack that has a minimum body length requirement, that layer is responsible for consuming any bytes following the range for use as padding (and zeroing those bytes for security). This logic is handled by EncapsulatingSerializer's implementation of serialize.
Serializer is implemented by a number of different types, allowing for a range of serialization scenarios including:
Serializers.In certain scenarios, it is necessary to modify an existing packet before re-serializing it. The canonical example of this is IP forwarding. When all packets are simply references into a pre-existing buffer, modifying and then re-serializing packets is cheap, as it can all be done in-place in the common case. For example, if an IP packet is received which needs to be forwarded, so long as the link-layer headers of the device over which it is to be forwarded are not larger than the link-layer headers of the device over which it was received, there will be enough space in the existing buffer to serialize new link-layer headers and deliver the resulting link-layer frame to the appropriate device without ever having to allocate extra buffers or copy any data between buffers.
If buffers that previously held other packets are re-used for serializing new packets, then there is a risk that data from the old packets will leak into the new packet if every byte of the new packet is not explicitly initialized. In order to prevent this:
wire module) is responsible for ensuring that all of the bytes of the header have been initialized.A special case of these requirements is post-body padding. For packet formats with minimum body size requirements, upper layers will provide extra buffer bytes beyond the end of the body. In the BufferMut used to store packets during serialization, these padding bytes are in the “suffix” just following the buffer's body.
The logic for adding padding is provided by EncapsulatingSerializer, described in the Prefix, Suffix, and Padding section above. EncapsulatingSerializer ensures that these padding bytes are zeroed.
See also: the _zeroed constructors of the zerocopy::LayoutVerified type