Netemul Runner

Netemul Runner is a set of components that allow creating hermetic test environments for integration tests.

Besides creating sandboxed and hermetic environments, Netemul provides a set of services that allow for quick and easy network emulation that can be used to manage virtual networks between sandboxed environments. The hermetic environments with the virtual networks between them can be used to emulate multiple fuchsia devices talking to each other over networks and, thus, allows for self-contained integration tests.

Layout

Two components are provided netemul_runner and netemul_sandbox:

netemul_runner can be used as a runner, as defined by the component framework, and provides the FIDL service fuchsia.sys.Runner. It expects the program.data field of your test component's cmx manifest to point to another cmx file within the same component package.

netemul_sandbox is responsible for creating sandboxed environments that can be configured to run integration tests. netemul_sandbox has two modes of operation: enclosing process or service provider, and it is selected based on its command-line arguments.

In enclosing process mode, netemul_sandbox receives a component as a fuchsia package url as a command-line argument and proceeds to create a hermetic environment and then launch the provided component within it. The exit code of the netemul_sandbox process will mimic the component‘s. When using enclosing process, clients will typically setup a layout for the test using the netemul facet in the component-under-test’s cmx manifest.

In service provider mode, netemul_sandbox will expose the fuchsia.netemul.sandbox.Sandbox protocol that allows users to create netemul's managed environments or run full sandboxes from components.

In sum, users have two options to use netemul for a given component under test:

• use netemul_runner, which will use netemul_sandbox in enclosing process mode and spawn the component under test within a netemul environment. See Runner sample setup for an example of how to run a component like this.
• use netemul_sandbox as a service provider injected into the component under test's environment by using the standard fuchsia.test facet. The component under test can then simply use the provided fuchsia.netemul.sandbox.Sandbox service to spawn and control hermetic environments and emulated networks.

Runner sample setup

To have a test run on the sandbox, you'll typically use the following pattern:

• Use the build rule test_package
• For every test, you'll typically have at least 2 .cmx files:
• [my_component].cmx, where my_component is the component you want to run inside the hermetic environment. It should be indistinguishable from a regular component manifest. You can run multiple different components inside a netemul environment by simply using their package url. For this example, though, we'll only have the one [my_component].cmx.
• [my_test].cmx, in turn will contain a fuchsia.netemul facet that defines what environments to create and which tests to run and, also, will tell the component launcher to use netemul to run it:

{
    "program": {
        "binary": "bin/app"
    },
    "runner": "fuchsia-pkg://fuchsia.com/netemul_runner#meta/netemul_runner.cmx",
    "facets" : {
      "fuchsia.netemul" : {
        "environment" : {
          "test" : [{"url" : "fuchsia-pkg://fuchsia.com/my_component#meta/my_component.cmx"}]
        }
      }
    }
}

netemul_sandbox's tests use the pattern thoroughly and can be a good source of examples.

Note 1: the bin/app entry is just there to match the manifest schema, it is not used nor points to anything of importance.

Note 2: Make sure to match the build file pattern used here for tests that are defined solely by a cmx manifest, otherwise you risk your test being enumerated by the integration test framework.

Sandbox service sample setup

If your component under test doesn‘t need to (or shouldn’t) be inside a netemul environment, you can use the fuchsia.netemul.sandbox.Sandbox service by injecting it into your test component‘s environment. Your component’s cmx manifest will look like:

{
    "facets": {
        "fuchsia.test": {
            "injected-services": {
                "fuchsia.netemul.sandbox.Sandbox": "fuchsia-pkg://fuchsia.com/netemul-sandbox#meta/netemul-sandbox.cmx"
            }
        }
    },
    "program": {
        "binary": "test/my_test"
    },
    "sandbox": {
        "services": [
            "fuchsia.netemul.sandbox.Sandbox"
        ]
    }
}

The fuchsia.test facet when run by run_test_component (the standard way of running tests) will spawn an instance of the sandbox service for use by the component under test.

A rust example of how to use the sandbox service can be found here.

Services

The sandboxed environments created by netemul_sandbox will always contain the following FIDL services that can be used to manipulate and configure the environment. Just don't forget to add them to your .cmx's sandbox parameters to have access.

NetworkContext

netemul_sandbox creates a single NetworkContext instance (see fuchsia.netemul.network) and makes it available to all environments that are created under it. That is to say, NetworkContext intentionally breaks hermeticity. This service is the core motivator behind Netemul Runner, as it is able to create virtual networks that can be used to connect different hermetic test environments.

A valid pattern is to have a root environment configure the network setup and then spawn children to use it or, alternatively, the virtual networks can be setup using the configuration facet.

Managed Environment

Every environment in the sandbox will be exposed to the service fuchsia.netemul.environment.ManagedEnvironment. You can use the provided service to launch other child environments (which may or may not inherit settings). Every managed environment is completely hermetic from a services perspective, apart from basic configuration that may be inherited. If you wish to have service inheritance in your spawned children environments, refer to fuchsia.sys.Environment. In other words, managed environments never inherit their parent‘s services in the same sense as fuchsia.sys.Environment does, that is, inheriting the same instance of a service. Rather, managed environments may optionally inherit their parent’s service configuration, that is, the configuration of which services to launch in the environment.

The ManagedEnvironment service also provides a special fuchsia.sys.Launcher instance that provides extended functionality, such as forwarding VirtualDevice instances in a /vdev path for components that request dev sandbox permissions.

VirtualDevice instances are a vfs hook for NetworkContext's Endpoints. That allows for clients to expose specific Endpoints on the vfs under the created root folder /vdev. This feature is used to test components that perform vfs scanning to retrieve devices with minimal intrusion. This is used to go around the limitation that /dev is never hermetic to sandboxed environments.

The ManagedEnvironment also provides hermetic fuchsia.logger.LogSink and fuchsia.logger.Log services so that environments can run components that use syslog. Users have the ability to enable and customize log printing from syslog -- they can also completely disable syslog outputs. See LoggerOptions for configuration information.

SyncManager

Along the same lines as NetworkContext, netemul_sandbox creates a single Syncmanager instance (see fuchsia.netemul.sync) that is shared between all created environments. Clients can (and are encouraged to) use the Bus service or the other provided primitives to synchronize multiple test agents that are spawned across different environments.

For example (see easy_nestack_cfg test for working code), a test that creates a TCP server on one environment and a client on another is encouraged to use the Bus service to have the server communicate with the client out-of-band first and ensure it's actually ready to listen for incoming connections.

Facet

netemul_sandbox will look for the facet fuchsia.netemul on the .cmx file your provide to it. The facet can be used to build and configure the testing environment before your code gets called, thus decreasing the amount of boilerplate setup for every test.

Below is the documentation of the objects accepted. The root object is of type Config

Config

capture values:

• "ALWAYS" will always generate network dumps once sandbox exits.
• "ON_ERROR" will only generate dumps if the suite failed for any reason. Same as true.
• "NO" disables capturing. Same as false.

Network dumps are captured as pcap files and printed hex-encoded to STDOUT at the end of the test run. You can save the dump in hex format and then convert it to binary with xxd -r -p [hexdata].

Environment

LaunchArgs

Note: Launch args can always just be specified as a String value. In that case, it's as if only the url field had been specified.

Network

Endpoint

LoggerOptions

Note that the enabled option in LoggerOptions does not affect the availability of the logger service on the environment, but rather just controls that the logs be printed to stdout.

The klogs_enabled flag controls, in turn, the logger process that is launched in each environment making it listen or not to the kernel logs.

LoggerFilterOptions

Guest

Note that only one guest is allowed. Also, networks cannot contain more than a single entry.

Helpers

netemul_sandbox provides helpers that can be launched in sandboxed environments to perform common operations.

netstack_cfg

The netstack_cfg helper is a CLI-like tool that uses the NetworkContext service to retrieve emulated endpoints and attach them to netstack instances in an emulated environment. You can launch it by using its package url: fuchsia-pkg://fuchsia.com/netemul-sandbox#meta/netstack-cfg.cmx. It receives the command line arguments shown below and is typically used as a “setup” process in an environment's facet definition.

netstack_cfg
Configure netstack from emulated environments.

USAGE:
    netstack_cfg [FLAGS] [OPTIONS] -e <endpoint>

FLAGS:
    -h, --help             Prints help information
        --skip-up-check    netstack_cfg will by default wait until it sees the interface be reported as "Up", skip-up-
                           check will override that behavior.
    -V, --version          Prints version information

OPTIONS:
    -e <endpoint>        Endpoint name to retrieve from network.EndpointManager
    -i <ip>              Static ip address to assign (don't forget /prefix termination). Omit to use DHCP.

mock_device_settings

The mock_device_settings helper is a mock implementation of the FIDL protocol fuchsia.devicesettings.DeviceSettingsManager. You can launch it by using its package url: fuchsia-pkg://fuchsia.com/netemul-sandbox#meta/mock-device-settings.cmx. It receives the command line arguments shown below and is intended to be used as a “services” entry in environment's facet definition. You must specify the values for every key that your tests may want to access as a command-line option.

Serves DeviceSettingsManager with a memory-backed database

USAGE:
    mock_device_settings [OPTIONS]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -i <int_key>...           Starts with a pre-set integer key, format [k]=[v] e.g. Audio=100
    -s <string_key>...        Starts with a pre-set string key, format [k]=[v] e.g. DeviceName=my-device-name