Client and server use test.proto.
The code for the xDS test server can be found at: Java (other language implementations are in progress).
Server should accept these arguments:
The base behavior of the xDS test client is to send a constant QPS of unary messages and record the remote-peer distribution of the responses. Further, the client must expose an implementation of the LoadBalancerStatsService
gRPC service to allow the test driver to validate the load balancing behavior for a particular test case (see below for more details).
The code for the xDS test client can be at: Java (other language implementations are in progress).
Clients should accept these arguments:
LoadBalancerStatsService
implementation.The xDS test client's behavior can be dynamically changed in the middle of tests. This is achieved by invoking the XdsUpdateClientConfigureService
gRPC service on the test client. This can be useful for tests requiring special client behaviors that are not desirable at test initialization and client warmup. The service is defined as:
message ClientConfigureRequest { // Type of RPCs to send. enum RpcType { EMPTY_CALL = 0; UNARY_CALL = 1; } // Metadata to be attached for the given type of RPCs. message Metadata { RpcType type = 1; string key = 2; string value = 3; } // The types of RPCs the client sends. repeated RpcType types = 1; // The collection of custom metadata to be attached to RPCs sent by the client. repeated Metadata metadata = 2; } message ClientConfigureResponse {} service XdsUpdateClientConfigureService { // Update the tes client's configuration. rpc Configure(ClientConfigureRequest) returns (ClientConfigureResponse); }
The test client changes its behavior right after receiving the ClientConfigureRequest
. Currently it only supports configuring the type(s) of RPCs sent by the test client and metadata attached to each type of RPCs.
Note that, unlike our other interop tests, neither the client nor the server has any notion of which of the following test scenarios is under test. Instead, a separate test driver is responsible for configuring the load balancer and the server backends, running the client, and then querying the client's LoadBalancerStatsService
to validate load balancer behavior for each of the tests described below.
The service is defined as:
message LoadBalancerStatsRequest { // Request stats for the next num_rpcs sent by client. int32 num_rpcs = 1; // If num_rpcs have not completed within timeout_sec, return partial results. int32 timeout_sec = 2; } message LoadBalancerStatsResponse { // The number of completed RPCs for each peer. map<string, int32> rpcs_by_peer = 1; // The number of RPCs that failed to record a remote peer. int32 num_failures = 2; } message LoadBalancerAccumulatedStatsRequest {} message LoadBalancerAccumulatedStatsResponse { // The total number of RPCs have ever issued for each type. map<string, int32> num_rpcs_started_by_method = 1; // The total number of RPCs have ever completed successfully for each type. map<string, int32> num_rpcs_succeeded_by_method = 2; // The total number of RPCs have ever failed for each type. map<string, int32> num_rpcs_failed_by_method = 3; } service LoadBalancerStatsService { // Gets the backend distribution for RPCs sent by a test client. rpc GetClientStats(LoadBalancerStatsRequest) returns (LoadBalancerStatsResponse) {} // Gets the accumulated stats for RPCs sent by a test client. rpc GetClientAccumulatedStats(LoadBalancerAccumulatedStatsRequest) returns (LoadBalancerAccumulatedStatsResponse) {} }
Note that the LoadBalancerStatsResponse
contains the remote peer distribution of the next num_rpcs
sent by the client after receiving the LoadBalancerStatsRequest
. It is important that the remote peer distribution be recorded for a block of consecutive outgoing RPCs, to validate the intended distribution from the load balancer, rather than just looking at the next num_rpcs
responses received from backends, as different backends may respond at different rates.
This test verifies that every backend receives traffic.
Client parameters:
Load balancer configuration:
Test driver asserts:
This test verifies that RPCs are evenly routed according to an unweighted round robin policy.
Client parameters:
Load balancer configuration:
Test driver asserts that:
This test verifies that the load balancer will resume sending traffic to a set of backends that is stopped and then resumed.
Client parameters:
Load balancer configuration:
Test driver asserts:
The test driver records the peer distribution for a subsequent block of 100 RPCs then stops the backends.
Test driver asserts:
The test driver resumes the backends.
Test driver asserts:
This test verifies that backends in a secondary locality receive traffic when all backends in the primary locality fail.
Client parameters:
Load balancer configuration:
Test driver asserts:
The test driver stops the backends in the primary locality.
Test driver asserts:
The test driver resumes the backends in the primary locality.
Test driver asserts:
This test verifies that backends in a failover locality do not receive traffic when at least one of the backends in the primary locality remain healthy.
Note: Future TD features may change the expected behavior and require changes to this test case.
Client parameters:
Load balancer configuration:
Test driver asserts:
The test driver stops one of the backends in the primary locality.
Test driver asserts:
This test verifies that a remaining instance group can successfully serve RPCs after removal of another instance group in the same zone.
Client parameters:
Load balancer configuration:
Test driver asserts:
The test driver removes one MIG.
Test driver asserts:
This test verifies that the backend service can be replaced and traffic routed to the new backends.
Client parameters:
Load balancer configuration:
Test driver asserts:
The test driver creates a new backend service containing a MIG with two backends and changes the TD URL map to point to this new backend service.
Test driver asserts:
This test verifies that the traffic will be distributed between backend services with the correct weights when route action is set to weighted backend services.
Client parameters:
Load balancer configuration:
Assert:
The test driver adds a new MIG with 1 backend, and changes the route action to weighted backend services with {a: 20, b: 80}.
Assert:
This test verifies that the traffic for a certain RPC can be routed to a specific cluster based on the RPC path.
Client parameters:
Load balancer configuration:
Assert:
The test driver changes route and asserts RPCs are sent to expected backends. Note that the default route "/"
is always pointing to MIG_default, so all RPCs not matching the new route will be sent to MIG_default.
{path: /grpc.testing.TestService/EmptyCall
}: MIG_2
{prefix: /grpc.testing.TestService/Unary
}: MIG_2
{prefix: /grpc.testing.TestService/Unary
}: MIG_default & {path: /grpc.testing.TestService/EmptyCall
}: MIG_2
{regex: ^\/.*\/UnaryCall$
}: MIG_2
{path: /gRpC.tEsTinG.tEstseRvice/empTycaLl
, ignoreCase: True
}: MIG_2
This test verifies that the traffic for a certain RPC can be routed to a specific cluster based on the RPC header (metadata).
Client parameters:
Load balancer configuration:
Assert:
The test driver changes route and asserts RPCs are sent to expected backends. Note that the default route "/"
is always pointing to MIG_default, so all RPCs not matching the new route will be sent to MIG_default.
{header xds_md
, exact: empty_ytpme
}: MIG_2
{header xds_md
, prefix: un
}: MIG_2
un
is the prefix of metadata sent with UnaryCall{header xds_md
, suffix: me
}: MIG_2
me
is the suffix of metadata sent with EmptyCall{header xds_md_numeric
, present: True
}: MIG_2
{header xds_md
, exact: unary_yranu
, invert: True
}: MIG_2
{header xds_md_numeric
, range [100,200]
}: MIG_2
{header xds_md
, regex: ^em.*me$
}: MIG_2
This test verifies that traffic is partially diverted to a secondary locality when > 50% of the instances in the primary locality are unhealthy.
Client parameters:
Load balancer configuration:
Test driver asserts:
The test driver stops 2 of 3 backends in the primary locality.
Test driver asserts:
The test driver resumes the backends in the primary locality.
Test driver asserts:
This test verifies that traffic is partially diverted to a secondary locality when the QPS is greater than the configured RPS in the priority locality.
Client parameters:
Load balancer configuration:
Test driver asserts:
The test driver sets balancingMode
is RATE
, and maxRate
to 20 in the primary locality.
Test driver asserts:
The test driver set maxRate
to 120 in the primary locality.
Test driver asserts:
This test verifies that the maximum number of outstanding requests is limited by circuit breakers of the backend service.
Client parameters:
Load balancer configuration:
The test driver configures the backend services with:
The test driver configures the test client to send both UnaryCall and EmptyCall, with all RPCs keep-open.
Assert:
The test driver updates MIG_1's circuit breakers with max_request = 800.
Test driver asserts:
This test verifies that traffic along a route with a max_stream_duration
set will cause timeouts on streams open longer than that duration.
Client parameters:
--num_channels=1
--qps=100
Route Configuration:
Two routes:
/grpc.testing.TestService/UnaryCall
, with a route_action
containing max_stream_duration
of 3 seconds.max_stream_duration
setting.There are four sub-tests:
app_timeout_exceeded
rpc-behavior: sleep-2
.DEADLINE_EXCEEDED
.timeout_not_exceeded
OK
.timeout_exceeded
(executed with the below test case)timeout_different_route
rpc-behavior: sleep-4
.OK
for EmptyCall and ~100% status DEADLINE_EXCEEDED
for UnaryCall.