docs(spanner): add decode performance trial guides
diff --git a/spanner/CUSTOMER-TRIAL.md b/spanner/CUSTOMER-TRIAL.md new file mode 100644 index 0000000..1babf6c --- /dev/null +++ b/spanner/CUSTOMER-TRIAL.md
@@ -0,0 +1,94 @@ +# Spanner allocation trial: three things to try + +Thank you for testing the earlier patch and sharing that its improvement was modest. Based on the allocator profile and the follow-up measurements, I suggest trying these in order. The first change works with your current released client; the next two use the experimental branch. + +> **Required to test the fast path:** change each benchmark call from `client.Single().Query(ctx, stmt)` to `QueryWithOptions` with `ExperimentalRawDecode: true`, as shown in Step 3. Changing only the dependency does not enable the fast path: plain `Query`, `QueryWithStats`, and every `Read*` API stay on the safe default. Without this call-site change, the trial measures the +38% safe default rather than the +154% opt-in path. + +## Step 1: hoist row destinations out of the callback + +This costs nothing and needs no branch. Declaring destination variables inside the per-row callback, then passing their addresses through the variadic `Row.Columns` interface, forces all eight destinations onto the heap: eight allocations per row before the Spanner library does any work. Hoisting them outside the callback reduces those caller-side allocations to zero. + +At your reported ~7 M rows/s, this removes approximately 56 million allocations/s from the allocator that is already saturated. This can be applied to your current released client today. + +| Before: destinations escape on every row | After: destinations are reused | +| --- | --- | +| <pre><code class="language-go">err := iter.Do(func(row *spanner.Row) error {<br> var c1, c2, c3, c4 string<br> var c5, c6, c7, c8 string<br><br> return row.Columns(<br> &c1, &c2, &c3, &c4,<br> &c5, &c6, &c7, &c8,<br> )<br>})</code></pre> | <pre><code class="language-go">var c1, c2, c3, c4 string<br>var c5, c6, c7, c8 string<br><br>err := iter.Do(func(row *spanner.Row) error {<br> return row.Columns(<br> &c1, &c2, &c3, &c4,<br> &c5, &c6, &c7, &c8,<br> )<br>})</code></pre> | + +Keep each set of reused destinations local to one iterator or worker; do not share it concurrently. + +## Step 2: try the branch's safe default + +This needs no application code change and preserves normal row and string lifetimes. From your module, run: + +```sh +go get cloud.google.com/go/spanner@spanner-decode-perf +go mod tidy +go list -m cloud.google.com/go/spanner +``` + +The final command must print: + +```text +cloud.google.com/go/spanner v1.94.1-0.20260803090513-da1e076ac0bc +``` + +Confirm this before benchmarking so you know which client version is in the build. A branch name is a moving target; for a reproducible build, use the explicit pin instead: + +```sh +go get cloud.google.com/go/spanner@v1.94.1-0.20260803090513-da1e076ac0bc +``` + +On our 30-vCPU host against a 10-node instance, the safe default produced: + +- throughput: **+38%**; +- allocations per row: **36.44 → 33.40**; +- allocated bytes per row: **1657 → 1317**. + +## Step 3: try the opt-in query fast path + +**Changing the dependency is not enough. `QueryWithOptions` is required to reach the fast path.** Replace every benchmark call to plain `Query` with: + +```go +iter := client.Single().QueryWithOptions(ctx, stmt, spanner.QueryOptions{ + ExperimentalRawDecode: true, +}) +defer iter.Stop() +``` + +On the same host and instance, the three measured arms produced: + +| Metric | Released client | Safe default | Opt-in fast path | +| --- | ---: | ---: | ---: | +| Throughput | 3.77M rows/s | 5.18M rows/s (+38%) | 9.57M rows/s (+154%) | +| Allocations per row | 36.44 | 33.40 | 15.49 | +| Bytes per row | 1657 | 1317 | 290 | +| `mcache.refill` CPU share | 7.52% | 11.78% | 2.57% | + +### Lifetime contract + +**A row, its column values, and strings decoded from it are valid only until the next `Next` or `Stop`. Anything retained must be copied first. Getting this wrong can silently corrupt data rather than return an error.** + +| Correct: copy before `Next` | Incorrect: retain aliases past `Next` | +| --- | --- | +| <pre><code class="language-go">var names []string<br>for {<br> row, err := iter.Next()<br> if err == iterator.Done {<br> break<br> }<br> if err != nil {<br> return err<br> }<br><br> var name string<br> if err := row.Column(0, &name); err != nil {<br> return err<br> }<br> names = append(names, strings.Clone(name))<br>}</code></pre> | <pre><code class="language-go">var rows []*spanner.Row<br>var names []string<br>for {<br> row, err := iter.Next()<br> if err == iterator.Done {<br> break<br> }<br> if err != nil {<br> return err<br> }<br><br> var name string<br> _ = row.Column(0, &name)<br> rows = append(rows, row)<br> names = append(names, name)<br>}<br>// Retained data may now be overwritten.</code></pre> | + +Use `strings.Clone` for every retained string before advancing or stopping the iterator. Any retained row or composite value needs an appropriate deep copy. If auditing ownership is risky, Step 2 alone is still worth trying. + +This opt-in applies only to SQL result rows produced by `QueryWithOptions`. `ExperimentalRawDecode` is not available on `ReadOptions`, so `Read`, `ReadWithOptions`, `ReadUsingIndex`, `ReadRow`, `ReadRowWithOptions`, and `ReadRowUsingIndex` stay on the safe default. Plain `Query` and `QueryWithStats` also stay on the safe default; they never enable the fast path. + +## What to measure + +Please compare each arm with your own released-client baseline on the same host and workload shape. Our absolute rates are not a useful target for a different environment. Capture: + +- rows/sec; +- allocations per row; +- `runtime.MemStats` mallocs and total allocation; +- `runtime.(*mcache).refill` share from a CPU profile; +- resident memory, including peak and end-of-run values. + +## Caveats + +- Our measurements used a 10-node test instance, 128 threads, strong reads, and rows with eight columns on a 30-vCPU host. Your absolute figures will differ; relative gains are the meaningful comparison. +- The stale-read arm failed on our benchmark rig with `Bad BeginTransaction request` under multiplexed sessions. We did not get stale-read measurements, so the stale-read path is unverified on this branch. Because your workload uses stale reads, please report the behavior and measurements you see. +- The opt-in path raised end-of-run resident memory roughly 2x because receive buffers stayed pinned. Peak resident memory stayed within ~3% of the released client on our host. Please watch both peak and steady-state resident memory in your environment. +- This is an experimental branch with no support or compatibility guarantee.
diff --git a/spanner/EXPERIMENTAL-DECODE.md b/spanner/EXPERIMENTAL-DECODE.md index e00ffbe..aca61e7 100644 --- a/spanner/EXPERIMENTAL-DECODE.md +++ b/spanner/EXPERIMENTAL-DECODE.md
@@ -2,6 +2,8 @@ This branch is a customer trial for Go applications whose Spanner reads are limited by allocation count or protobuf decode CPU. It is not a released or supported API. +> **Required to test the fast path:** use `QueryWithOptions` with `ExperimentalRawDecode: true` at every benchmark call site. Changing only the dependency does not enable it. Plain `Query`, `QueryWithStats`, and every `Read*` API stay on the safe default, so a benchmark that continues to call `client.Single().Query(ctx, stmt)` measures the +38% safe default rather than the +154% opt-in path. + ## What changes All RPCs made by the Spanner data client use vtprotobuf at the gRPC connection level for both request marshaling and response unmarshaling. This includes session RPCs, unary calls such as `ExecuteSql` and `Commit`, and streaming calls such as `StreamingRead` and `ExecuteStreamingSql`. @@ -10,24 +12,27 @@ ## Use this branch from another module -Pin the trial. Do not depend on a moving branch name in a reproducible build. - -```mod -require cloud.google.com/go/spanner v1.91.0 - -replace cloud.google.com/go/spanner => cloud.google.com/go/spanner v1.91.1-0.20260803085303-2e16398ef74c -``` - -Equivalent commands: +From your module, run: ```sh -go mod edit -require=cloud.google.com/go/spanner@v1.91.0 -go mod edit -replace=cloud.google.com/go/spanner=cloud.google.com/go/spanner@v1.91.1-0.20260803085303-2e16398ef74c +go get cloud.google.com/go/spanner@spanner-decode-perf go mod tidy go list -m cloud.google.com/go/spanner ``` -`go list` must print `v1.91.1-0.20260803085303-2e16398ef74c`. Commit both `go.mod` and `go.sum`. Remove the `replace` directive to return to a released client. +The final command must print: + +```text +cloud.google.com/go/spanner v1.94.1-0.20260803090513-da1e076ac0bc +``` + +This output confirms which client version is in the build. If it differs, stop before benchmarking. A branch name is a moving target; for a reproducible build, use the explicit pin instead: + +```sh +go get cloud.google.com/go/spanner@v1.94.1-0.20260803090513-da1e076ac0bc +``` + +Commit both `go.mod` and `go.sum` used for the trial. ## Two decode modes @@ -37,11 +42,11 @@ The default codec calls `UnmarshalVT`, which copies strings and bytes out of gRPC receive buffers. Rows, `ColumnValue` results, and decoded Go strings keep normal client semantics: callers may retain them after another `Next`, after `Stop`, or after the iterator is gone. -This mode primarily reduces protobuf decode CPU. It intentionally keeps copy allocations needed for ordinary Go ownership. In the 800-string decode benchmark on this branch, safe vtprotobuf was about 58% faster than reflection but had essentially the same allocation count. +This mode primarily reduces protobuf decode CPU. It intentionally keeps copy allocations needed for ordinary Go ownership. In the verified 800-string decode benchmark on this branch, safe vtprotobuf was 57.97% faster than reflection while allocations per operation changed from 2,412 to 2,411. ### Opt-in fast query path -Enable the existing query option: +**Changing the dependency is not enough. `QueryWithOptions` is required to reach the fast path.** Enable the existing query option at every benchmark call site: ```go iter := client.Single().QueryWithOptions(ctx, stmt, spanner.QueryOptions{ @@ -50,7 +55,32 @@ defer iter.Stop() ``` -This mode uses `UnmarshalVTUnsafe`, pooled `PartialResultSet` and `structpb.Value` objects, and a row reused by `RowIterator.Next`. It removes string copies and recycles the receive representation. The isolated 800-string benchmark used exactly one scalar oneof allocation per column instead of about three allocations per column on the stock path. +This mode uses `UnmarshalVTUnsafe`, pooled `PartialResultSet` and `structpb.Value` objects, and a row reused by `RowIterator.Next`. It removes string copies and recycles the receive representation. The verified 800-string benchmark used 1.000 scalar oneof allocation per column instead of 3.015 allocations per column on the stock path. + +### API coverage + +`ExperimentalRawDecode` exists only on `QueryOptions` and is wired only through the SQL query path. The opt-in fast path applies to result rows returned by `QueryWithOptions` when `ExperimentalRawDecode` is `true`. + +There is no corresponding field on `ReadOptions`. `Read`, `ReadWithOptions`, `ReadUsingIndex`, `ReadRow`, `ReadRowWithOptions`, and `ReadRowUsingIndex` remain on the safe default. Plain `Query` and `QueryWithStats` also remain on the safe default because they do not enable this option. + +### Verified end-to-end results + +On the same 30-vCPU host against a 10-node instance, the three measured arms produced: + +| Metric | Released client | Safe default | Opt-in fast path | +| --- | ---: | ---: | ---: | +| Throughput | 3.77M rows/s | 5.18M rows/s (+38%) | 9.57M rows/s (+154%) | +| Allocations per row | 36.44 | 33.40 | 15.49 | +| Bytes per row | 1657 | 1317 | 290 | +| `mcache.refill` CPU share | 7.52% | 11.78% | 2.57% | + +### Read consistency coverage + +The verification runs for this branch used strong reads. The stale-read arm failed on the benchmark rig with `Bad BeginTransaction request` under multiplexed sessions, so the stale-read path is unverified on this branch. If your workload uses stale reads, please report the behavior and measurements you observe. + +### Memory coverage + +The opt-in path raised end-of-run resident memory roughly 2x because receive buffers stayed pinned. Peak resident memory stayed within ~3% of the released client on the measured host. Monitor both peak and steady-state resident memory in your environment. ## Opt-in lifetime contract @@ -73,4 +103,4 @@ - generated vtprotobuf methods being part of the public Spanner API; - this branch receiving release support, compatibility guarantees, or security updates. -The safe default preserves public data-lifetime semantics, but the codec implementation itself is still a branch experiment. Re-test workload correctness and performance before moving to any later commit or released replacement. +The safe default preserves public data-lifetime semantics, but the codec implementation itself is still a branch experiment. Re-test workload correctness and performance before moving to any later commit or released implementation.