fix(datastore): add retries to emulator (#14591) **Summary** This PR refines the Datastore client's resilience logic to fix regressions identified during TAP testing, specifically regarding emulator startup races, Unix domain sockets, and IPv6 literals. It ensures that the resilience benefits for bug **[b/473841984](http://b/473841984)** are applied consistently to both production and emulator environments without breaking existing tests. **Why these changes are needed** 1. **Handling Emulator Startup Races**: Many tests (e.g., `exchange:datastore_test`) experience a "connection refused" error if an RPC is attempted immediately after starting the emulator process. By adding **`"waitForReady": true`** to the gRPC Service Config, the client will now block and wait for the emulator to finish binding its port instead of failing the RPC immediately. 2. **Preventing Infinite Hangs**: With `waitForReady` enabled, an unreachable host could cause a test to hang indefinitely. This PR adds a canonical **`"timeout": "60s"`** to the generic `methodConfig` to ensure RPCs eventually fail with a clear error. 3. **Robust Scheme Handling (Unix Sockets & IPv6)**: Internal test environments like `tin` often use **Unix domain sockets** (`unix:///`) or IPv6 literals (`[::1]`) for emulator addresses. Previous logic would mangle these into invalid targets (e.g., `passthrough:///unix:///...`), leading to "too many colons" errors or connection hangs. The updated logic defensively checks for existing valid schemes before applying the `passthrough:///` optimization. 4. **Idempotent Retries for `Commit`**: A catch-all configuration block is added to the Service Config to ensure that **`UNAVAILABLE`** errors are retried for all methods, including `Commit` (used by `PutMulti`), which was previously omitted. **Why PR #14411 was insufficient** While PR #14411 introduced Keepalives and expanded retries for idempotent calls, it had several gaps: * It only applied the new configuration to the **production** code path, leaving the emulator path without resilience logic or standardized retries. * By removing the manual 100ms retry loop without adding `waitForReady: true` to the Service Config, it removed the "wait" period needed for emulators to become ready. * The aggressive prepending of `passthrough:///` did not account for `unix:///` schemes or bracketed IPv6 literals, causing dialer errors in specialized test environments. (Even though the passthrough was not added in PR #14411, the current google3 imported code did not have this logic. While trying to import [cl/906545069](http://cl/906545069) latest code with psssthrough, this issue was discovered. ) **Related Bug** [b/473841984](http://b/473841984)
Go packages for Google Cloud Platform services.
go get cloud.google.com/go/firestore@latest # Replace firestore with the package you want to use.
NOTE: Some of these packages are under development, and may occasionally make backwards-incompatible changes.
For an updated list of all of our released APIs please see our reference docs.
Our libraries are compatible with the two most recent major Go releases, the same policy the Go programming language follows. This means the currently supported versions are:
By default, each client library will use Application Default Credentials (ADC) to automatically configure the credentials used in calling the API endpoint. When using the libraries in a Google Cloud Platform environment such as Compute Engine, Kubernetes Engine, or App Engine, no additional authentication steps are necessary. See Authentication methods at Google and Authenticate for using client libraries for more information.
client, err := storage.NewClient(ctx)
For applications running elsewhere, such as your local development environment, you can use the gcloud auth application-default login command from the Google Cloud CLI to set user credentials in your local filesystem. Application Default Credentials will automatically detect these credentials. See Set up ADC for a local development environment for more information.
Alternately, you may need to provide an explicit path to your credentials. To authenticate using a service account key file, either set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path to your key file, or programmatically pass option.WithCredentialsFile to the NewClient function of the desired package. For example:
client, err := storage.NewClient(ctx, option.WithCredentialsFile("path/to/keyfile.json"))
You can exert even more control over authentication by using the credentials package to create an auth.Credentials. Then pass option.WithAuthCredentials to the NewClient function:
creds, err := credentials.DetectDefault(&credentials.DetectOptions{...})
...
client, err := storage.NewClient(ctx, option.WithAuthCredentials(creds))
Contributions are welcome. Please, see the CONTRIBUTING document for details.
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Contributor Code of Conduct for more information.