feat(spanner): implement generation and propagation of "x-goog-spanner-request-id" Header (#11048)

* spanner: implement generation and propagation of "x-goog-spanner-request-id" Header

In tandem with the specification:

    https://orijtech.notion.site/x-goog-spanner-request-id-always-on-gRPC-header-to-aid-in-quick-debugging-of-errors-14aba6bc91348091a58fca7a505c9827

this change adds sending over the "x-goog-spanner-request-id" header
for every unary and streaming call, in the form:

	<version>.<processId>.<clientId>.<channelId>.<requestCountForClient>.<rpcCountForRequest>

where:
* version is the version of the specification
* processId is a randomly generated uint64 singleton for the lifetime of a process
* clientId is the monotonically increasing id/number of gRPC Spanner clients created
* requestCountForClient is the monotonically increasing number of requests made by the client
* channelId currently at 1 is the Id of the client for Go
* rpcCountForRequest is the number of RPCs/retries within a specific request

This header is to be sent on both unary and streaming calls and it'll
help debug latencies for customers. On an error, customers can assert against
.Error and retrieve the associated .RequestID and log it, or even better
it'll be printed out whenever errors are logged.

Importantly making randIdForProcess to be a uint6 which is 64bits and not
a UUID4 which is 128bits which surely massively reduces the possibility of collisions
to ensure that high QPS applications can function and accept bursts of traffic
without worry, as the prior design used uint32 aka 32 bits for
which just 50,000 new processes being created could get the probability
of collisions to 25%, with this new change a company would have to
create 82 million QPS every second for 1,000 years for a 1% collision
with 2.6e18 for which the collision would be 1%.
Using 64-bits still provides really good protection whereby for a 1% chance of collision,
we would need 810 million objects, so we have good protection.
However, Google Cloud Spanner's backend has to store every one of the always on
headers for a desired retention period hence 64-bits is a great balance between collision
protection vs storage.

Fixes #11073

* Rebase with main; rename nthRPC to attempt

* Infer channelID from ConnPool directly

* Attach nthRequest to sessionClient instead of to grpcClient given channelID is derived from sessionClient.connPool

* Retain reference to grpc.Header(*metadata.MD)

We have to re-insert the request-id even after gax.Invoke->grpc
internals clear it. Added test to validate retries.

* Fix up Error.Error() to show RequestID for both cases

* spanner: bring in tests contributed by Knut

* spanner: allow errors with grpc.codes: Canceled and DeadlineExceeded to be wrapped with request-id

* spanner: correctly track and increment retry attempts for each ExecuteStreamingSql request

* spanner: propagate RequestID even for DeadlineExceeded

* spanner: assert .RequestID exists

* Address code reivew nits+feedback

* spanner: account for stream resets and retries

This change accounts for logic graciously raised by Knut
along with his test contribution.

* Address more updates
13 files changed
tree: 4906ac7653939b66660e9dfa625225546dd9f79c
  1. .devcontainer/
  2. .github/
  3. accessapproval/
  4. accesscontextmanager/
  5. advisorynotifications/
  6. ai/
  7. aiplatform/
  8. alloydb/
  9. analytics/
  10. apigateway/
  11. apigeeconnect/
  12. apigeeregistry/
  13. apihub/
  14. apikeys/
  15. appengine/
  16. apphub/
  17. apps/
  18. area120/
  19. artifactregistry/
  20. asset/
  21. assuredworkloads/
  22. auth/
  23. automl/
  24. backupdr/
  25. baremetalsolution/
  26. batch/
  27. beyondcorp/
  28. bigquery/
  29. bigtable/
  30. billing/
  31. binaryauthorization/
  32. certificatemanager/
  33. channel/
  34. chat/
  35. civil/
  36. cloudbuild/
  37. cloudcontrolspartner/
  38. clouddms/
  39. cloudprofiler/
  40. cloudquotas/
  41. cloudtasks/
  42. commerce/
  43. compute/
  44. confidentialcomputing/
  45. config/
  46. contactcenterinsights/
  47. container/
  48. containeranalysis/
  49. datacatalog/
  50. dataflow/
  51. dataform/
  52. datafusion/
  53. datalabeling/
  54. dataplex/
  55. dataproc/
  56. dataqna/
  57. datastore/
  58. datastream/
  59. debugger/
  60. deploy/
  61. developerconnect/
  62. dialogflow/
  63. discoveryengine/
  64. dlp/
  65. documentai/
  66. domains/
  67. edgecontainer/
  68. edgenetwork/
  69. errorreporting/
  70. essentialcontacts/
  71. eventarc/
  72. filestore/
  73. firestore/
  74. functions/
  75. gkebackup/
  76. gkeconnect/
  77. gkehub/
  78. gkemulticloud/
  79. grafeas/
  80. gsuiteaddons/
  81. httpreplay/
  82. iam/
  83. iap/
  84. identitytoolkit/
  85. ids/
  86. internal/
  87. iot/
  88. kms/
  89. language/
  90. lifesciences/
  91. logging/
  92. longrunning/
  93. managedidentities/
  94. managedkafka/
  95. maps/
  96. mediatranslation/
  97. memcache/
  98. metastore/
  99. migrationcenter/
  100. monitoring/
  101. netapp/
  102. networkconnectivity/
  103. networkmanagement/
  104. networksecurity/
  105. networkservices/
  106. notebooks/
  107. optimization/
  108. oracledatabase/
  109. orchestration/
  110. orgpolicy/
  111. osconfig/
  112. oslogin/
  113. parallelstore/
  114. phishingprotection/
  115. policysimulator/
  116. policytroubleshooter/
  117. privatecatalog/
  118. privilegedaccessmanager/
  119. profiler/
  120. pubsub/
  121. pubsublite/
  122. rapidmigrationassessment/
  123. recaptchaenterprise/
  124. recommendationengine/
  125. recommender/
  126. redis/
  127. resourcemanager/
  128. resourcesettings/
  129. retail/
  130. rpcreplay/
  131. run/
  132. scheduler/
  133. secretmanager/
  134. securesourcemanager/
  135. security/
  136. securitycenter/
  137. securitycentermanagement/
  138. securityposture/
  139. servicecontrol/
  140. servicedirectory/
  141. servicehealth/
  142. servicemanagement/
  143. serviceusage/
  144. shell/
  145. shopping/
  146. spanner/
  147. speech/
  148. storage/
  149. storageinsights/
  150. storagetransfer/
  151. streetview/
  152. support/
  153. talent/
  154. telcoautomation/
  155. texttospeech/
  156. third_party/
  157. tpu/
  158. trace/
  159. translate/
  160. vertexai/
  161. video/
  162. videointelligence/
  163. vision/
  164. visionai/
  165. vmmigration/
  166. vmwareengine/
  167. vpcaccess/
  168. webrisk/
  169. websecurityscanner/
  170. workflows/
  171. workstations/
  172. .gitignore
  173. .release-please-manifest-individual.json
  174. .release-please-manifest-submodules.json
  175. .release-please-manifest.json
  176. CHANGES.md
  177. CODE_OF_CONDUCT.md
  178. CONTRIBUTING.md
  179. debug.md
  180. doc.go
  181. go.mod
  182. go.sum
  183. go.work
  184. go.work.sum
  185. LICENSE
  186. migration.md
  187. README.md
  188. release-please-config-individual.json
  189. release-please-config-yoshi-submodules.json
  190. release-please-config.json
  191. RELEASING.md
  192. SECURITY.md
  193. testing.md
README.md

Google Cloud Client Libraries for Go

Go Reference

Go packages for Google Cloud Platform services.

Installation

go get cloud.google.com/go/firestore@latest # Replace firestore with the package you want to use.

NOTE: Some of these packages are under development, and may occasionally make backwards-incompatible changes.

Supported APIs

For an updated list of all of our released APIs please see our reference docs.

Go Versions Supported

Note: As of Jan 1, 2025 the Cloud Client Libraries for Go will support the two most-recent major Go releases -- the same policy the Go programming language follows.

Our libraries are compatible with at least the three most recent, major Go releases. They are currently compatible with:

  • Go 1.23
  • Go 1.22
  • Go 1.21

Authorization

By default, each API will use Google Application Default Credentials for authorization credentials used in calling the API endpoints. This will allow your application to run in many environments without requiring explicit configuration.

client, err := storage.NewClient(ctx)

To authorize using a JSON key file, pass option.WithCredentialsFile to the NewClient function of the desired package. For example:

client, err := storage.NewClient(ctx, option.WithCredentialsFile("path/to/keyfile.json"))

You can exert more control over authorization by using the credentials package to create an auth.Credentials. Then pass option.WithAuthCredentials to the NewClient function:

creds := ...
client, err := storage.NewClient(ctx, option.WithAuthCredentials(creds))

Contributing

Contributions are welcome. Please, see the CONTRIBUTING document for details.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Contributor Code of Conduct for more information.

Links