transport: fix error handling on Stream deletion (#1275)

This patch writes client-side error before closing the active stream
to fix blocking `RecvMsg` issue on `grpc.ClientStream` [1].

Previous gRPC client stream just exits on `ClientTransport.Error` [2].
And latest gRPC added another select case on client connection context
cancel [3]. Now when client stream closes from client connection context
cancel, it calls `CloseStream` with `ErrClientConnClosing` error. And then
the stream gets deleted from `*http2Client.activeStreams`, without processing
the error [4]. Then in-flight `RecvMsg` call on this client will block on
`*parser.Reader.recvMsg` [5].

In short,

1. `ClientConn.Close`.
2. in-flight streams will receive case `<-cc.ctx.Done()`
   https://github.com/grpc/grpc-go/blob/master/stream.go#L253-L255.
3. `cs.closeTransportStream(ErrClientConnClosing)` calls `cs.t.CloseStream(cs.s, err)`.
4. `CloseStream(cs.s, err)` calls `delete(t.activeStreams, s.id)`
   without handling the error.
5. in-flight streams will never receive error, left hanging.

I can reproduce in etcd tests with in-flight `recvMsg` calls to `Observe` RPC.

---
[1] https://github.com/coreos/etcd/pull/7896#issuecomment-305241742
[2] https://github.com/grpc/grpc-go/blob/v1.2.x/stream.go#L235-L238
[3] https://github.com/grpc/grpc-go/pull/1136
[4] https://github.com/grpc/grpc-go/blob/master/transport/http2_client.go#L569
[5] https://github.com/grpc/grpc-go/blob/master/rpc_util.go#L280

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2 files changed
tree: e05af374567de1a17ab01a39c265cdbf398dff80
  1. .github/
  2. benchmark/
  3. codes/
  4. credentials/
  5. Documentation/
  6. examples/
  7. grpclb/
  8. grpclog/
  9. health/
  10. internal/
  11. interop/
  12. keepalive/
  13. metadata/
  14. naming/
  15. peer/
  16. reflection/
  17. stats/
  18. status/
  19. stress/
  20. tap/
  21. test/
  22. testdata/
  23. transport/
  24. .travis.yml
  25. backoff.go
  26. backoff_test.go
  27. balancer.go
  28. balancer_test.go
  29. call.go
  30. call_test.go
  31. clientconn.go
  32. clientconn_test.go
  33. codec.go
  34. codec_benchmark_test.go
  35. codec_test.go
  36. codegen.sh
  37. CONTRIBUTING.md
  38. coverage.sh
  39. doc.go
  40. go16.go
  41. go17.go
  42. grpclb.go
  43. interceptor.go
  44. LICENSE
  45. Makefile
  46. PATENTS
  47. proxy.go
  48. proxy_test.go
  49. README.md
  50. rpc_util.go
  51. rpc_util_test.go
  52. server.go
  53. server_test.go
  54. stream.go
  55. trace.go
README.md

gRPC-Go

Build Status GoDoc

The Go implementation of gRPC: A high performance, open source, general RPC framework that puts mobile and HTTP/2 first. For more information see the gRPC Quick Start guide.

Installation

To install this package, you need to install Go and setup your Go workspace on your computer. The simplest way to install the library is to run:

$ go get google.golang.org/grpc

Prerequisites

This requires Go 1.6 or later.

Constraints

The grpc package should only depend on standard Go packages and a small number of exceptions. If your contribution introduces new dependencies which are NOT in the list, you need a discussion with gRPC-Go authors and consultants.

Documentation

See API documentation for package and API descriptions and find examples in the examples directory.

Performance

See the current benchmarks for some of the languages supported in this dashboard.

Status

General Availability Google Cloud Platform Launch Stages.

FAQ

Compiling error, undefined: grpc.SupportPackageIsVersion

Please update proto package, gRPC package and rebuild the proto files:

  • go get -u github.com/golang/protobuf/{proto,protoc-gen-go}
  • go get -u google.golang.org/grpc
  • protoc --go_out=plugins=grpc:. *.proto