tree d01482e9433bd065a36afc7f97756c8249184c0e
parent 4013f8d559720d4d0087cca0a3f18e0fdf25093d
author Steeve Morin <steeve.morin@gmail.com> 1494459625 +0200
committer dfawley <dfawley@google.com> 1494459625 -0700

Use pooled gzip.{Writer,Reader} in gzip{Compressor,Decompressor} (#1217)

This change saves a lot of memory by reusing the underlying
gzip.{Writer,Reader}, which allocates up to 1.4mb at every instanciation
according to [1]. This was fixed by adding a Reset method by to the
object at [2].

The amount of memory (and GC time) saved is pretty high, as reported by
pprof:

      flat  flat%   sum%        cum   cum%
   28.33GB 85.70% 85.70%    32.74GB 99.05%  compress/flate.NewWriter

      flat  flat%   sum%        cum   cum%
   19.39MB 16.74% 16.74%    22.07MB 19.05%  compress/flate.NewWriter

And the benchmarks:

benchmark                           old ns/op     new ns/op     delta
BenchmarkGZIPCompressor1B-4         215170        22291         -89.64%
BenchmarkGZIPCompressor1KiB-4       225971        27213         -87.96%
BenchmarkGZIPCompressor8KiB-4       246696        54785         -77.79%
BenchmarkGZIPCompressor64KiB-4      444851        286924        -35.50%
BenchmarkGZIPCompressor512KiB-4     2279043       2115863       -7.16%
BenchmarkGZIPCompressor1MiB-4       4412989       4258635       -3.50%

benchmark                           old allocs     new allocs     delta
BenchmarkGZIPCompressor1B-4         17             0              -100.00%
BenchmarkGZIPCompressor1KiB-4       17             0              -100.00%
BenchmarkGZIPCompressor8KiB-4       17             0              -100.00%
BenchmarkGZIPCompressor64KiB-4      17             0              -100.00%
BenchmarkGZIPCompressor512KiB-4     17             0              -100.00%
BenchmarkGZIPCompressor1MiB-4       17             0              -100.00%

benchmark                           old bytes     new bytes     delta
BenchmarkGZIPCompressor1B-4         813872        8             -100.00%
BenchmarkGZIPCompressor1KiB-4       813872        16            -100.00%
BenchmarkGZIPCompressor8KiB-4       813875        27            -100.00%
BenchmarkGZIPCompressor64KiB-4      813918        190           -99.98%
BenchmarkGZIPCompressor512KiB-4     814928        1871          -99.77%
BenchmarkGZIPCompressor1MiB-4       820889        9735          -98.81%

[1] https://github.com/golang/go/issues/6138
[2] https://github.com/golang/go/commit/db12f9d4e406dcab81b476e955c8e119112522fa

Signed-off-by: Steeve Morin <steeve.morin@gmail.com>