commit | 133142449b22fc2799d922479514e677ae91f0f5 | [log] [tgz] |
---|---|---|

author | Søren Sandmann Pedersen <soren.sandmann@gmail.com> | Fri Apr 08 22:32:30 2016 -0400 |

committer | Søren Sandmann Pedersen <soren.sandmann@gmail.com> | Sat Sep 03 14:53:06 2016 -0400 |

tree | 554ebcd8b3b0d97f45bb3456154454659e9626eb | |

parent | 3b46fce6fec566e93a8a6b90df113272d203aafc [diff] |

pixman-filter: Fix several issues related to normalization There are a few bugs in the current normalization code (1) The normalization is based on the sum of the *floating point* values generated by integral(). But in order to get the sum to be close to pixman_fixed_1, the sum of the rounded fixed point values should be used. (2) The multiplications in the normalization loops often round the same way, so the residual error can fairly large. (3) The residual error is added to the sample located at index (width - width / 2), which is not the midpoint for odd widths (and for width 1 is in fact outside the array). This patch fixes these issues by (1) using the sum of the fixed point values as the total to divide by, (2) doing error diffusion in the normalization loop, and (3) putting any residual error (which is now guaranteed to be less than pixman_fixed_e) at the first sample, which is the only one that didn't get any error diffused into it. Signed-off-by: Søren Sandmann <soren.sandmann@gmail.com>

1 file changed