pvq: fix artificial chroma boost with --enable-float-pvq
authorTristan Matthews <tmatth@videolan.org>
Wed, 16 Nov 2016 14:32:13 +0000 (09:32 -0500)
committerTristan Matthews <tmatth@videolan.org>
Wed, 16 Nov 2016 17:45:43 +0000 (12:45 -0500)
commit963d3a92e24d8474fc60c15cbaeb172c72d3f09a
treeedd75af648c942643110693ea714c6339b772eb2
parent7522712b94289f853227f17ea22d09cd1bbc5801
pvq: fix artificial chroma boost with --enable-float-pvq

Fixes regression on --enable-float-pvq from commit
c774e25ecd051357e95f850520c61ba82dca3180.

Caught by David Michael Barr.

objective-fast1 (with/without --enable-float-pvq):
daala-fix-float-pvq-rounding2016-11-16T143315122Z -> master-2017-11-13-4403315

   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.0693 | -0.0757 |  0.1661 |  -0.0222 | -0.0493 | -0.0262 |    -0.0173

subset1 (with/without --enable-float-pvq):
daala-fix-pvq-float-rounding2016-11-16T163903172Z -> daala-master-2016-11-16T13-02-41-114Z

   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.1306 | -0.0399 |  0.0673 |  -0.1358 | -0.0611 | -0.0999 |    -0.0946
src/pvq.c