pvq: fix artificial chroma boost with --enable-float-pvq
authorTristan Matthews <tmatth@videolan.org>
Wed, 16 Nov 2016 14:32:13 +0000 (09:32 -0500)
committerTristan Matthews <tmatth@videolan.org>
Wed, 16 Nov 2016 17:45:43 +0000 (12:45 -0500)
Fixes regression on --enable-float-pvq from commit
c774e25ecd051357e95f850520c61ba82dca3180.

Caught by David Michael Barr.

objective-fast1 (with/without --enable-float-pvq):
daala-fix-float-pvq-rounding2016-11-16T143315122Z -> master-2017-11-13-4403315

   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.0693 | -0.0757 |  0.1661 |  -0.0222 | -0.0493 | -0.0262 |    -0.0173

subset1 (with/without --enable-float-pvq):
daala-fix-pvq-float-rounding2016-11-16T163903172Z -> daala-master-2016-11-16T13-02-41-114Z

   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.1306 | -0.0399 |  0.0673 |  -0.1358 | -0.0611 | -0.0999 |    -0.0946

src/pvq.c

index 8f6a141..e4423bb 100644 (file)
--- a/src/pvq.c
+++ b/src/pvq.c
@@ -343,7 +343,7 @@ void od_init_qm(int16_t *x, int16_t *x_inv, const int *qm) {
           mag = OD_DEFAULT_MAG;
 #else
           /*FIXME: Do this rounding when generating OD_BASIS_MAG.*/
-          mag = (od_val32)floor(.5 + OD_DEFAULT_MAG*OD_BASIS_MAG[xydec][bs][i]*
+          mag = OD_ROUND32(OD_DEFAULT_MAG*OD_BASIS_MAG[xydec][bs][i]*
            OD_BASIS_MAG[xydec][bs][j]);
 #endif
           if (i == 0 && j == 0) {