daala.git
20 months agothis should be safe exp_fixed_pvq8
Jean-Marc Valin [Fri, 5 Feb 2016 14:57:08 +0000 (09:57 -0500)]
this should be safe

20 months agoshift
Jean-Marc Valin [Fri, 5 Feb 2016 03:42:10 +0000 (22:42 -0500)]
shift

20 months agoPVQ: Quantizing input and reference to 16 bits
Tristan Matthews [Wed, 3 Feb 2016 22:15:27 +0000 (17:15 -0500)]
PVQ: Quantizing input and reference to 16 bits

ntt-short1:
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -0.197569 -0.015300 0.114378
 PSNRHVS -0.179562 -0.021735 0.345082
    SSIM -0.300684 -0.035082 0.140643
FASTSSIM 0.023627 0.092786 -0.191730

subset1: [0.05 0.2 0.5 1] bpp
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -0.008094 0.032327 0.023671
 PSNRHVS -0.015952 0.019637 0.058469
    SSIM -0.019402 0.023681 0.019000
FASTSSIM -0.050832 -0.012060 -0.470097

Signed-off-by: Jean-Marc Valin <jmvalin@jmvalin.ca>
20 months agoPVQ: change qm to Q11
Tristan Matthews [Wed, 3 Feb 2016 22:19:47 +0000 (17:19 -0500)]
PVQ: change qm to Q11

ntt-short1:
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR 0.040491 -0.036410 0.105655
 PSNRHVS 0.044918 0.039912 0.101922
    SSIM 0.065253 -0.074332 0.065741
FASTSSIM 0.030335 -0.185887 0.227473

subset1: [0.05 0.2 0.5 1] bpp
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -0.009935 -0.003732 -0.000482
 PSNRHVS -0.011805 -0.020754 -0.014471
    SSIM 0.014639 0.004848 -0.004597
FASTSSIM 0.097470 -0.010453 0.028947

Signed-off-by: Jean-Marc Valin <jmvalin@jmvalin.ca>
20 months agoComputing the base (quantizer-dependent) deringing threshold outside od_dering()
Jean-Marc Valin [Fri, 29 Jan 2016 16:52:05 +0000 (11:52 -0500)]
Computing the base (quantizer-dependent) deringing threshold outside od_dering()

20 months agoFree memory if od_dec_init fails.
Nathan E. Egge [Fri, 29 Jan 2016 01:47:11 +0000 (20:47 -0500)]
Free memory if od_dec_init fails.

20 months agoAdd 10 bit support to yuv2yuv4mpeg.
Thomas Daede [Wed, 13 Jan 2016 01:03:21 +0000 (17:03 -0800)]
Add 10 bit support to yuv2yuv4mpeg.

20 months agostate: assert that debug file was actually opened before writing
Tristan Matthews [Thu, 28 Jan 2016 20:32:56 +0000 (15:32 -0500)]
state: assert that debug file was actually opened before writing

20 months agoconst fixes in the SSE2 deringing code
Jean-Marc Valin [Thu, 28 Jan 2016 19:56:39 +0000 (14:56 -0500)]
const fixes in the SSE2 deringing code

20 months agoMaking the skip overlap an option in od_dering()
Jean-Marc Valin [Thu, 28 Jan 2016 19:19:45 +0000 (14:19 -0500)]
Making the skip overlap an option in od_dering()

20 months agoAdds VPx defines
Jean-Marc Valin [Thu, 28 Jan 2016 16:25:45 +0000 (11:25 -0500)]
Adds VPx defines

20 months agoEnable 8-point lapping in dcttest.
Timothy B. Terriberry [Wed, 27 Jan 2016 23:51:45 +0000 (15:51 -0800)]
Enable 8-point lapping in dcttest.

This makes sure we'll catch any overflows, giving us the option to
 turn 8-point lapping back on someday.

20 months agoFix overflows in 64x64 DCT with 8-point lapping.
Timothy B. Terriberry [Wed, 27 Jan 2016 23:34:57 +0000 (15:34 -0800)]
Fix overflows in 64x64 DCT with 8-point lapping.

We don't have 8-point lapping enabled, but it would be nice to
 allow our transforms to support it in case we want to (it is still
 a gain on keyframes/still images).

This has exactly zero impact on measured MSE in dcttest.
There is a very small impact on the reported biases, but the first
 several significant figures of each one do not change (thanks to
 Nathan Egge for helping with the testing).

The metrics impact on ntt-short-1 is a little bigger, but still
 reasonable:

         RATE (%)  DSNR (dB)
    PSNR 0.04584  -0.00138
 PSNRHVS 0.05040  -0.00237
    SSIM 0.05538  -0.00127
FASTSSIM 0.03483  -0.00094

20 months agoFix DCT dynamic range computations.
Timothy B. Terriberry [Wed, 27 Jan 2016 22:18:12 +0000 (14:18 -0800)]
Fix DCT dynamic range computations.

They would completely fail if OD_MAX_FILT_SIZE was set to anything
 but 0 before this.

20 months ago Make 'luma-only' options in metric really compute luma only.
Yushin Cho [Thu, 28 Jan 2016 17:21:35 +0000 (09:21 -0800)]
 Make 'luma-only' options in metric really compute luma only.

20 months agoPre-built single vtable for deringing in C
Jean-Marc Valin [Thu, 28 Jan 2016 15:12:33 +0000 (10:12 -0500)]
Pre-built single vtable for deringing in C

20 months agoAdds consts to deringing code
Jean-Marc Valin [Thu, 28 Jan 2016 02:55:47 +0000 (21:55 -0500)]
Adds consts to deringing code

20 months agoReplace references to enc->curr_img with enc->state.
Nathan E. Egge [Thu, 28 Jan 2016 00:31:16 +0000 (19:31 -0500)]
Replace references to enc->curr_img with enc->state.

20 months agoFixes the tools unix Makefile broken in 4db55959b878f5ea2
Jean-Marc Valin [Thu, 28 Jan 2016 00:05:17 +0000 (19:05 -0500)]
Fixes the tools unix Makefile broken in 4db55959b878f5ea2

20 months agoRemove unnecessary curr_img pointer.
Nathan E. Egge [Wed, 27 Jan 2016 23:20:05 +0000 (18:20 -0500)]
Remove unnecessary curr_img pointer.

The only way that curr_img is referenced in the od_decode methods is to
 get the plane xdec and ydec which are also available in dec->state.info.

20 months agoUse od_output_queue instead of output_img[].
Nathan E. Egge [Wed, 27 Jan 2016 19:46:24 +0000 (14:46 -0500)]
Use od_output_queue instead of output_img[].

This patch replaces the output_img[] array and all of the implicit logic
 that manages which frame should be output based on the frame type with
 a simple circular queue based on a signaled frame number.
This is done in preparation of supporting more complicated frame orders
 like pyramid B-frames.

20 months agochanging od_coeff to int16_t in deringing filter
Jean-Marc Valin [Wed, 27 Jan 2016 18:36:32 +0000 (13:36 -0500)]
changing od_coeff to int16_t in deringing filter

20 months agoDeringing filter no longer needs state.h
Jean-Marc Valin [Wed, 27 Jan 2016 18:13:48 +0000 (13:13 -0500)]
Deringing filter no longer needs state.h

20 months agorenames direction_offsets_table and od_thresh_table_q8
Jean-Marc Valin [Wed, 27 Jan 2016 17:03:26 +0000 (12:03 -0500)]
renames direction_offsets_table and od_thresh_table_q8

20 months agoMoving the deringing filter to dering.[ch]
Jean-Marc Valin [Wed, 27 Jan 2016 08:06:32 +0000 (03:06 -0500)]
Moving the deringing filter to dering.[ch]

Making it as self-contained as possible

20 months agoRemove unused curr_frame from od_enc_ctx.
Nathan E. Egge [Wed, 27 Jan 2016 18:30:09 +0000 (13:30 -0500)]
Remove unused curr_frame from od_enc_ctx.

20 months agoAdd a curr_img pointer to reference the output image.
Nathan E. Egge [Wed, 27 Jan 2016 18:08:05 +0000 (13:08 -0500)]
Add a curr_img pointer to reference the output image.

This patch removes references to the output frame structure found in the
 various od_decode functions by replacing them with a daala_image
 pointer to the currently decoding frame.
This is done in preparation for moving to use an od_output_queue.

20 months agoReplace dec_order_count with first_frame flag.
Nathan E. Egge [Wed, 27 Jan 2016 15:34:29 +0000 (10:34 -0500)]
Replace dec_order_count with first_frame flag.

The only place we use the decode frame number is infer the output
 display order in daala_decode_img_out().
This is being replaced by explicitly signaling the display order
 when using an od_output_queue.

20 months agoRemove unused out_imgs_id[].
Nathan E. Egge [Wed, 27 Jan 2016 14:58:51 +0000 (09:58 -0500)]
Remove unused out_imgs_id[].

The out_imgs_id[] array holds the display order of frames as they are
 decoded.
This functionality is being replaced by the use of an od_output_queue.

20 months agoFix double keyframe bug.
Nathan E. Egge [Tue, 26 Jan 2016 13:23:52 +0000 (08:23 -0500)]
Fix double keyframe bug.

The od_input_queue_batch() code was incorrectly inserting an extra
 keyframe every keyframe_rate.

20 months agoSignal the derinring threshold
Jean-Marc Valin [Thu, 21 Jan 2016 22:50:35 +0000 (17:50 -0500)]
Signal the derinring threshold

The deringing filter is changed to operate on a superblock basis to limit the
signalling cost.

subset1 results: low = 0.05-0.1, medium = 0.1-0.2, high = 0.2-0.5
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -0.494899 -0.487463 -0.757194
 PSNRHVS -0.301645 -0.044495 -0.078095
    SSIM -0.767848 -0.314205 -0.200530
FASTSSIM -1.209264 -0.479098 0.676856

ntt-short1 results:
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -1.154509 -1.565189 -1.868308
 PSNRHVS -0.555466 -0.264924 -0.008079
    SSIM -0.817276 -0.665336 -0.727965
FASTSSIM 0.904384 2.292124 3.605684

20 months agopvq: remove logging
Tristan Matthews [Thu, 21 Jan 2016 22:13:07 +0000 (17:13 -0500)]
pvq: remove logging

20 months agoReorders the CfL flip bit in the bitstream
Jean-Marc Valin [Thu, 21 Jan 2016 20:57:25 +0000 (15:57 -0500)]
Reorders the CfL flip bit in the bitstream

This is done so that we can always call od_pvq_compute_gain() on the
already flipped reference. Without that, we need od_pvq_compute_gain()
to return exacty the same result for ref and -ref, which makes the fixed-point
more difficult to write.

20 months agopvq_encoder: fix typo
Tristan Matthews [Thu, 21 Jan 2016 21:33:03 +0000 (16:33 -0500)]
pvq_encoder: fix typo

20 months agofixes qm scaling
Jean-Marc Valin [Wed, 20 Jan 2016 20:50:05 +0000 (15:50 -0500)]
fixes qm scaling

ntt-short1:
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -0.003285 -0.034771 -0.003112
 PSNRHVS -0.003348 -0.052157 -0.030843
    SSIM -0.059175 -0.076170 0.008238
FASTSSIM -0.126060 0.031565 -0.017549

21 months agoGet rid of the bilinear filter
Jean-Marc Valin [Tue, 19 Jan 2016 23:18:26 +0000 (18:18 -0500)]
Get rid of the bilinear filter

Its enhancement effect has been almost completely eliminated by a combination
of the deringing filter and 64x64 DCTs so that right now it's mostly a
loss.

subset1 results: low = 0.05-0.1, medium = 0.1-0.2, high = 0.2-0.5
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -0.497171 -0.175219 -0.060687
 PSNRHVS -0.366111 -0.160004 -0.095976
    SSIM -0.683048 -0.336761 -0.128636
FASTSSIM -3.630780 -2.687704 -1.982495

No ntt-short1 since AWCY is broken, but the bilinear filter was only enabled
on keyframes anyway (which was already a problem).

21 months agoencode: fix parentheses warnings
Tristan Matthews [Thu, 14 Jan 2016 02:45:28 +0000 (21:45 -0500)]
encode: fix parentheses warnings

Fixes #188

21 months agoRename od_img to daala_image
Vittorio Giovara [Fri, 8 Jan 2016 14:08:40 +0000 (15:08 +0100)]
Rename od_img to daala_image

Currently od_ is reserved for private types only.

21 months agoNew frame QP modulation in conventional style.
Yushin Cho [Tue, 12 Jan 2016 19:10:12 +0000 (11:10 -0800)]
New frame QP modulation in conventional style.

The idea is to have larger delta QP for lower bit rates when doing frame QP
modulation.
Assume that the base_frame_QP is the coded quantizer.
Then,
I frame QP = base_frame_QP - 2
P frame QP = base_frame_QP * 1.05
B frame QP = base_frame_QP * 1.1 + 1

Note that increasing the multiplication factors (1.05 and 1.1) can give us more
coding gains but also significant regression.

           RATE (%)  DSNR (dB)
    PSNR -1.89545  0.05734
 PSNRHVS -1.70937  0.08181
    SSIM -1.93016  0.04450
FASTSSIM -2.21168  0.06100

          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -4.271001 -1.118354 -0.158924
 PSNRHVS -3.860607 -0.849587 -0.199489
    SSIM -4.401081 -0.995457 -0.215219
FASTSSIM -4.946268 -1.404133 -0.465403

21 months agoCleanup of od_enc_init() and daala_enc_ctx struct.
Nathan E. Egge [Wed, 6 Jan 2016 01:55:41 +0000 (20:55 -0500)]
Cleanup of od_enc_init() and daala_enc_ctx struct.

Now that the encoder contains an input queue of buffered frames, we can
 remove the old input_img buffer from the daala_enc_ctx struct.
In od_enc_init() memory is now allocated for the input queue inside
 od_input_queue_init() so the only image memory allocation needed is
 for dumping images with --enable-dump-images which is now done inside a
 single if defined(OD_DUMP_IMAGES) block.

21 months agoAdded an input_queue to cache input frames.
Nathan E. Egge [Fri, 18 Dec 2015 00:58:47 +0000 (19:58 -0500)]
Added an input_queue to cache input frames.

Centralized all code that deals with caching input frames into an
 od_input_queue struct that returns frames in encode order.
Updated the daala_encode_img_in() and daala_encode_packet_out() APIs to
 use this structure.
Simplified encoder_example to have less crazy state management in
 fetch_and_process_video().

21 months agoChange encoder_example to read a hex serial.
Nathan E. Egge [Tue, 5 Jan 2016 20:47:25 +0000 (15:47 -0500)]
Change encoder_example to read a hex serial.

Change serial from unsigned int to int to match ogg_stream_init() API.
Use %i to read serial number provided by -s, this allows for passing
 hex values prefixed with 0x.

21 months agoAdd missing edits to address comments on ad170a97.
Nathan E. Egge [Tue, 5 Jan 2016 17:19:26 +0000 (12:19 -0500)]
Add missing edits to address comments on ad170a97.

21 months agoSplit frame encoding out of daala_encode_img_in().
Nathan E. Egge [Mon, 28 Dec 2015 20:31:08 +0000 (15:31 -0500)]
Split frame encoding out of daala_encode_img_in().

Created an od_frame_encode() function that contains all the frame
 encoding logic from daala_encode_img_in().
This is done in preparation for landing code that manages an input
 queue of frames.

21 months agoReplace od_mm128i_mullo_epi32 in od_mc_predict1fmv16_sse2.
Kyle Siefring [Sat, 2 Jan 2016 00:30:29 +0000 (19:30 -0500)]
Replace od_mm128i_mullo_epi32 in od_mc_predict1fmv16_sse2.

21 months agoAdd sse2 code for FPR satd.
Kyle Siefring [Sat, 2 Jan 2016 00:25:17 +0000 (19:25 -0500)]
Add sse2 code for FPR satd.

21 months agoAdd curr_img to od_enc_ctx to point at currently encoding frame.
Nathan E. Egge [Wed, 23 Dec 2015 16:28:09 +0000 (11:28 -0500)]
Add curr_img to od_enc_ctx to point at currently encoding frame.

This is done in preparation of landing code that buffers more than two
 input frames.

21 months agoRefactor encoder_example to split frame fetching.
Nathan E. Egge [Wed, 23 Dec 2015 16:00:04 +0000 (11:00 -0500)]
Refactor encoder_example to split frame fetching.

Split the frame fetching code into its own function fetch() separate from
 fetch_and_process_video().

21 months agoRemove redundant input image checks.
Nathan E. Egge [Mon, 28 Dec 2015 19:41:42 +0000 (14:41 -0500)]
Remove redundant input image checks.

This removes extra checks on the input frame dimensions in
 daala_encode_img_in() that should have been removed in 0ad86e9a.

21 months agoRewrite od_img_copy_pad() so it takes a dst od_img.
Nathan E. Egge [Thu, 24 Dec 2015 00:05:11 +0000 (19:05 -0500)]
Rewrite od_img_copy_pad() so it takes a dst od_img.

Rewriting od_img_copy_pad() so that it uses an arbitrary od_img as a
 copy destination instead of depending on internal od_enc_ctx state.
This is being done so that od_img_copy_pad() can later be used to copy
 input images into a buffer of frames to encode.

21 months agoAdded an output_queue to cache decoded frames.
Nathan E. Egge [Wed, 23 Dec 2015 15:14:19 +0000 (10:14 -0500)]
Added an output_queue to cache decoded frames.

21 months agoSeparate OD_DUMP_RECONS from OD_DUMP_IMAGES.
Nathan E. Egge [Wed, 23 Dec 2015 14:11:41 +0000 (09:11 -0500)]
Separate OD_DUMP_RECONS from OD_DUMP_IMAGES.

21 months agoadd parentheses to OD_PLANE_SZ macro
Tristan Matthews [Tue, 22 Dec 2015 20:19:32 +0000 (15:19 -0500)]
add parentheses to OD_PLANE_SZ macro

21 months agoadd OD_PLANE_SZ macro
Tristan Matthews [Wed, 16 Dec 2015 21:19:23 +0000 (16:19 -0500)]
add OD_PLANE_SZ macro

Fixes #179

22 months agoDon't waste bits on mv:ref when the two references are the same
Jean-Marc Valin [Sat, 19 Dec 2015 02:25:46 +0000 (21:25 -0500)]
Don't waste bits on mv:ref when the two references are the same

           RATE (%)  DSNR (dB)
    PSNR -0.01599  0.00049
 PSNRHVS -0.01745  0.00083
    SSIM  0.00444 -0.00011
FASTSSIM -0.03153  0.00087

22 months agoMaking it possible to use only one predictor again
Jean-Marc Valin [Sat, 19 Dec 2015 01:58:16 +0000 (20:58 -0500)]
Making it possible to use only one predictor again

22 months agoRemoving TODO on OD_LUMA_QM_Q4 now 64x64 has been tuned.
Nathan E. Egge [Fri, 18 Dec 2015 23:32:05 +0000 (18:32 -0500)]
Removing TODO on OD_LUMA_QM_Q4 now 64x64 has been tuned.

22 months agoAdd fpr flag.
Michael Bebenita [Fri, 18 Dec 2015 22:24:17 +0000 (14:24 -0800)]
Add fpr flag.

22 months agoAddressing nit from 572f15e2ba, aka review 1162
Jean-Marc Valin [Fri, 18 Dec 2015 22:01:58 +0000 (17:01 -0500)]
Addressing nit from 572f15e2ba, aka review 1162

22 months agoTune activity masking QM for 32x32 and 64x64
David Michael Barr [Fri, 18 Dec 2015 21:52:10 +0000 (16:52 -0500)]
Tune activity masking QM for 32x32 and 64x64

Hand tuned by fixing block size and toggling activity masking with the goal of
minimizing the difference in bitrate for each band.

ntt-short-1:
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR 0.126975 -0.053934 0.361041
 PSNRHVS -0.106454 -0.238131 -0.083673
    SSIM -0.251859 0.564909 0.183929
FASTSSIM -2.056769 -3.544245 -1.402435

Signed-off-by: Jean-Marc Valin <jmvalin@jmvalin.ca>
22 months agoFlat initialization for the cdf of valid flags
Jean-Marc Valin [Fri, 18 Dec 2015 15:31:13 +0000 (10:31 -0500)]
Flat initialization for the cdf of valid flags

ntt-short-1:
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -0.139648 -0.091977 -0.026785
 PSNRHVS -0.161864 -0.056156 -0.043681
    SSIM -0.170633 -0.026771 -0.005387
FASTSSIM -0.245701 -0.074333 0.045202

22 months agoCorrect/clarify comment before od_enc_determine_frame_type()
Monty Montgomery [Fri, 18 Dec 2015 16:20:36 +0000 (11:20 -0500)]
Correct/clarify comment before od_enc_determine_frame_type()

22 months agopvq: don't signal Q scaling by default
Tristan Matthews [Fri, 18 Dec 2015 00:48:37 +0000 (19:48 -0500)]
pvq: don't signal Q scaling by default

It's currently unimplemented.

22 months agoUpdate enc->frame_delay in daala_encode_ctl().
Nathan E. Egge [Fri, 18 Dec 2015 01:30:33 +0000 (20:30 -0500)]
Update enc->frame_delay in daala_encode_ctl().

22 months agoReordering flags so that zeros cause a skip
Jean-Marc Valin [Wed, 16 Dec 2015 22:54:08 +0000 (17:54 -0500)]
Reordering flags so that zeros cause a skip

Skip values 0 and 1 swapped with 2 and 3.

          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -0.036983 0.045017 -0.047151
 PSNRHVS -0.099898 -0.053653 -0.224311
    SSIM -0.110367 -0.007900 -0.119557
FASTSSIM -0.355404 0.059345 -0.003413

22 months agoencode: tweak od_compute_dist() for lower rate
Tristan Matthews [Wed, 16 Dec 2015 20:22:02 +0000 (15:22 -0500)]
encode: tweak od_compute_dist() for lower rate

Interpolate from 1.7 to 1.2 for coded_q=36 to coded_q=47

ntt-short1:
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -1.625286 -0.664816 0.010636
 PSNRHVS -1.222277 -0.221461 0.000836
    SSIM -1.940892 -0.658784 0.008855
FASTSSIM -2.263498 -0.693365 0.007891

         RATE (%)  DSNR (dB)
    PSNR -0.74377  0.02299
 PSNRHVS -0.54053  0.02597
    SSIM -0.94424  0.02256
FASTSSIM -0.90805  0.02531

22 months agoDisable MC refinement at low complexity settings.
Timothy B. Terriberry [Thu, 17 Dec 2015 02:44:08 +0000 (18:44 -0800)]
Disable MC refinement at low complexity settings.

Running at complexity 4 (compared to the default, 7) causes roughly
 a 70% reduction in encode time with the following quality penalty:

           RATE (%)  DSNR (dB)
    PSNR 12.26178  -0.36017
 PSNRHVS 11.94026  -0.54861
    SSIM 11.48287  -0.26299
FASTSSIM  9.46425  -0.25445

Both numbers will probably be larger with FPR and B-frames enabled.

22 months agoEnable 64x64 blocks.
Nathan E. Egge [Tue, 22 Sep 2015 14:07:26 +0000 (10:07 -0400)]
Enable 64x64 blocks.

ntt-short-1:

         RATE (%)  DSNR (dB)
    PSNR -1.49191  0.04668
 PSNRHVS -1.60946  0.07829
    SSIM -1.32128  0.03204
FASTSSIM -1.58318  0.04510

subset-1:

         RATE (%)  DSNR (dB)
    PSNR -1.07725  0.04733
 PSNRHVS -0.99019  0.06959
    SSIM -0.73592  0.02708
FASTSSIM -0.88688  0.03044

22 months agoTruncate 64-point DCT constants to avoid overflow.
Nathan E. Egge [Wed, 7 Oct 2015 03:16:45 +0000 (23:16 -0400)]
Truncate 64-point DCT constants to avoid overflow.

22 months agoAdd integer 64-point DCT.
Nathan E. Egge [Tue, 25 Aug 2015 17:01:46 +0000 (13:01 -0400)]
Add integer 64-point DCT.

Added a bit-exact, integer, reversible, orthonormal 32-point DCT based
 on Loeffler's factorization with 215 adds, 38 shifts and 87 fixed-point
 multiplies.
This patch does not enable testing of the 64-point DCT because does not
 use truncated constants and is known to overflow.

MSE: 0.00019605494433023361528292682898211

22 months agoAdding the true 64-point Type-II DCT basis constants.
Nathan E. Egge [Tue, 25 Aug 2015 16:05:19 +0000 (12:05 -0400)]
Adding the true 64-point Type-II DCT basis constants.

22 months agoMove to 64x64 superblocks.
Nathan E. Egge [Thu, 6 Aug 2015 18:02:28 +0000 (14:02 -0400)]
Move to 64x64 superblocks.

ntt-short-1:

          RATE (%)  DSNR (dB)
    PSNR  0.38760  -0.01200
 PSNRHVS  0.08639  -0.00417
    SSIM  0.09467  -0.00226
FASTSSIM  0.42415  -0.01188

subset-1:

          RATE (%)  DSNR (dB)
    PSNR  0.01334  -0.00059
 PSNRHVS -0.02729   0.00188
    SSIM -0.13009   0.00479
FASTSSIM  0.01064  -0.00036

22 months agoRevert accidental change commited in 7928e790.
Nathan E. Egge [Tue, 15 Dec 2015 22:08:45 +0000 (17:08 -0500)]
Revert accidental change commited in 7928e790.

22 months agopvq: improve packing
Tristan Matthews [Mon, 14 Dec 2015 22:04:50 +0000 (17:04 -0500)]
pvq: improve packing

Uses 5440 bytes per QM array instead of 16384.

22 months agoRevert "Fix for encoder check, back to original version."
Yushin Cho [Tue, 15 Dec 2015 02:12:16 +0000 (18:12 -0800)]
Revert "Fix for encoder check, back to original version."

This reverts commit 2de27cfebab0c315a61a9592baf1e0d910e47030.

22 months agoFix for encoder check, back to original version.
Yushin Cho [Mon, 14 Dec 2015 20:08:43 +0000 (12:08 -0800)]
Fix for encoder check, back to original version.

22 months agoencode: use OD_LOSSLESS
Tristan Matthews [Mon, 14 Dec 2015 00:05:06 +0000 (19:05 -0500)]
encode: use OD_LOSSLESS

22 months agopvq: fix integer overflow
Tristan Matthews [Sun, 13 Dec 2015 23:32:20 +0000 (18:32 -0500)]
pvq: fix integer overflow

22 months agoFix analyzer to show frames in display order.
Nathan E. Egge [Fri, 11 Dec 2015 18:36:06 +0000 (13:36 -0500)]
Fix analyzer to show frames in display order.

22 months agotools: fix Makefile for upsample
Tristan Matthews [Fri, 11 Dec 2015 21:49:57 +0000 (16:49 -0500)]
tools: fix Makefile for upsample

22 months agopvq: remove od_apply_qm
Tristan Matthews [Thu, 8 Oct 2015 19:11:12 +0000 (15:11 -0400)]
pvq: remove od_apply_qm

Instead, apply this scaling at the PVQ level. Fixes #135.

subset1:

         RATE (%)  DSNR (dB)
    PSNR -0.18631  0.00820
 PSNRHVS -0.14242  0.01004
    SSIM -0.17364  0.00642
FASTSSIM -0.20725  0.00703

ntt-short-1:

         RATE (%)  DSNR (dB)
    PSNR 0.00592  -0.00020
 PSNRHVS 0.03283  -0.00160
    SSIM -0.06473  0.00156
FASTSSIM 0.06960  -0.00195

Note that when FPR is turned on, we will see a larger regression
on FastSSIM on video due to this patch. This is because this patch has less
noise on P-frames, which hurts FastSSIM.

This patch also helps chroma-from-luma, since before we were predicting
blocks with od_apply_qm applied to them from neighbouring blocks which
had it unapplied.

22 months agoanalyzer: fix parantheses warnings
Basar Koc [Fri, 11 Dec 2015 19:33:46 +0000 (14:33 -0500)]
analyzer: fix parantheses warnings

22 months agodump_psnrhvs: fix -Wabsolute-value
Tristan Matthews [Fri, 11 Dec 2015 13:05:51 +0000 (08:05 -0500)]
dump_psnrhvs: fix -Wabsolute-value

Fixes "using floating point absolute value function 'fabs'
when argument is of integer type".

22 months agodownsample: fix -Wtautological-compare
Tristan Matthews [Fri, 11 Dec 2015 13:01:48 +0000 (08:01 -0500)]
downsample: fix -Wtautological-compare

22 months agoMatch the samples used by AWCY.
Nathan E. Egge [Wed, 7 Oct 2015 14:48:47 +0000 (10:48 -0400)]
Match the samples used by AWCY.

22 months agoRevert "bidir: fix for context decision for mv block split pattern."
Yushin Cho [Thu, 10 Dec 2015 22:08:26 +0000 (17:08 -0500)]
Revert "bidir: fix for context decision for mv block split pattern."

This reverts commit 5c3ae64857301e44292044590f030af6a9f025ae.

22 months agobidir: fix for context decision for mv block split pattern.
Yushin Cho [Thu, 10 Dec 2015 20:09:37 +0000 (15:09 -0500)]
bidir: fix for context decision for mv block split pattern.
For 2 B-frames, this gives rate change of:
           RATE (%)  DSNR (dB)
    PSNR -0.00508  0.00015
 PSNRHVS 0.00074  -0.00003
    SSIM -0.00176  0.00004
FASTSSIM 0.00272  -0.00007

22 months agoBidir: Fix on calculating equal_mvs for mv predictor in B frame.
Yushin Cho [Thu, 10 Dec 2015 20:07:25 +0000 (15:07 -0500)]
Bidir: Fix on calculating equal_mvs for mv predictor in B frame.
For 2 B-frames, this gives rate change of:
           RATE (%)  DSNR (dB)
    PSNR 0.07932  -0.00236
 PSNRHVS 0.07082  -0.00332
    SSIM 0.07414  -0.00168
FASTSSIM 0.06983  -0.00188

22 months agoMissed change for Tim's code review on b frame.
Yushin Cho [Thu, 10 Dec 2015 19:42:52 +0000 (14:42 -0500)]
Missed change for Tim's code review on b frame.

22 months agotools: fix parentheses warnings
Basar Koc [Thu, 10 Dec 2015 18:51:44 +0000 (13:51 -0500)]
tools: fix parentheses warnings

22 months agostate: fix parentheses warnings
Basar Koc [Thu, 10 Dec 2015 18:51:30 +0000 (13:51 -0500)]
state: fix parentheses warnings

22 months agoRemove extra parenthesis caused by removing shifts of negatives.
Kyle Siefring [Thu, 10 Dec 2015 18:33:49 +0000 (13:33 -0500)]
Remove extra parenthesis caused by removing shifts of negatives.

22 months agopvq: don't repeat conditional expression
Tristan Matthews [Thu, 10 Dec 2015 16:29:25 +0000 (11:29 -0500)]
pvq: don't repeat conditional expression

22 months agodecode: don't clobber encoder's reconstructed output
Tristan Matthews [Wed, 9 Dec 2015 22:33:53 +0000 (17:33 -0500)]
decode: don't clobber encoder's reconstructed output

22 months agoDon't attempt to update the prediction outside the viewable region
Jean-Marc Valin [Fri, 4 Dec 2015 07:21:39 +0000 (02:21 -0500)]
Don't attempt to update the prediction outside the viewable region

ntt-short1:
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -1.741754 -1.677933 -1.973686
 PSNRHVS -1.842773 -1.798926 -2.005522
    SSIM -1.825643 -1.719517 -1.947608
FASTSSIM -1.868060 -1.710595 -1.625155

22 months agoDon't do the {pre,post}filter split on the edge of the image
Jean-Marc Valin [Thu, 3 Dec 2015 22:10:43 +0000 (17:10 -0500)]
Don't do the {pre,post}filter split on the edge of the image

ntt-short1:
          LOW (%)  MEDIUM (%) HIGH (%)
    PSNR -0.136692 -0.087764 0.035574
 PSNRHVS -0.136282 -0.067894 -0.001412
    SSIM -0.101104 -0.054236 0.025419
FASTSSIM -0.082263 0.052124 0.152271

22 months agoencode: fix the overflow at coded quantizer 63
Basar Koc [Wed, 9 Dec 2015 16:05:34 +0000 (11:05 -0500)]
encode: fix the overflow at coded quantizer 63

22 months agoencoder_example: remove unused variable
Basar Koc [Wed, 9 Dec 2015 16:03:38 +0000 (11:03 -0500)]
encoder_example: remove unused variable

22 months agoRemove unused variables added with b-frames.
Nathan E. Egge [Wed, 9 Dec 2015 16:00:50 +0000 (11:00 -0500)]
Remove unused variables added with b-frames.

22 months agoencode: fix encoder_check for B-frames
Tristan Matthews [Tue, 8 Dec 2015 22:40:02 +0000 (17:40 -0500)]
encode: fix encoder_check for B-frames