Fix some broken clamping in rate control.
authorTimothy B. Terriberry <tterribe@xiph.org>
Wed, 14 Sep 2016 17:47:34 +0000 (10:47 -0700)
committerTimothy B. Terriberry <tterribe@xiph.org>
Fri, 23 Sep 2016 19:47:36 +0000 (12:47 -0700)
If we have some dupe frames right near the end of the buffer, our
 metric window can be slightly larger than the buffer.
However, the clamping we use to determine how many frames' worth of
 metrics to add to the buffer had a mix of signed and unsigned
 types, so if that number when negative, instead of clamping
 against zero, it would ask for all remaining frames in the file.
That would cause an infinite loop when we tried to find the last
 keyframe in the circular buffer in oc_enc_select_qi() (which was
 not big enough to actually hold that many frames).

This patch changes the clamp to a form that works with all unsigned
 values.

Thanks to Brion Vibber for the report.

Fixes #2229

Forward-port of r19507 from svn.

lib/rate.c

index 6de74d9..bf2b139 100644 (file)
@@ -1076,8 +1076,8 @@ int oc_enc_rc_2pass_in(oc_enc_ctx *_enc,unsigned char *_buf,size_t _bytes){
       else{
         int frames_needed;
         /*We're using a finite buffer:*/
-        frames_needed=OC_CLAMPI(0,_enc->rc.buf_delay
-         -(_enc->rc.scale_window_end-_enc->rc.scale_window0),
+        frames_needed=OC_MINI(_enc->rc.buf_delay-OC_MINI(_enc->rc.buf_delay,
+         _enc->rc.scale_window_end-_enc->rc.scale_window0),
          _enc->rc.frames_left[0]+_enc->rc.frames_left[1]
          -_enc->rc.nframes[0]-_enc->rc.nframes[1]);
         while(frames_needed>0){
@@ -1114,8 +1114,8 @@ int oc_enc_rc_2pass_in(oc_enc_ctx *_enc,unsigned char *_buf,size_t _bytes){
             _enc->rc.scale_window_end+=m->dup_count+1;
             /*Compute an upper bound on the number of remaining packets needed
                for the current window.*/
-            frames_needed=OC_CLAMPI(0,_enc->rc.buf_delay
-             -(_enc->rc.scale_window_end-_enc->rc.scale_window0),
+            frames_needed=OC_MINI(_enc->rc.buf_delay-OC_MINI(_enc->rc.buf_delay,
+             _enc->rc.scale_window_end-_enc->rc.scale_window0),
              _enc->rc.frames_left[0]+_enc->rc.frames_left[1]
              -_enc->rc.nframes[0]-_enc->rc.nframes[1]);
             /*Clear the buffer for the next frame.*/