vaapi_h264: Scale log2_max_pic_order_cnt_lsb with max_b_frames
authorMark Thompson <sw@jkqxz.net>
Wed, 4 Jan 2017 23:05:10 +0000 (23:05 +0000)
committerMark Thompson <sw@jkqxz.net>
Wed, 11 Jan 2017 23:03:58 +0000 (23:03 +0000)
Before this change, it was possible to overflow pic_order_cnt_lsb and
generate a stream with invalid POC numbering.  This makes sure that
the field is large enough that a single IDR B* P sequence uses fewer
than half the available POC lsb values.

libavcodec/vaapi_encode_h264.c

index deb99a7..74e7cb1 100644 (file)
@@ -800,6 +800,8 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx)
         vseq->seq_fields.bits.direct_8x8_inference_flag = 1;
         vseq->seq_fields.bits.log2_max_frame_num_minus4 = 4;
         vseq->seq_fields.bits.pic_order_cnt_type = 0;
+        vseq->seq_fields.bits.log2_max_pic_order_cnt_lsb_minus4 =
+            av_clip(av_log2(avctx->max_b_frames + 1) - 2, 0, 12);
 
         if (avctx->width  != ctx->surface_width ||
             avctx->height != ctx->surface_height) {