xwddec: prevent overflow of lsize * avctx->height
authorAndreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fri, 18 Dec 2015 18:28:51 +0000 (19:28 +0100)
committerAnton Khirnov <anton@khirnov.net>
Wed, 13 Jan 2016 08:39:13 +0000 (09:39 +0100)
This is used to check if the input buffer is larger enough, so if this
overflows it can cause a false negative leading to a segmentation fault
in bytestream2_get_bufferu.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
libavcodec/xwddec.c

index f6d3d97..1c9874a 100644 (file)
@@ -141,7 +141,7 @@ static int xwd_decode_frame(AVCodecContext *avctx, void *data,
         return AVERROR_INVALIDDATA;
     }
 
-    if (bytestream2_get_bytes_left(&gb) < ncolors * XWD_CMAP_SIZE + avctx->height * lsize) {
+    if (bytestream2_get_bytes_left(&gb) < ncolors * XWD_CMAP_SIZE + (uint64_t)avctx->height * lsize) {
         av_log(avctx, AV_LOG_ERROR, "input buffer too small\n");
         return AVERROR_INVALIDDATA;
     }