libav.git
5 years agoarm: Don't clobber callee saved registers in scalarproduct release/0.6 github/release/0.6 gitlab/release/0.6 gitorious/release/0.6 videolan/release/0.6
Martin Storsjö [Fri, 20 Dec 2013 13:02:35 +0000 (15:02 +0200)]
arm: Don't clobber callee saved registers in scalarproduct

q4-q7/d8-d15 are supposed to not be clobbered by the callee.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d307e408d4a9ada22df443cc38be77cc5e492694)

Signed-off-by: Martin Storsjö <martin@martin.st>
7 years agovorbis: Validate that the floor 1 X values contain no duplicates.
Alex Converse [Tue, 5 Jun 2012 01:27:03 +0000 (18:27 -0700)]
vorbis: Validate that the floor 1 X values contain no duplicates.

Duplicate values in this vector are explicitly banned by the Vorbis I spec
and cause divide-by-zero crashes later on.
(cherry picked from commit ecf79c4d3e8baaf2f303278ef81db6f8407656bc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 9aaaeba45c41cf2b3fa4100abbdee7437428f93c)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit d6e250abfc36b239ef0c1fc9d45d588b853bfcb9)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
7 years agolavfi: avfilter_merge_formats: handle case where inputs are same
Mina Nagy Zaki [Wed, 8 Jun 2011 16:24:25 +0000 (19:24 +0300)]
lavfi: avfilter_merge_formats: handle case where inputs are same

This fixes a double-free crash if lists are the same due to the two
merge_ref() calls at the end of the (useless) merging that happens.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 11b6a82412bcd372adf694a26d83b07d337e1325)

Conflicts:

libavfilter/formats.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit e5f4e249422834f727bcd432b73af971277f1371)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit b6c5848a1f8fc2755ea70d325acaddae9fac45ab)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
7 years agompegvideo: Don't use ff_mspel_motion() for vc1
Michael Niedermayer [Sun, 20 Nov 2011 16:19:25 +0000 (17:19 +0100)]
mpegvideo: Don't use ff_mspel_motion() for vc1

Using ff_mspel_motion assumes that s (a MpegEncContext
poiinter) really is a Wmv2Context.

This fixes crashes in error resilience on vc1/wmv3 videos.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 18f2d5cb9c48d06895960f37467576725c9dc2d1)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit da0c457663479bc1828918e1bb3e4a5e4de0d557)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 899d95efe12f1e250b361837c1c8c06df9ac9b86)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
7 years agoimgconvert: avoid undefined left shift in avcodec_find_best_pix_fmt
Janne Grunau [Mon, 2 Jul 2012 08:46:39 +0000 (10:46 +0200)]
imgconvert: avoid undefined left shift in avcodec_find_best_pix_fmt

CC: libav-stable@libav.org
(cherry picked from commit 39bb27bf79bc4c2d8beaed637a14176264cb1916)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 7a7229b52d1900279041991fadbd29b27e8dfe95)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 8812b5f164109553f009ce385e17a1af16b6ea53)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
7 years agonuv: check RTjpeg header for validity
Janne Grunau [Mon, 6 Aug 2012 11:59:04 +0000 (13:59 +0200)]
nuv: check RTjpeg header for validity

CC: libav-stable@libav.org
(cherry picked from commit 859a579e9bbf47fae2e09494c43bcf813dcb2fad)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 6704522ca9dd32c858ee474492be568c386910f9)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit f31170d4e7f9671e019315391160d454b18d7296)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
7 years agovc1dec: add flush function for WMV9 and VC-1 decoders
Kostya Shishkov [Thu, 27 Sep 2012 17:25:06 +0000 (19:25 +0200)]
vc1dec: add flush function for WMV9 and VC-1 decoders

CC: libav-stable@libav.org
(cherry picked from commit 4dc8c8386eef942dba35c4f2fb3210e22b511a5b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 02b72394627933dc8ce26445231a69f00dba491b)

Conflicts:
libavcodec/vc1dec.c

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 0173a7966b331105158a88f96b9afcc431d2fef8)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
7 years agomov: set AVCodecContext.width/height for h264
Mans Rullgard [Wed, 30 May 2012 03:06:00 +0000 (04:06 +0100)]
mov: set AVCodecContext.width/height for h264

This is required for correct cropping of files from Canon
cameras.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 8aa93e900449c88c3169ff5636fed03f41779cac)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 2fb4be9a99a2c2a9435339830e3d940171cc0d9b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 0054d70f23edd1f61a10a1c2c687b3a04831feb9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh264: allow cropping to AVCodecContext.width/height
Mans Rullgard [Wed, 30 May 2012 03:04:54 +0000 (04:04 +0100)]
h264: allow cropping to AVCodecContext.width/height

Override the frame size from the SPS with AVCodecContext values
if the latter specify a size smaller by less than one macroblock.
This is required for correct cropping of MOV files from Canon cameras.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 30f515091c323da59c0f1b533703dedca2f4b95d)

Conflicts:

libavcodec/h264.c
(cherry picked from commit e1608014c50eeb9f4744a53de0794eb6bb1269a2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit b102d5d97daedb717c023ec7bfa43047d97de284)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoRelease notes for 0.6.6 v0.6.6
Reinhard Tartler [Sat, 9 Jun 2012 10:05:53 +0000 (12:05 +0200)]
Release notes for 0.6.6

7 years agoUpdate changelog for 0.6.6 release
Derek Buitenhuis [Fri, 8 Jun 2012 19:20:14 +0000 (15:20 -0400)]
Update changelog for 0.6.6 release

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
7 years agoBump version number for 0.6.6 release.
Reinhard Tartler [Sun, 3 Jun 2012 20:42:54 +0000 (22:42 +0200)]
Bump version number for 0.6.6 release.

7 years agotqi: Pass errors from the MB decoder
Michael Niedermayer [Mon, 19 Dec 2011 03:13:37 +0000 (04:13 +0100)]
tqi: Pass errors from the MB decoder

This silences some valgrind warnings.
CC: libav-stable@libav.org
Fixes second half of http://ffmpeg.org/trac/ffmpeg/ticket/794
Bug found by: Oana Stratulat

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit f85334f58e1286287d0547a49fa9c93b40cbf48f)
(cherry picked from commit 90290a5150e84fb138ccde57657dc03830f08c1c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 5872580e65aab026b77754eb184f97ba7cc6ea35)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 2f2fd8c6d1c51a6b817e6c0bc4eff308b8f9cd18)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoea: check chunk_size for validity.
Ronald S. Bultje [Fri, 4 May 2012 23:06:26 +0000 (16:06 -0700)]
ea: check chunk_size for validity.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 273e6af47b38391f2bcc157cca0423fe7fcbf55c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 6a86b705e1d4b72f0dddfbe23ad3eed9947001d5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit e74bc64dd376c4691a610ba62a66ed30affc97ec)

Conflicts:

libavformat/electronicarts.c

7 years agopng: check bit depth for PAL8/Y400A pixel formats.
Ronald S. Bultje [Wed, 2 May 2012 17:58:55 +0000 (10:58 -0700)]
png: check bit depth for PAL8/Y400A pixel formats.

Wrong bit depth can lead to invalid rowsize values, which crashes the
decoder further down.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit d2205d6543881f2e6fa18c8a354bbcf91a1235f7)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit b8d6ba9d50e80fdce2ed74cdaffd4960df8a21c5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 33f93005f1a86c108302b4c5978aa1a3d8e092cc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agodxva2: define required feature selection macros
Kyle [Sat, 19 Feb 2011 00:42:11 +0000 (00:42 +0000)]
dxva2: define required feature selection macros

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 04973f8082c5a822112d2e42d535b7f3f59dccc0)
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
7 years agomingw32: merge checks for mingw-w64 and mingw32-runtime >= 3.15 into one
Ramiro Polla [Sun, 11 Jul 2010 22:31:41 +0000 (22:31 +0000)]
mingw32: merge checks for mingw-w64 and mingw32-runtime >= 3.15 into one

Originally committed as revision 24204 to svn://svn.ffmpeg.org/ffmpeg/trunk
(cherry picked from commit e26011d0f495de1148b8014995cbe923611b6b76)
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
7 years agomingw32: properly check if vfw capture is supported by the system headers
Ramiro Polla [Sun, 11 Jul 2010 22:17:17 +0000 (22:17 +0000)]
mingw32: properly check if vfw capture is supported by the system headers

Remove check for an specific w32api version, checking instead if vfw.h
supports vfw capture. The defines in w32api 3.12 were wrong, so this must be
accounted for in the check.

Originally committed as revision 24203 to svn://svn.ffmpeg.org/ffmpeg/trunk
(cherry picked from commit ec1ee802a2e1cb3317bd44851cc28f95b5916051)
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Conflicts:

configure

7 years agoconfigure: properly check for mingw-w64 through installed headers. mingw-w64 can...
Ramiro Polla [Sat, 10 Jul 2010 04:08:02 +0000 (04:08 +0000)]
configure: properly check for mingw-w64 through installed headers. mingw-w64 can also target 32-bit code.

Originally committed as revision 24156 to svn://svn.ffmpeg.org/ffmpeg/trunk
(cherry picked from commit 0a4307d6307516d333ce2cde2a2ffa0f50bc176c)
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
7 years agoqdm2: clip array indices returned by qdm2_get_vlc().
Ronald S. Bultje [Wed, 2 May 2012 16:12:46 +0000 (16:12 +0000)]
qdm2: clip array indices returned by qdm2_get_vlc().

Prevents subsequent overreads when these numbers are used as indices
in arrays.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 64953f67f98da2e787aeb45cc7f504390fa32a69)
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Conflicts:

libavcodec/qdm2.c

7 years agokmvc: Check palsize.
Alex Converse [Thu, 26 Jan 2012 16:30:49 +0000 (17:30 +0100)]
kmvc: Check palsize.

Fixes: CVE-2011-3952

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Based on fix by Michael Niedermayer
(cherry picked from commit 386741f887714d3e46c9e8fe577e326a7964037b)
(cherry picked from commit 416849f2e06227b1b4a451c392f100db1d709a0c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoshorten: Use separate pointers for the allocated memory for decoded samples.
Michael Niedermayer [Sun, 25 Dec 2011 11:28:50 +0000 (12:28 +0100)]
shorten: Use separate pointers for the allocated memory for decoded samples.

Fixes invalid free() if any of the buffers are not allocated due to either
not decoding a header or an error prior to allocating all buffers.

Fixes CVE-2012-0858
CC: libav-stable@libav.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 204cb29b3c84a74cbcd059d353c70c8bdc567d98)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 6fc3287b9ccece290c5881b92948772bbf72e68c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 96ed18cab1048f03ff1c825f46b25d49218f1da4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoshorten: check for realloc failure
Justin Ruggles [Thu, 15 Sep 2011 22:08:52 +0000 (18:08 -0400)]
shorten: check for realloc failure

(cherry picked from commit 9e5e2c2d010c05c10337e9c1ec9d0d61495e0c9c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit a207a2fecc6a77735ab0cf209fdba0b4dd942a86)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoshorten: Fix out of bound writes in fix_bitshift()
Laurent Aimar [Fri, 30 Sep 2011 01:26:22 +0000 (01:26 +0000)]
shorten: Fix out of bound writes in fix_bitshift()

The data pointers s->decoded[*] already take into account s->nwrap.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 5f05cf4ea9aaafed8edcabe785c2719786103ec1)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 737bea21b6c2c1d4dca0b7b18824c0a3205556d2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoshorten: Prevent block size from increasing
Laurent Aimar [Sun, 2 Oct 2011 00:48:12 +0000 (00:48 +0000)]
shorten: Prevent block size from increasing

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 95010d18b2d808db9a49377e41bc2f7cf4dfa03e)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 22949c42edf5352c5fa8c43870efe20698432b35)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoshorten: remove VLA and check for buffer overflow
Måns Rullgård [Sat, 26 Jun 2010 14:34:21 +0000 (14:34 +0000)]
shorten: remove VLA and check for buffer overflow

Originally committed as revision 23798 to svn://svn.ffmpeg.org/ffmpeg/trunk
(cherry picked from commit 02591641f88097aec2a573f0ae384c8b87bcfe3b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoadpcm: ADPCM Electronic Arts has always two channels
Janne Grunau [Thu, 5 Jan 2012 19:50:55 +0000 (20:50 +0100)]
adpcm: ADPCM Electronic Arts has always two channels

Fixes half of http://ffmpeg.org/trac/ffmpeg/ticket/794
Adresses CVE-2012-0852

(cherry picked from commit bb5b3940b08d8dad5b7e948e8f3b02cd2eb70716)

Conflicts:

libavcodec/adpcm.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit b581580bd1cc8506befa65b0a5c9ae429240f21f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh264: Add check for invalid chroma_format_idc
Alexander Strange [Sat, 24 Mar 2012 21:32:14 +0000 (17:32 -0400)]
h264: Add check for invalid chroma_format_idc

Fixes a crash when FF_DEBUG_PICT_INFO is used.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 6ef4063957aa5025c8d2cd757b6a537e4b6874df)

Fixes: CVE-2012-0851

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 47132345184dc3d0ff962a57a1225564fe979548)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit c5f7c755cfccd7aa01010a2d566104c2b0fa6d86)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoaacsbr: prevent out of bounds memcpy().
Alex Converse [Tue, 10 Jan 2012 21:07:09 +0000 (13:07 -0800)]
aacsbr: prevent out of bounds memcpy().

Fixes Libav Bug 195.
Fixes CVE-2012-0850

This doesn't make the code handle sample rate or upsample/downsample
change properly but this is still a good sanity check.

Based on change by Michael Niedermayer.

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 17ce52912f59a74ecc265e062578fb1181456e18)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 01804cc91ab231ac79092eee21325d7644357975)

Conflicts:

libavcodec/aacsbr.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agodpcm: ignore extra unpaired bytes in stereo streams.
Alex Converse [Fri, 17 Feb 2012 22:13:40 +0000 (14:13 -0800)]
dpcm: ignore extra unpaired bytes in stereo streams.

Fixes: CVE-2011-3951

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit ce7aee9b733134649a6ce2fa743e51733f33e67e)
(cherry picked from commit eaeaeb265fe46e1d81452960de918227541873b4)

Conflicts:

libavcodec/dpcm.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 1ce9c93198fc997e8f23934a78e2937af670e4e9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovqavideo: return error if image size is not a multiple of block size
Mans Rullgard [Mon, 23 Apr 2012 12:16:33 +0000 (13:16 +0100)]
vqavideo: return error if image size is not a multiple of block size

The decoder assumes in various places that the image size
is a multiple of the block size, and there is no obvious
way to support odd sizes.  Bailing out early if the header
specifies a bad size avoids various errors later on.

Fixes CVE-2012-0947.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 58b2e0f0f2fc96c1158e04f8aba95cbe6157a1a3)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit d5207e2af81580dd5e6277b354c8b459c3624f26)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit c71c77e56fcc6d469d45e1c8ce04aa053124d3f8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agocelp filters: Do not read earlier than the start of the 'out' vector.
Alex Converse [Fri, 4 May 2012 17:27:03 +0000 (10:27 -0700)]
celp filters: Do not read earlier than the start of the 'out' vector.

CC: libav-stable@libav.org
(cherry picked from commit 37ddd3833219fa7b913fff3f5cccc6878b047e6b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 9ea94c44b1b414ab3bc6e9220ebb77621423ca38)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 08c81f7365af96c1655767e68d6ec85bea50600c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agomotionpixels: Clip YUV values after applying a gradient.
Alex Converse [Wed, 2 May 2012 19:08:03 +0000 (12:08 -0700)]
motionpixels: Clip YUV values after applying a gradient.

Prevents illegal reads on truncated and malformed input.

CC: libav-stable@libav.org
(cherry picked from commit b5da848facd41169283d7bfe568b83bdfa7fc42e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit aaa6a666774eb02c351c84e80622a5c69e9b642e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 50073e2395522b6e2b8698ff0dd06ffaf8cbf8ce)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agomotionpixels: decode only the 111 complete frames for fate
Janne Grunau [Fri, 7 Oct 2011 16:08:55 +0000 (18:08 +0200)]
motionpixels: decode only the 111 complete frames for fate

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit c2f2dfb3dd20e036b8b08c0fd1486a3044e8f02a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 90d7146511db0e2dd2d2b1baf2ceb7177b30dd8d)

Conflicts:

tests/fate.mak
tests/ref/fate/motionpixels

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agokgv1dec: Increase offsets array size so it is large enough.
Michael Niedermayer [Wed, 25 Jan 2012 22:23:35 +0000 (23:23 +0100)]
kgv1dec: Increase offsets array size so it is large enough.

Fixes CVE-2011-3945

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 807a045ab7f51993a2c1b3116016cbbd4f3d20d6)

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit a02e8df973f5478ec82f4c507f5b5b191a5ecb6b)
(cherry picked from commit d5f2382d0389ed47a566ea536887af908bf9b14f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit a0b65938b7cf37680a4ce0667444a217a151c551)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agomjpegbdec: Fix overflow in SOS.
Alex Converse [Wed, 25 Jan 2012 21:39:24 +0000 (13:39 -0800)]
mjpegbdec: Fix overflow in SOS.

Based in part by a fix from Michael Niedermayer <michaelni@gmx.at>

Fixes CVE-2011-3947

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit b57d262412204e54a7ef8fa1b23ff4dcede622e5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 083a8a00373b12dc06b8ae4c49eec61fb5e55f4b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 6ae95a0b93e8df15fe5f364535a7214be0817736)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoatrac3: Fix crash in tonal component decoding.
Michael Niedermayer [Sat, 17 Dec 2011 02:18:58 +0000 (03:18 +0100)]
atrac3: Fix crash in tonal component decoding.

Add a check to avoid writing past the end of the channel_unit.components[]
array.

Bug Found by: cosminamironesei
Fixes CVE-2012-0853
CC: libav-stable@libav.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit c509f4f74713b035a06f79cb4d00e708f5226bc5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit f43b6e2b1ed47a1254a5d44c700a7fad5e9784be)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit f728ad26f0ec87650d2986a892785c0e2b97d161)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agodv: Fix small stack overread related to CVE-2011-3929 and CVE-2011-3936.
Alex Converse [Thu, 26 Jan 2012 23:08:26 +0000 (15:08 -0800)]
dv: Fix small stack overread related to CVE-2011-3929 and CVE-2011-3936.

Found with asan.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 2d1c0dea5f6b91bec7f5fa53ec050913d851e366)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 00fa6ffe1a0b252d6a81815e51f125225cd0b97a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agodv: Fix null pointer dereference due to ach=0
Michael Niedermayer [Tue, 24 Jan 2012 16:51:40 +0000 (17:51 +0100)]
dv: Fix null pointer dereference due to ach=0

dv: Fix null pointer dereference due to ach=0

Fixes part2 of CVE-2011-3929

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Roman Shaposhnik <roman@shaposhnik.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 5a396bb3a66a61a68b80f2369d0249729bf85e04)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 44e182d41e3a73548f3f5e8445ec428d3846e6d6)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agodv: check stype
Michael Niedermayer [Tue, 24 Jan 2012 16:48:23 +0000 (17:48 +0100)]
dv: check stype

dv: check stype

Fixes part1 of CVE-2011-3929
Possibly fixes part of CVE-2011-3936

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Roman Shaposhnik <roman@shaposhnik.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 635bcfccd439480003b74a665b5aa7c872c1ad6b)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit bb737d381f6d6413899a0697f426fb082eac66fc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agonsvdec: Propagate errors
Alex Converse [Fri, 27 Jan 2012 01:23:09 +0000 (17:23 -0800)]
nsvdec: Propagate errors

Related to CVE-2011-3940.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit c898431ca5ef2a997fe9388b650f658fb60783e5)

Conflicts:

libavformat/nsvdec.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 0100c4b1b0736e0f5b3c98f9b0ab8acbef574888)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agonsvdec: Be more careful with av_malloc().
Alex Converse [Fri, 27 Jan 2012 01:21:46 +0000 (17:21 -0800)]
nsvdec: Be more careful with av_malloc().

Check results for av_malloc() and fix an overflow in one call.

Related to CVE-2011-3940.

Based in part on work from Michael Niedermayer.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 8fd8a48263ff1437f9d02d7e78dc63efb9b5ed3a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit be524c186b50337db64d34a5726dfe3e8ea94f09)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agonsvdec: Fix use of uninitialized streams.
Michael Niedermayer [Tue, 24 Jan 2012 21:20:26 +0000 (22:20 +0100)]
nsvdec: Fix use of uninitialized streams.

Fixes CVE-2011-3940 (Out of bounds read resulting in out of bounds write)

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c011706bc752d34bc6ada31d7df2ca0c9af7c6b)

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 6a89b41d9780325ba6d89a37f2aeb925aa68e6a3)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 65beb8c1173906b0541442713cb29e8ba44c47ef)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoid3v2: fix skipping extended header in id3v2.4
Anton Khirnov [Sat, 31 Mar 2012 05:52:42 +0000 (07:52 +0200)]
id3v2: fix skipping extended header in id3v2.4

In v2.4, the length includes the length field itself.
(cherry picked from commit ddb4431208745ea270dce8fce4cba999f0ed4303)

Conflicts:

libavformat/id3v2.c

Signed-off-by: Anton Khirnov <anton@khirnov.net>
7 years agoRelease notes and changelog for 0.6.5 v0.6.5
Reinhard Tartler [Tue, 10 Jan 2012 20:03:20 +0000 (21:03 +0100)]
Release notes and changelog for 0.6.5

7 years agoBump version number for 0.6.5 release.
Reinhard Tartler [Tue, 10 Jan 2012 20:02:32 +0000 (21:02 +0100)]
Bump version number for 0.6.5 release.

7 years agovorbis: An additional defense in the Vorbis codec.
Chris Evans [Thu, 5 Jan 2012 20:25:41 +0000 (21:25 +0100)]
vorbis: An additional defense in the Vorbis codec.

Fixes Bug: #190
Chromium Bug: #100543
Related to CVE-2011-3893

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit afb2aa537954db537d54358997b68f46561fd5a7)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit b0283ccb9e8945ce9e56f7c6ba0c676e7179d7a3)

Conflicts:

libavcodec/vorbis_dec.c

7 years agovorbisdec: Fix decoding bug with channel handling
Reinhard Tartler [Thu, 5 Jan 2012 20:40:18 +0000 (21:40 +0100)]
vorbisdec: Fix decoding bug with channel handling

Fixes Bug: #191
Chromium Bug: #101458
CVE-2011-3895

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit e6d527ff729e42d80e4756cab779ff4ad693631b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 97f23c72a3815739ab28e297ce60f943349f6939)

Conflicts:

libavcodec/vorbis_dec.c

7 years agomatroskadec: Fix a bug where a pointer was cached to an array that might later move...
Chris Evans [Thu, 5 Jan 2012 20:19:30 +0000 (21:19 +0100)]
matroskadec: Fix a bug where a pointer was cached to an array that might later move due to a realloc()

Fixes bug #190
Chromium bug #100492
related to CVE-2011-3893

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry-picked from commit faaec4676cb4c7a2303d50df66c6290bc96a7657)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 1f625431e2bb9564760fba3ab8077ae07ce7c7a1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovorbis: Avoid some out-of-bounds reads
Chris Evans [Thu, 5 Jan 2012 20:25:41 +0000 (21:25 +0100)]
vorbis: Avoid some out-of-bounds reads

Fixes Bug: #190
Chromium Bug: #100543
Related to CVE-2011-3893

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 57cd6d709565e84e84385f8f2a9641ca3fa718be)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 4a94678f1be4b7d47f862e9523ca3358255da5d4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovp3: fix streams with non-zero last coefficient
Janne Grunau [Tue, 3 Jan 2012 12:38:01 +0000 (13:38 +0100)]
vp3: fix streams with non-zero last coefficient

Fixes a regression introduced in 8b94df0f2047e972.
(cherry picked from commit 9b4767e4784577f3107730316fe652ccaccd9b3a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 82a11fcff24d9827070d77f1a3c6ba5d4dc12984)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovp3: fix oob read for negative tokens and memleaks on error.
Ronald S. Bultje [Sat, 29 Oct 2011 06:50:04 +0000 (23:50 -0700)]
vp3: fix oob read for negative tokens and memleaks on error.
(cherry picked from commit 8370e426e42f2e4b9d14a1fb8107ecfe5163ce7f)

Fixes: #189
Chromium-Bug: 101172,100465
CVE-2011-3892

Removed the parts that are related to multi-threading, which is not
included before 0.7.

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit c624935554332f8921a15265b8720f0c7b3c8cc2)

Conflicts:

libavcodec/vp3.c

7 years agoRelease notes and changelog for 0.6.4 v0.6.4
Reinhard Tartler [Sun, 25 Dec 2011 08:41:03 +0000 (09:41 +0100)]
Release notes and changelog for 0.6.4

7 years agoBump version number for 0.6.4 release.
Reinhard Tartler [Sat, 24 Dec 2011 14:59:10 +0000 (15:59 +0100)]
Bump version number for 0.6.4 release.

7 years agoqdm2: check output buffer size before decoding
Justin Ruggles [Wed, 14 Sep 2011 17:57:04 +0000 (13:57 -0400)]
qdm2: check output buffer size before decoding

(cherry picked from commit 7d49f79f1cd47783a963a757a6563b9cac29db62)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 73472053516f82b7d273a3d42c583f894077a191)

Conflicts:

libavcodec/qdm2.c

7 years agoFix qdm2 decoder packet handling to match the api
Baptiste Coudurier [Fri, 19 Nov 2010 06:52:30 +0000 (06:52 +0000)]
Fix qdm2 decoder packet handling to match the api

Originally committed as revision 25767 to svn://svn.ffmpeg.org/ffmpeg/trunk

7 years ago4xm: Add a check in decode_i_frame to prevent buffer overreads
Shitiz Garg [Wed, 14 Dec 2011 12:59:21 +0000 (18:29 +0530)]
4xm: Add a check in decode_i_frame to prevent buffer overreads

Fixes bugzilla #135

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 355d917c0bd8163a3f1c7d4a6866dac749efdb84)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit d912a30c7d5cf9b8fdb26402804c9b0f999b4ff1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agowma: initialize prev_block_len_bits, next_block_len_bits, and block_len_bits.
Justin Ruggles [Tue, 22 Nov 2011 18:37:52 +0000 (13:37 -0500)]
wma: initialize prev_block_len_bits, next_block_len_bits, and block_len_bits.

The initial values are not checked against the number of block sizes.
Initializing them to frame_len_bits will result in a block size index of 0
in these cases instead of something that might be out-of-range.

Fixes Bug 81.
(cherry picked from commit 05d1e45d1f42cc90d1f2f36c546d0096cea126a8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 8dba5608dcf76032d8a9aa4bd8a3fc1392682281)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoswscale: #include "libavutil/mathematics.h"
Reinhard Tartler [Thu, 1 Dec 2011 17:48:33 +0000 (18:48 +0100)]
swscale: #include "libavutil/mathematics.h"

this file uses the M_PI macro since
4e74187db2f5db52f88729efc662df9d6bc763e1, so include the correct header
directly.

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 5089ce1b5abe2ecbbfd7235aeb0ad47ba38305c1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 851098c9e004b2ce294b687cb18633b038dcc3fe)

Conflicts:

libswscale/utils.c

7 years agovp3dec: Check coefficient index in vp3_dequant()
Reinhard Tartler [Sun, 4 Dec 2011 09:10:33 +0000 (10:10 +0100)]
vp3dec: Check coefficient index in vp3_dequant()

Based on a patch by Michael Niedermayer <michaelni@gmx.at>

Fixes NGS00145, CVE-2011-4352

Found-by: Phillip Langlois
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 8b94df0f2047e9728cb872adc9e64557b7a5152f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit bba709214a51ffd665a67404d3beb3727bb3f319)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agosvq1dec: call avcodec_set_dimensions() after dimensions changed.
Michael Niedermayer [Fri, 18 Nov 2011 18:10:21 +0000 (19:10 +0100)]
svq1dec: call avcodec_set_dimensions() after dimensions changed.

Fixes NGS00148, CVE-2011-4579

Found-by: Phillip Langlois
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e24b9488e67849a28e64a8056e05f83cf439229)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 0eca0da06e40b73af495cc05fbcfaa030fcf78ea)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovp6: Fix illegal read.
Thierry Foucu [Thu, 17 Nov 2011 17:39:52 +0000 (09:39 -0800)]
vp6: Fix illegal read.

Found with Address Sanitizer

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit e0966eb140b3569b3d6b5b5008961944ef229c06)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit ba4b08b78918f399f9c9524750b26e904d146078)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovp6: Fix illegal read.
Alex Converse [Thu, 3 Nov 2011 22:55:52 +0000 (15:55 -0700)]
vp6: Fix illegal read.

(cherry picked from commit 2a6eb06254df79e96b3d791b6b89b2534ced3119)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 67a7ed623b678a84c992dd7bf3e3d0329f83621b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovp6: Reset the internal state when aborting key frames header parsing
Laurent Aimar [Fri, 23 Sep 2011 20:36:11 +0000 (22:36 +0200)]
vp6: Reset the internal state when aborting key frames header parsing

It prevents leaving the state only half initialized.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit a72cad0a6c05aa74940101e937cb3dc602d7d67b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit c76505e0dee0890e39636ddebd2707ab3ea5b8de)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovp6: Check for huffman tree build errors
Laurent Aimar [Wed, 21 Sep 2011 18:46:32 +0000 (20:46 +0200)]
vp6: Check for huffman tree build errors

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 066fff755a5d8edc660c010ddb08474d208eeade)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 30c08e226156e5a36a835c008c67114f22c8da8f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovp6: partially propagate huffman tree building errors during coeff model parsing...
Dustin Brody [Tue, 16 Aug 2011 20:46:34 +0000 (16:46 -0400)]
vp6: partially propagate huffman tree building errors during coeff model parsing and fix misspelling

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit f913eeea43078b3b9052efd8d8d29e7b29b39208)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 7367cbec1b8cf0cbb49707fb0fdfded8ec397b0d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoFix out of bound reads in the QDM2 decoder.
Laurent Aimar [Fri, 30 Sep 2011 22:45:04 +0000 (00:45 +0200)]
Fix out of bound reads in the QDM2 decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 5a19acb17ceb71657b0eec51dac651953520e5c8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 0d93d5c4614fafea74bdac681673f5b32eb49063)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoCheck for out of bound writes in the QDM2 decoder.
Laurent Aimar [Fri, 30 Sep 2011 22:45:05 +0000 (00:45 +0200)]
Check for out of bound writes in the QDM2 decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 291d74a46d32183653db07818c7b3407fd50a288)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit a31ccacb1a9b2abc0e140a812fb0ffca6f7c2591)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovmd: fix segfaults on corruped streams
Laurent Aimar [Sun, 11 Sep 2011 17:17:45 +0000 (19:17 +0200)]
vmd: fix segfaults on corruped streams

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 494cfacdb9ba3f0549e37f76b3a2f86a7aeeac3c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agorv34: Check for invalid slice offsets
Laurent Aimar [Mon, 19 Sep 2011 20:48:53 +0000 (22:48 +0200)]
rv34: Check for invalid slice offsets

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 4cc7732386eb36661ed22d1200339b38a5fa60bc)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 2bbb142a140173e1870017b66c439f4d430a6f67)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agorv34: Fix potential overreads
Laurent Aimar [Sat, 17 Sep 2011 14:56:30 +0000 (16:56 +0200)]
rv34: Fix potential overreads

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit b4ed3d78cb6c41c9d3ee5918c326ab925edd6a89)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit b4a1bf0bbf53cc6a736a608732b2ac1de5c2447b)

Conflicts:

libavcodec/rv34.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agorv34: Avoid NULL dereference on corrupted bitstream
Laurent Aimar [Sat, 17 Sep 2011 21:43:58 +0000 (23:43 +0200)]
rv34: Avoid NULL dereference on corrupted bitstream

rv34_decode_slice() can return without allocating any pictures.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d0f6ab0298f2309c6104626787ed73416298b019)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agorv10: Reject slices that does not have the same type as the first one
Laurent Aimar [Sat, 17 Sep 2011 22:03:08 +0000 (00:03 +0200)]
rv10: Reject slices that does not have the same type as the first one

This prevents crashes with some corrupted bitstreams.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 4a29b471869353c3077fb4b25b6518eb1047afb7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 28d948ac44e38e8bec2f6268ccf4747ff4d992a9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agooggdec: fix out of bound write in the ogg demuxer
Laurent Aimar [Sun, 11 Sep 2011 21:26:12 +0000 (23:26 +0200)]
oggdec: fix out of bound write in the ogg demuxer

Between ogg_save() and ogg_restore() calls, the number of streams
could have been reduced.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 0e7efb9d23c3641d50caa288818e8c27647ce74d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit a3d471e500674c31fa4f52a62ef789d5e7fdbd3c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agosmacker: fix a few off by 1 errors
Michael Niedermayer [Tue, 13 Sep 2011 21:24:56 +0000 (23:24 +0200)]
smacker: fix a few off by 1 errors

stereo & 16bit is untested due to lack of samples

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 5166376f24545207607f61ed8ff4e1b0572ff320)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 78cd2e18a4aa2835f6d04cf145121fc82099c1a5)

Conflicts:

libavcodec/smacker.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoCheck for invalid VLC value in smacker decoder.
Laurent Aimar [Mon, 12 Sep 2011 21:49:36 +0000 (23:49 +0200)]
Check for invalid VLC value in smacker decoder.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 6489455495fc5bfbebcfe3f57e5d4fdd6a781091)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoCheck and propagate errors when VLC trees cannot be built in smacker decoder.
Laurent Aimar [Mon, 12 Sep 2011 21:46:49 +0000 (23:46 +0200)]
Check and propagate errors when VLC trees cannot be built in smacker decoder.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 9676ffba8346791f494451e68d2a3b37a2918a9b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoFixed off by one packet size allocation in the smacker demuxer.
Laurent Aimar [Mon, 12 Sep 2011 18:50:34 +0000 (20:50 +0200)]
Fixed off by one packet size allocation in the smacker demuxer.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit a92d0fa5d234582583d41b67dddecffc2c819573)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoCheck for invalid packet size in the smacker demuxer.
Laurent Aimar [Mon, 12 Sep 2011 18:50:13 +0000 (20:50 +0200)]
Check for invalid packet size in the smacker demuxer.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit e055932f5636a82275837968eea9c8fcb5bca474)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoape demuxer: fix segfault on memory allocation failure.
Laurent Aimar [Sun, 11 Sep 2011 17:17:40 +0000 (19:17 +0200)]
ape demuxer: fix segfault on memory allocation failure.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 273aab99bf7be2bcda95dd64101c2317ee0fcb99)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 4ee014309c377f7cfaa9578a393864ae500136f6)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoFixed size given to init_get_bits() in xan decoder.
Laurent Aimar [Fri, 9 Sep 2011 22:32:12 +0000 (00:32 +0200)]
Fixed size given to init_get_bits() in xan decoder.

(cherry picked from commit 393d5031c6aaaf8c2dda4eb5d676974c349fae85)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agosmacker demuxer: handle possible av_realloc() failure.
Kostya Shishkov [Mon, 12 Sep 2011 07:40:42 +0000 (09:40 +0200)]
smacker demuxer: handle possible av_realloc() failure.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 47a8589f7bc69d1a29da1dfdfbd0dfa78a9e31fd)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 0b9b3570a3e3f3eff088ee061dbab165ff3eff2f)

Conflicts:

libavformat/smacker.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoFixed segfault with wavpack decoder on corrupted decorrelation terms sub-blocks.
Laurent Aimar [Wed, 7 Sep 2011 19:43:03 +0000 (21:43 +0200)]
Fixed segfault with wavpack decoder on corrupted decorrelation terms sub-blocks.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 8bfea4ab4e2cb32bc7bf6f697ee30a238c65d296)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoindeo2: fail if input buffer too small
Alex Converse [Fri, 9 Sep 2011 20:26:49 +0000 (13:26 -0700)]
indeo2: fail if input buffer too small

(cherry picked from commit b7ce4f1d1c3add86ece7ca595ea6c4a10b471055)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoindeo2: init_get_bits size in bits instead of bytes
Alex Converse [Fri, 9 Sep 2011 20:24:19 +0000 (13:24 -0700)]
indeo2: init_get_bits size in bits instead of bytes

(cherry picked from commit 68ca330cbd479111db9cb7649d7530ad59f04cc8)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agowavpack: Check error codes rather than working around error conditions.
Alex Converse [Thu, 8 Sep 2011 18:02:43 +0000 (11:02 -0700)]
wavpack: Check error codes rather than working around error conditions.

(cherry picked from commit dba2b63a98bdcac7bda1a8a2c48950518c075e17)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 5d4c065476da547fd1a8a604e3047e1b3a7a29d8)

Conflicts:

libavcodec/wavpack.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoFixed invalid writes in wavpack decoder on corrupted bitstreams.
Laurent Aimar [Wed, 7 Sep 2011 20:17:39 +0000 (22:17 +0200)]
Fixed invalid writes in wavpack decoder on corrupted bitstreams.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 0aedab03405849962b469277afe047aa2c61a87f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 685940da4c843beb9283a21718cbd2fa4fa5d796)

Conflicts:

libavcodec/wavpack.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoFixed invalid access in wavpack decoder on corrupted bitstream.
Laurent Aimar [Wed, 7 Sep 2011 20:02:55 +0000 (22:02 +0200)]
Fixed invalid access in wavpack decoder on corrupted bitstream.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 55354b7de21e7bb4bbeb1c12ff55ea17f807c70c)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 4b84e995ad88f3bfa533c38218f2791c14fd72f0)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoFixed invalid access in wavpack decoder on corrupted extra bits sub-blocks.
Laurent Aimar [Wed, 7 Sep 2011 21:12:32 +0000 (23:12 +0200)]
Fixed invalid access in wavpack decoder on corrupted extra bits sub-blocks.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit beefafda639dd53fc59c21d8a7cf8334da9a1062)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agocpu detection: avoid a signed overflow
Sean McGovern [Mon, 25 Jul 2011 22:51:02 +0000 (18:51 -0400)]
cpu detection: avoid a signed overflow

1<<31 overflows because 1 is signed, so force it to unsigned.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 5938e02185430ca711106aaec9b5622dbf588af3)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh264: correct implicit weight table computation for long ref pics
Jeff Downs [Wed, 6 Jul 2011 15:54:36 +0000 (11:54 -0400)]
h264: correct implicit weight table computation for long ref pics

Correct computation of implicit weight tables when referencing pictures
that are marked for long reference.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 87cf70eb237e7586cc7399627dafa1b980ec0b7d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh264: correct the check for invalid long term frame index in MMCO decode
Jeff Downs [Tue, 5 Jul 2011 18:21:54 +0000 (14:21 -0400)]
h264: correct the check for invalid long term frame index in MMCO decode

The current check on MMCO parameters prohibits a "max long term frame index
plus 1" of 16 (frame idx of 15) for the "set max long term frame index" MMCO.
Fix this off-by-one error to allow the full range of legal values.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 29a09eae9a827f4dbc9c4517180d8fe2ecef321a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agorv10/20: tell decoder to use edge emulation
Kostya Shishkov [Wed, 17 Aug 2011 08:36:33 +0000 (10:36 +0200)]
rv10/20: tell decoder to use edge emulation

This removes out-of-edge motion compensation artifacts (easily spotted green
blocks in avplay, gray blocks in transcoding), for example here:
http://samples.libav.org/samples/real/tv_watching_t1.rm

Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 331971116d7d36743601bd2dc5384c5211d3bb48)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoflvenc: use int64_t to store offsets
Luca Barbato [Wed, 8 Jun 2011 14:32:07 +0000 (14:32 +0000)]
flvenc: use int64_t to store offsets

Metadata currently is written only at the start of the file in normal
cases, when transcoding from a rtmp source metadata could be
written later and the offset recorded can exceed 32bit.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 7f5bf4fbaf1f2142547321a16358f9871fabdcc6)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit fe3e7297fe56a383baca484dea2c0d603ae305f8)

Conflicts:

libavformat/flvenc.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoVC-1: fix reading of custom PAR.
Reimar Döffinger [Sat, 13 Aug 2011 09:58:18 +0000 (11:58 +0200)]
VC-1: fix reading of custom PAR.

Custom PAR num/denum are in 1-256 range.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 0e8696551414d4ea0aab2559f9475d1fe49d08f3)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh264: notice memory allocation failure
Dustin Brody [Thu, 11 Aug 2011 12:57:58 +0000 (08:57 -0400)]
h264: notice memory allocation failure

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit bac3ab13ea6a9dd8853e79ef3eacf51d234c8774)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 59a22afa0b50b9037133a7bc26bdc5023e7e1df9)

Conflicts:

libavcodec/h264.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agolibx264: do not set pic quality if no frame is output
Baptiste Coudurier [Sun, 30 Jan 2011 01:05:42 +0000 (17:05 -0800)]
libx264: do not set pic quality if no frame is output

Avoids uninitialized reads.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 5caa2de19ece830e32c95731bc92a423d55cff0c)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agomxfdec: Include FF_INPUT_BUFFER_PADDING_SIZE when allocating extradata.
Alex Converse [Fri, 29 Jul 2011 22:27:36 +0000 (15:27 -0700)]
mxfdec: Include FF_INPUT_BUFFER_PADDING_SIZE when allocating extradata.

This prevents out of bounds reads when extradata is being decoded.
(cherry picked from commit 1f6f58d5855288492fc2640a9f1035c01c75d356)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agorv30: return AVERROR(EINVAL) instead of EINVAL
Diego Biurrun [Thu, 21 Jul 2011 12:25:01 +0000 (14:25 +0200)]
rv30: return AVERROR(EINVAL) instead of EINVAL

On some platforms EINVAL could be positive, ensure we return negative values.
(cherry picked from commit e5985185d2eda942333ebbb72bd7d043ffe40be7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoDo not decode RV30 files if the extradata is too small
Rafaël Carré [Sat, 16 Jul 2011 15:41:08 +0000 (11:41 -0400)]
Do not decode RV30 files if the extradata is too small

Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 289c60001fb0a9a1d7a97c876d8a42b84c6874ac)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>