avcodec/hevc_mp4toannexb_bsf: Check NAL size against available input
The hevc_mp4toannexb bsf does not explicitly check whether a NAL unit
is so big that it extends beyond the end of the input packet; it does so
only implicitly by using the checked version of the bytestream2 API.
But this has downsides compared to real checks: It can lead to huge
allocations (up to 2GiB) even when the input packet is just a few bytes.
And furthermore it leads to uninitialized data being output.
So add a check to error out early if it happens.
Also check directly whether there is enough data for the length field.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit ea1b71e82f5a1752d59d3bfb9704092a79eba6b5)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
diff --git a/libavcodec/hevc_mp4toannexb_bsf.c b/libavcodec/hevc_mp4toannexb_bsf.c
index 30f733d..477d86d 100644
--- a/libavcodec/hevc_mp4toannexb_bsf.c
+++ b/libavcodec/hevc_mp4toannexb_bsf.c
@@ -141,10 +141,14 @@
int nalu_type;
int is_irap, add_extradata, extra_size, prev_size;
+ if (bytestream2_get_bytes_left(&gb) < s->length_size) {
+ ret = AVERROR_INVALIDDATA;
+ goto fail;
+ }
for (i = 0; i < s->length_size; i++)
nalu_size = (nalu_size << 8) | bytestream2_get_byte(&gb);
- if (nalu_size < 2) {
+ if (nalu_size < 2 || nalu_size > bytestream2_get_bytes_left(&gb)) {
ret = AVERROR_INVALIDDATA;
goto fail;
}
