tree 7dbe1e07dcfe78ce25a48a027808ca9523b8505c
parent 67e957b43a2fdc197018968b0adc2f6a9c04e961
author Andreas Rheinhardt <andreas.rheinhardt@gmail.com> 1585861675 +0200
committer Andreas Rheinhardt <andreas.rheinhardt@gmail.com> 1587411046 +0200

avformat/matroskaenc: Make ebml_num_size() more robust

Matroska (or actually EBML) uses variable-length numbers where only
seven bits of every byte is usable for the length; the other bits encode
the length of the variable-length number. So in order to find out how
many bytes one needs to encode a given number one can use a loop like
while (num >> 7 * bytes) bytes++; the Matroska muxer effectively did this.

Yet it has a disadvantage: It is impossible for the result of a single
right shift of an unsigned number with most significant bit set to be
zero, because one can only shift by 0..(width - 1). On some
architectures like x64 it is not even possible to do it with undefined
right shifts in which case this leads to an infinite loop.

This can be easily avoided by switching to a loop whose condition is
(num >>= 7). The maximum value the so modified function can return
is 10; any value > 8 is invalid and will now lead to an assert in
put_ebml_num() or in start_ebml_master() (or actually in
put_ebml_size_unknown()).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>