lame: multiple left shift

["Agostino Sarubbo" <ago () gentoo org>]

Description:
lame is a high quality MPEG Audio Layer III (MP3) encoder licensed under the LGPL.

Few notes before the details of this bug. Time ago a fuzz was done by Brian Carpenter and Jakub Wilk which posted the 
results on the debian 
bugtracker. In cases like this, when upstream is not active and people do not post on the upstream bugzilla is easy 
discover duplicates, so I 
downloaded all available testcases, and noone of the bug you will see on my blog is a duplicate of an existing issue. 
Upstream seems a bit 
dead, latest release was into 2011, so this blog post will probably forwarded on the upstream bugtracker just for the 
record.

The complete ASan output of the issue:

# lame -f -V 9 $FILE out.wav
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:263:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:265:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:266:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:267:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:268:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:269:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:271:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:272:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:273:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:274:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:276:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:277:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:278:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:279:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/VbrTag.c:280:5: runtime error: left shift of 
negative value -1
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:845:48: runtime error: left shift of 
negative value -18
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:848:52: runtime error: left shift of 
negative value -10
Reproducer:
https://github.com/asarubbo/poc/blob/master/00295-lame-leftshift1
CVE:
N/A

#######################################

# lame -f -V 9 $FILE out.wav
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:848:52: runtime error: left shift of 
negative value -29398
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/libmp3lame/bitstream.c:181:50: runtime error: left shift 
of 45389699 by 6 places 
cannot be represented in type 'int'
Reproducer:
https://github.com/asarubbo/poc/blob/master/00296-lame-leftshift2
CVE:
N/A

#######################################

# lame -f -V 9 $FILE out.wav
/var/tmp/portage/media-sound/lame-3.99.5-r1/work/lame-3.99.5/frontend/get_audio.c:1195:52: runtime error: left shift of 
255 by 24 places 
cannot be represented in type 'int'
Reproducer:
https://github.com/asarubbo/poc/blob/master/00297-lame-leftshift3
CVE:
N/A

#######################################

Affected version:
3.99.5

Fixed version:
N/A

Commit fix:
N/A

Credit:
These bugs were discovered by Agostino Sarubbo of Gentoo.

Timeline:
2017-06-01: bug discovered
2017-06-17: blog post about the issue

Note:
These bugs were found with American Fuzzy Lop.

Permalink:
https://blogs.gentoo.org/ago/2017/06/17/lame-multiple-left-shift/

--
Agostino Sarubbo
Gentoo Linux Developer