opusfile by Xiph.Org Foundation, DoS vulnerability (SIGFPE)

[Alex Sarum <rum.274.4 () gmail com>]

Hi,

I was doing vulnerability research on a project that uses the library as a
dependency. During the research, I found a vulnerability that I have
already informed the vendor about. It's been a long time, but the vendor
hasn't passed the information on to Xiph.Org Foundation, so I decided to do
it myself. I created an issue[1], but still have not received a comment
about reproducibility or fixing the vulnerability.

A copy of the vulnerability details below.

One of the possible paths of execution:

./opusfile/opusfile.c: op_open_file -> op_open_close_on_failure ->
op_open_callbacks -> op_open2 -> op_open_seekable2 ->
op_open_seekable2_impl -> op_bisect_forward_serialno ->
op_predict_link_start

SIGFPE:
https://github.com/xiph/opusfile/blob/9d718345ce03b2fad5d7d28e0bcd1cc69ab2b166/src/opusfile.c#L1089

Trigger: crash.zip[2]

[1]: https://github.com/xiph/opusfile/issues/48
[2]: https://github.com/xiph/opusfile/files/14397558/crash.zip