Libde265 v1.0.12 was discovered that requested allocation size exceeds maximum supported size of 0x10000000000

[Frank-Z7]

Libde265 v1.0.12 was discovered that requested allocation size exceeds maximum supported size of 0x10000000000

Description

Libde265 v1.0.12 was discovered that requested allocation size 0xffffffffffff5b00 (0xffffffffffff6b00 after adjustments for alignment, red zones etc.) exceeds maximum supported size of 0x10000000000 (thread T0).

This vulnerability allows attackers to cause a Denial of Service (DoS) and cause the system storage space to be used up.

It is important to note that we recommend reproducing this vulnerability in a docker environment, as it is likely to affect your operating system and storage space!

The following two images show that my docker storage space was full at the time of fuzzing this vulnerability.

Version

libde265/dec265  v1.0.12

ASAN Log

./dec265/dec265 --noaccel --disable-deblocking --disable-sao -L poc2libde265

=================================================================
==69==ERROR: AddressSanitizer: requested allocation size 0xffffffffffff5b00 (0xffffffffffff6b00 after adjustments for alignment, red zones etc.) exceeds maximum supported size of 0x10000000000 (thread T0)
    #0 0x4c662d in operator new[](unsigned long) (/afltest/libde265/dec265/dec265+0x4c662d)
    #1 0x4c90d6 in convert_to_8bit(unsigned char const*, int, int, int, int) /afltest/libde265/dec265/dec265.cc:243:18
    #2 0x4c90d6 in display_sdl(de265_image const*) /afltest/libde265/dec265/dec265.cc:298:12
    #3 0x4cba68 in output_image(de265_image const*) /afltest/libde265/dec265/dec265.cc:353:12
    #4 0x4cd633 in main /afltest/libde265/dec265/dec265.cc:802:20
    #5 0x7ffff790d082 in __libc_start_main /build/glibc-BHL3KM/glibc-2.31/csu/../csu/libc-start.c:308:16

==69==HINT: if you don't care about these errors you may set allocator_may_return_null=1
SUMMARY: AddressSanitizer: allocation-size-too-big (/afltest/libde265/dec265/dec265+0x4c662d) in operator new[](unsigned long)
==69==ABORTING

Reproduction

./autogen.sh
export CFLAGS="-g -lpthread -fsanitize=address"
export CXXFLAGS="-g -lpthread -fsanitize=address"
CC=clang CXX=clang++ ./configure --disable-shared
make -j 32

./dec265/dec265 --noaccel --disable-deblocking --disable-sao -L poc2libde265

PoC

poc2libde265: https://github.com/Frank-Z7/z-vulnerabilitys/blob/main/poc2libde265

Reference

https://github.com/strukturag/libde265

Environment

ubuntu:20.04
gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.2)
clang version 10.0.0-4ubuntu1
afl-cc++4.09

Credit

Zeng Yunxiang
Song Jiaxuan

[farindk]

Thank you. Should be fixed with the above commit.

[Frank-Z7]

Thank you. Should be fixed with the above commit.

Glad to do it.