UBSAN: index out of bounds

[pietroborrello]

Describe the bug
Several UBSAN violations possibly related to #1289
The zip attached contains the testcases that trigger the violation

To Reproduce
Built stb according to the oss-fuzz script with CXXFLAGS='-O1 -fsanitize=address -fsanitize=array-bounds,bool,builtin,enum,float-divide-by-zero,function,integer-divide-by-zero,null,object-size,return,returns-nonnull-attribute,shift,signed-integer-overflow,unreachable,vla-bound,vptr'

Sample output

INFO: Seed: 1010447616
INFO: Loaded 1 modules   (6883 inline 8-bit counters): 6883 [0x5e1b33, 0x5e3616), 
INFO: Loaded 1 PC tables (6883 PCs): 6883 [0x573228,0x58e058), 
stbi_read_fuzzer: Running 1 inputs 1 time(s) each.
Running: id:000092,sig:06,src:002772+001250,time:5407822,op:splice,rep:8,trial:1492432
src/stb/tests/../stb_image.h:1990:10: runtime error: index 257 out of bounds for type 'stbi_uc [257]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/stb/tests/../stb_image.h:1990:10 in 
src/stb/tests/../stb_image.h:1991:4: runtime error: index 750 out of bounds for type 'stbi_uc [257]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/stb/tests/../stb_image.h:1991:4 in 
src/stb/tests/../stb_image.h:2001:13: runtime error: index 256 out of bounds for type 'stbi__uint16 [256]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/stb/tests/../stb_image.h:2001:13 in 
src/stb/tests/../stb_image.h:2000:17: runtime error: index 257 out of bounds for type 'stbi_uc [257]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/stb/tests/../stb_image.h:2000:17 in 
src/stb/tests/../stb_image.h:1999:11: runtime error: index 264 out of bounds for type 'stbi_uc [257]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/stb/tests/../stb_image.h:1999:11 in 
src/stb/tests/../stb_image.h:2015:18: runtime error: index 256 out of bounds for type 'stbi__uint16 [256]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/stb/tests/../stb_image.h:2015:18 in 
src/stb/tests/../stb_image.h:2013:15: runtime error: index 257 out of bounds for type 'stbi_uc [257]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/stb/tests/../stb_image.h:2013:15 in 
src/stb/tests/../stb_image.h:2115:4: runtime error: index 56695 out of bounds for type 'stbi_uc [257]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/stb/tests/../stb_image.h:2115:4 in 
AddressSanitizer:DEADLYSIGNAL
=================================================================
==1324703==ERROR: AddressSanitizer: SEGV on unknown address 0x629000019ebf (pc 0x000000510cd6 bp 0x7fffffffcfe0 sp 0x7fffffffcf70 T0)
==1324703==The signal is caused by a READ memory access.
    #0 0x510cd6 in stbi__jpeg_huff_decode(stbi__jpeg*, stbi__huffman*) (stbi_read_fuzzer+0x510cd6)
    #1 0x50fb8e in stbi__jpeg_decode_block_prog_ac(stbi__jpeg*, short*, stbi__huffman*, short*) (stbi_read_fuzzer+0x50fb8e)
    #2 0x508bee in stbi__parse_entropy_coded_data(stbi__jpeg*) (stbi_read_fuzzer+0x508bee)
    #3 0x505f3a in stbi__decode_jpeg_image(stbi__jpeg*) (stbi_read_fuzzer+0x505f3a)
    #4 0x500cd3 in load_jpeg_image(stbi__jpeg*, int*, int*, int*, int) (stbi_read_fuzzer+0x500cd3)
    #5 0x4d5ae3 in stbi__jpeg_load(stbi__context*, int*, int*, int*, int, stbi__result_info*) (stbi_read_fuzzer+0x4d5ae3)
    #6 0x4cf171 in stbi__load_main(stbi__context*, int*, int*, int*, int, stbi__result_info*, int) (stbi_read_fuzzer+0x4cf171)
    #7 0x4c9642 in stbi__load_and_postprocess_8bit(stbi__context*, int*, int*, int*, int) (stbi_read_fuzzer+0x4c9642)
    #8 0x4cadbc in stbi_load_from_memory (stbi_read_fuzzer+0x4cadbc)
    #9 0x4ced22 in LLVMFuzzerTestOneInput (stbi_read_fuzzer+0x4ced22)
    #10 0x531329 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (stbi_read_fuzzer+0x531329)
    #11 0x51c239 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (stbi_read_fuzzer+0x51c239)
    #12 0x521142 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (stbi_read_fuzzer+0x521142)
    #13 0x51bfc2 in main (stbi_read_fuzzer+0x51bfc2)
    #14 0x7ffff7a6b0b2 in __libc_start_main /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16
    #15 0x41e98d in _start (stbi_read_fuzzer+0x41e98d)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (stbi_read_fuzzer+0x510cd6) in stbi__jpeg_huff_decode(stbi__jpeg*, stbi__huffman*)
==1324703==ABORTING

Crashing files
ubsan-out-of-bounds.zip

[NBickford-NV]

I did a quick analysis of file id_000154,sig_06,src_002783+000969,time_39921237,op_splice,rep_4,trial_1492432 (which also crashes a Windows Address Sanitizer build) as part of putting together PR #1297.

This crashes in a slightly different place than in issue #1289: the immediate cause within stbi__jpeg_huff_decode is that
h->maxcode[k] is supposed to be monotonic, but gets into a state where it isn't.
As a result, k steps past the end of h->maxcode until it reaches some memory (at k == 31) that
happens to satisfy the condition. This passes the k == 17 check at the end, and
crashes in the line

c = ((j->code_buffer >> (32 - k)) & stbi__bmask[k]) + h->delta[k];

However, this doesn't answer how h->maxcode[k] became non-monotonic by itself. This seems to be ultimately the same mechanism as in issue #1289! The sum of the sizes array is greater than 256, so we get writes past the end of huff_dc[*].values or huff_ac[*].values both in the "DHT - define huffman table" block and in stbi__build_huffman.

[rygorous]

Tested and confirmed fixed by Neil's changes, now in dev branch. Will be fixed in the next release.

[rygorous]

Fixed in 2.28.