json:parse_bjdata_fuzzer reaches assertion

[nlohmann]

Description

The input 0x5b, 0x24, 0x5b triggers an assertion in the fuzzer for BJData.

• https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47145
• https://oss-fuzz.com/testcase-detail/5643490366717952

Reproduction steps

• Call the fuzzer on input 0x5b, 0x24, 0x5b.
• Observe the assertion.

Expected vs. actual results

No assertion should be triggered. Either the fuzzer must be made more robust against or there is a bug in the library that must be fixed.

Minimal code example

Adapted fuzzer:

#include <nlohmann/json.hpp>

using json = nlohmann::json;

int main() {
    std::vector<std::uint8_t> vec1 = {{0x5b, 0x24, 0x5b}};

    json j1 = json::from_bjdata(vec1);

    try
    {
        // step 2.1: round trip without adding size annotations to container types
        std::vector<uint8_t> vec2 = json::to_bjdata(j1, false, false);

        // step 2.2: round trip with adding size annotations but without adding type annonations to container types
        std::vector<uint8_t> vec3 = json::to_bjdata(j1, true, false);

        // step 2.3: round trip with adding size as well as type annotations to container types
        std::vector<uint8_t> vec4 = json::to_bjdata(j1, true, true);

        // parse serialization
        json j2 = json::from_bjdata(vec2);
        json j3 = json::from_bjdata(vec3);
        json j4 = json::from_bjdata(vec4);

        // serializations must match
        assert(json::to_bjdata(j2, false, false) == vec2);
        assert(json::to_bjdata(j3, true, false) == vec3);
        assert(json::to_bjdata(j4, true, true) == vec4);
    }
    catch (const json::parse_error&)
    {
        // parsing a BJData serialization must not fail
        assert(false);
    }
}

Error messages

Assertion

// parsing a BJData serialization must not fail
assert(false);

is triggered. In the original context, the following stack trace is produced:

	+----------------------------------------Release Build Stacktrace----------------------------------------+
Command: /mnt/scratch0/clusterfuzz/resources/platform/linux/unshare -c -n /mnt/scratch0/clusterfuzz/bot/builds/clusterfuzz-builds_json_26b1464c0c18fac23c49bf26ed996090f90e682a/revisions/parse_bjdata_fuzzer -runs=100 '/mnt/scratch0/clusterfuzz/bot/inputs/fuzzer-testcases/95a0cdc1152f187d8703ab25ef2e19bfE.-6.ADDR.0.INSTR.[UNKNOWN].fuzz'
Time ran: 0.028229713439941406
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 2105145144
INFO: Loaded 1 modules   (5975 inline 8-bit counters): 5975 [0x6918a0, 0x692ff7),
INFO: Loaded 1 PC tables (5975 PCs): 5975 [0x6213e0,0x638950),
/mnt/scratch0/clusterfuzz/bot/builds/clusterfuzz-builds_json_26b1464c0c18fac23c49bf26ed996090f90e682a/revisions/parse_bjdata_fuzzer: Running 1 inputs 100 time(s) each.
Running: /mnt/scratch0/clusterfuzz/bot/inputs/fuzzer-testcases/95a0cdc1152f187d8703ab25ef2e19bfE.-6.ADDR.0.INSTR.[UNKNOWN].fuzz
parse_bjdata_fuzzer: src/fuzzer-parse_bjdata.cpp:66: int LLVMFuzzerTestOneInput(const uint8_t *, size_t): Assertion `false' failed.
==79868== ERROR: libFuzzer: deadly signal
    #0 0x52d841 in __sanitizer_print_stack_trace /src/llvm-project/compiler-rt/lib/asan/asan_stack.cpp:87:3
    #1 0x46dbd8 in fuzzer::PrintStackTrace() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerUtil.cpp:210:5
    #2 0x453553 in fuzzer::Fuzzer::CrashCallback() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:233:3
    #3 0x7fa26b7053bf in libpthread.so.0
    #4 0x7fa26b51818a in __libc_signal_restore_set /build/glibc-eX1tMB/glibc-2.31/sysdeps/unix/sysv/linux/internal-signals.h:86:3
    #5 0x7fa26b51818a in raise /build/glibc-eX1tMB/glibc-2.31/sysdeps/unix/sysv/linux/raise.c:48:3
    #6 0x7fa26b4f7858 in abort /build/glibc-eX1tMB/glibc-2.31/stdlib/abort.c:79:7
    #7 0x7fa26b4f7728 in __assert_fail_base /build/glibc-eX1tMB/glibc-2.31/assert/assert.c:92:3
    #8 0x7fa26b508f35 in __assert_fail /build/glibc-eX1tMB/glibc-2.31/assert/assert.c:101:3
    #9 0x55de59 in LLVMFuzzerTestOneInput json/test/src/fuzzer-parse_bjdata.cpp:66:13
    #10 0x454ab2 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
    #11 0x440c32 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6
    #12 0x446394 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9
    #13 0x46e392 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #14 0x7fa26b4f90b2 in __libc_start_main /build/glibc-eX1tMB/glibc-2.31/csu/libc-start.c:308:16
    #15 0x41f6ed in _start
NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal

Compiler and operating system

macOS 12.3.1, Apple clang version 13.1.6

Library version

develop

Validation

• The bug also occurs if the latest version from the develop branch is used.
• I can successfully compile and run the unit tests.

[nlohmann]

Introduced with #3336. /cc @fangq

[fangq]

working on it, will update