XS 3.8.7 has bad arithmetic on BigInt when __has_builtin(__builtin_add_overflow)

[mhofman]

Describe the bug

We noticed an anachrophobia when replaying vat transcripts (see #6929) using newer compilers.

We traced it to the following change: agoric-labs/moddable@77dcbad which was later reverted upstream in Moddable-OpenSource/moddable@055ce64

To Reproduce

Follow the steps of #6929 to replay vat42

Expected behavior

No anachrophobia

Actual observed behavior

/home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsMapSet.c:937:16: runtime error: unsigned integer overflow: 18446744073709551614 + 262144 cannot be represented in type 'unsigned long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsMapSet.c:937:16 in
/home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsMapSet.c:905:22: runtime error: unsigned integer overflow: 8474409842283144096 + 14099883362853511965 cannot be represented in type 'unsigned long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsMapSet.c:905:22 in
/home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsMapSet.c:939:13: runtime error: unsigned integer overflow: 1145074143708216745 * 21 cannot be represented in type 'unsigned long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsMapSet.c:939:13 in
created manager from snapshot a7a5dfce89abdc517aaf81db92fcdef60aade93d14bc73f761c0af83382cfeee, worker PID: 1685819
/home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsCommon.c:1159:10: runtime error: unsigned integer overflow: 113723913172083 - 72340172838076673 cannot be represented in type 'unsigned long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsCommon.c:1159:10 in
/home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsArray.c:1573:3: runtime error: unsigned integer overflow: 50047 + 4294967295 cannot be represented in type 'unsigned int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsArray.c:1573:3 in
/home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsBigInt.c:1451:9: runtime error: unsigned integer overflow: 805306368 - 2684354560 cannot be represented in type 'unsigned int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsBigInt.c:1451:9 in
/home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsMapSet.c:941:13: runtime error: unsigned integer overflow: 18437737054755356672 + 17870294860641075200 cannot be represented in type 'unsigned long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/raphs/agoric-sdk/packages/xsnap/moddable/xs/sources/xsMapSet.c:941:13 in
anachrophobia strikes vat v42
expected: {"0":"vatstoreSet","1":"vom.o+17/413","2":"{\"currentBalance\":{\"body\":\"{\\\"brand\\\":{\\\"@qclass\\\":\\\"slot\\\",\\\"iface\\\":\\\"Alleged: DAI_axl brand\\\",\\\"index\\\":0},\\\"value\\\":{\\\"@qclass\\\":\\\"bigint\\\",\\\"digits\\\":\\\"5852128000000000000000\\\"}}\",\"slots\":[\"o+15/1\"]},\"recoverySet\":{\"body\":\"{\\\"@qclass\\\":\\\"slot\\\",\\\"iface\\\":\\\"Alleged: setStore\\\",\\\"index\\\":0}\",\"slots\":[\"o+8/428\"]}}","length":3}
got     : {"0":"vatstoreSet","1":"vom.o+17/413","2":"{\"currentBalance\":{\"body\":\"{\\\"brand\\\":{\\\"@qclass\\\":\\\"slot\\\",\\\"iface\\\":\\\"Alleged: DAI_axl brand\\\",\\\"index\\\":0},\\\"value\\\":{\\\"@qclass\\\":\\\"bigint\\\",\\\"digits\\\":\\\"5852127999987115098112\\\"}}\",\"slots\":[\"o+15/1\"]},\"recoverySet\":{\"body\":\"{\\\"@qclass\\\":\\\"slot\\\",\\\"iface\\\":\\\"Alleged: setStore\\\",\\\"index\\\":0}\",\"slots\":[\"o+8/428\"]}}","length":3}
during transcript num= 1006 for worker PID 1685809 (start delivery 0)

Additional information

The initial acceptance test to merge 3.8.7 were run on an older gcc before __has_builtin was required to be present. As such we didn't trigger this upstream bug when we initially verified 3.8.7 was valid.