feat: value quota based idl decoder limiting

[crusso]

Simplifies #4624 to a simple linear limit on the number of decoded values as a function of decoded payload size,
instead of using two linear functions on perfcounter (simulated or real) and allocation counter.

The function is:

value_quota(blob) : Nat64 = blob.size() * (numerator/denominator) + bias

where blob is the candid blob to be decoded, and numerator (default 1), denominator (default 1) and bias (default 1024) are Nat32s.

Much simpler than #4624 and doesn't depend on vagaries of instruction metering and byte allocation which varies with gc and compiler options, but is it good enough?

The constants can be (globally) modified/inspected using prims (Prim.getCandidLimits/Prim.setCandidLimits) which will need to get exposed in base eventually.

The quota is decremented on every call to deserialise or skip a value in vanilla candid mode (destabilization is not metered).
The quota is eagerly checked before deserializing or skipping arrays.

One possible refinement would be to combine the value quota with a memory quota (though the latter would still vary with gc flavour and perhaps word-size unless we count logical words)

• Disable for destabilization (iff Registers.get_rel_buf_opt is zero)
• Port new candid spacebomb test suite to drun-tests, to test against real perf counter provided by drun.
• Bump candid dependency to most recent
• Pass new spacebomb tests, both in candid test suite on wasmtime using value counter.

[luc-blaeser]

One thing that I am unsure or I have probably missed (sorry for the potentially stupid question): Does the metering limit also need to be checked on ordinary decoding of Candid when it is not skipped and not recursively called?

[luc-blaeser]

Very nice PR!
I have only some minor comments and one question (probably not relevant).

[crusso]

One thing that I am unsure or I have probably missed (sorry for the potentially stupid question): Does the metering limit also need to be checked on ordinary decoding of Candid when it is not skipped and not recursively called?

Actually, I'm not sure I understand the question (which worries me). I think all calls, initial, skip and recursive should be doing the check, but I maybe you've seen something I haven't.

[crusso]

@luc-blaeser PTAL (and thanks for the review!)

[chenyan-dfinity]

Can we document the cost model somewhere, so that people can roughly know how to tune the parameters?

[chenyan-dfinity]

Can count be 0?

[crusso]

No, because we return on previous line 311 when count == 0

[luc-blaeser]

@luc-blaeser PTAL (and thanks for the review!)

Not that I have any concrete concern, I was just curious whether I got all aspects.
If I understood correctly, the metering covers skipping and recursive calls. Also, the recursive call is done for each decoded Candid value.
I was wondering about the following cases of whether metering could be relevant there or would already be implicitly handled (by the recursion metering).

• If a value would have dynamic encoded length such as an array of listed elements, I guess the recursive function would be called for each sub-element. This would be fine.
• Would there be a need for size-dependent metering of blobs? I guess there is no recursive call involved per blob byte.
• IIRC, some constant Candid values can inflate to variable-sized objects (arrays, maybe also blob). This would be a non-linear effort (initialization) that is maybe not metered.
  These are just thoughts of mine, probably not relevant.

[crusso]

@luc-blaeser PTAL (and thanks for the review!)

Not that I have any concrete concern, I was just curious whether I got all aspects. If I understood correctly, the metering covers skipping and recursive calls. Also, the recursive call is done for each decoded Candid value. I was wondering about the following cases of whether metering could be relevant there or would already be implicitly handled (by the recursion metering).

• If a value would have dynamic encoded length such as an array of listed elements, I guess the recursive function would be called for each sub-element. This would be fine.

Yes, that's the idea.

• Would there be a need for size-dependent metering of blobs? I guess there is no recursive call involved per blob byte.

Fortunately, the blob decoder first checks that the blob does not exceed the length of the candid payload, which is limited to 10MB by the IC (in the worst case). The Nat and Int decoders also check they don't decode past the candid payload, so I was hoping that would limit the allocation.

• IIRC, some constant Candid values can inflate to variable-sized objects (arrays, maybe also blob). This would be a non-linear effort (initialization) that is maybe not metered.

Hmm, for arrays, maybe one could decrement the quota by number of elements before allocation to trigger then check, and then bulk-increment the quota before deserializing the element (to compensate for the earlier bulk check). Not sure.

These are just thoughts of mine, probably not relevant.

[github-actions]

Comparing from 367145d to 8e07a63:
In terms of gas, 5 tests regressed and the mean change is +0.0%.
In terms of size, 5 tests regressed and the mean change is +0.2%.

[crusso]

Can we document the cost model somewhere, so that people can roughly know how to tune the parameters?
Done in Changelog for now. Will add to base eventually...