You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
When encoding a Go struct to a CBOR map with SortNone, keys are encoded based on the field order stored in the fields field of encodingStructType. The current implementation happens to sort fields in a certain way, but even if it did not, the encodingStructType for a given struct type is built once and cached for the life of the program. The result is that encoding a Go struct to a CBOR map appears to be deterministic, even with SortNone.
This is of course not wrong, but it does create opportunity for users to erroneously build into their programs and APIs the assumption that struct encoding is deterministic. I would like to make it harder for users to accidentally depend on this behavior.
Changing SortNone may be a breaking change for someone, precisely because they have inadvertently assumed that struct encoding is deterministic (https://www.hyrumslaw.com/). A new mode would be fine for my use case, but in that case it may still be a good idea to rename SortNone based on its implicit order (SortFieldIndex?) and add a const SortNone SortMode = SortFieldIndex // DEPRECATED to save future users. Edit: I tried changing the behavior of SortNone in a POC and it reinforced my concerns.
Randomly permuting the field encode order on every encode seems likely to incur some runtime overhead. I don't think that would be worth it since randomness is not necessary.
Additional context
The text was updated successfully, but these errors were encountered:
The POC reinforced this risks of changing the behavior of SortNone. I'm now proposing a new sort mode to handle this, and #515 has been updated accordingly.
fxamacker
changed the title
feature: Protect users from assuming a deterministic encoding of structs using SortNone
feature: Add a new SortMode to encode struct fields in a less predictable order
Apr 21, 2024
Is your feature request related to a problem? Please describe.
When encoding a Go struct to a CBOR map with
SortNone
, keys are encoded based on the field order stored in thefields
field ofencodingStructType
. The current implementation happens to sortfields
in a certain way, but even if it did not, theencodingStructType
for a given struct type is built once and cached for the life of the program. The result is that encoding a Go struct to a CBOR map appears to be deterministic, even withSortNone
.This is of course not wrong, but it does create opportunity for users to erroneously build into their programs and APIs the assumption that struct encoding is deterministic. I would like to make it harder for users to accidentally depend on this behavior.
Describe the solution you'd like
I propose a new SortMode that begins struct encoding at a random field offset (in the spirit of map iterator initialization https://github.com/golang/go/blob/b6efc3b755b74147a3700ad51773b01fa68f76e8/src/runtime/map.go#L837-L840). This would have negligible runtime overhead but would be, I think, sufficient to prevent programs from relying on order.
Describe alternatives you've considered
SortNone
may be a breaking change for someone, precisely because they have inadvertently assumed that struct encoding is deterministic (https://www.hyrumslaw.com/). A new mode would be fine for my use case, but in that case it may still be a good idea to renameSortNone
based on its implicit order (SortFieldIndex
?) and add aconst SortNone SortMode = SortFieldIndex // DEPRECATED
to save future users. Edit: I tried changing the behavior ofSortNone
in a POC and it reinforced my concerns.Additional context
The text was updated successfully, but these errors were encountered: