retain transcripts from all versions of a vat, or just the most recent one?

[warner]

What is the Problem Being Solved?

In #4927 (comment) we talk about whether or not to retain all vat transcripts from the beginning of time, vs just the transcript since the most recent vat upgrade.

When a vat is upgraded, we currently (once #4927 lands) delete the transcript upon vat upgrade. (We effectively delete the transcript, by zeroing out the start/end offsets, but eventually we should have a proper streamStore.deleteTranscript API which reclaims the space correctly). We don't need the earlier version's transcript to perform a replay-from-empty-heap of the current version. And #1691 can't meaningfully do anything with the earlier versions either: even the sleeper agent gets evicted upon vat upgrade, along with everything in the rest of the heap, unless we give the sleeper agent a secret diary that it can write to without transcripted syscalls and which is not allowed to influence its visible behavior until after the activation event.

But if sleeper-agent upgrade can benefit from earlier versions, then deleting those earlier transcripts is an information-losing action which might make some forms of upgrade difficult or impossible.

In general I think we want to delete data as early as possible, to minimize storage requirements. If we're interested in history, we can have non-block-producing nodes keep a record, but the main validators should not be obligated unless it provides some data that might eventually get used within a consensus action.

Description of the Design

If we do decide we want to retain transcripts from the very beginning of a vat's definition, then it is important to retain the source bundles used by each version. The transcript isn't meaningful without the source code it delivers into.

The most elegant data structure I can think of would put the bundleID into the startVat delivery. The extended transcript is then shaped like:

• startVat(bundleID1, vatParameters1)
• deliveries to version 1
• stopVat
• startVat(bundleID2, vatParameters2)
• deliveries to version 2
• ...

This would require significant changes to the way we create the worker and send the setBundle command to it. The manager would probably need to snoop the startVat deliveries, remove the bundleID, send a separate setBundle command with the full bundle contents (which would create the liveslots instance), and only then send startVat.

Security Considerations

Test Plan

[warner]

For MN-1, we decided to keep all transcripts, but not retain the bundlecaps (or their bundles). We figure that a #1691 -style sleeper-agent upgrade would use the transcripts (from all versions, starting from zero) but not the original bundles (since it's running brand new code anyways). And a new validator that wants to catch up would start with the DB state and the most recent bundle/transcript, ignoring all previous versions.