Track transcript SwingStore start position and have upgrade reset to end rather than beginning

[FUDCo]

(Note: this task has been redefined, see #3293 (comment) below. We are retaining the issue with its original summary for continuity, but the work to be done here is rather different from what was originally outlined.)

What is the Problem Being Solved?

The SwingStore's stream API provides read and write operations but not delete. Delete is not currently needed, but we will need it if we wish to retire a vat completely and reclaim its resources.

Description of the Design

The proposal here is to augment the StreamStore API with a deleteStream(streamName) method, which will operate in a manner similar to the existing closeStream call but which will also delete the associated stream file.

Security Considerations

I do not believe there are any particular security considerations here above and beyond those posed by access to the StreamStore API in the first place. Obviously there will remain policy questions about whether or not deleting a stream in any particular case is a good idea or not, in particular when we think keeping old streams around may be useful as a diagnostic aid. However, that is a pragmatic consideration rather than a security consideration per se.

Test Plan

Augment the existing StreamStore API tests to exercise this operation and verify that the associated stream file is, in fact, gone after its use.

[FUDCo]

I think in retrospect that we do not need this, and we can close this issue. @warner ?

[FUDCo]

After discussion, Brian and I concluded that we actually do need this, though it's relatively low priority. Moving back onto the backlog.

[Tartuffo]

Let's revisit whether we need this for MN-1, or if this is subsumed by other work that @warner is doing.

[FUDCo]

Redefine this task in terms of keeping track of the historical state, possibly archiving the historical transcript prior to upgrade.

[FUDCo]

@warner and I discussed this and concluded that the task needs doing but needs to be reconceptualized a bit.

When a vat is upgraded, the transcript of its pre-upgrade life cannot safely be replayed by the post-upgrade version of its code. So upgrade was discarding the old transcript by resetting the lastSnapshot pointer to 0, resulting in the start of a new transcript that began overwriting the old one. This is problematic for a couple of reasons. First, although replays of the new transcript terminate at whatever position endPosition indicates, and thus work correctly, residual transcript data from the pre-upgrade world will still be present in the stream store that holds the transcript from the current end position up to the point where the upgrade happened. This leftover junk is potentially confusing to anyone trying to diagnose problems by inspection of the contents of the stream store. This could be addressed by actually deleting the stream store (then creating a new one) as part of the upgrade operation, as this ticket originally conceived of how things would work. However, a second problem is that by losing the older transcript data (whether by deletion or by overwriting), we make historical replay impossible, and thus preclude later employment of the sleeper agent upgrade protocol should the need arise at a future time.

Instead, we have decided that the correct strategy is to continue appending to the existing transcript, but keep track of the upgrade point so that replays will start from there instead of from 0 when there are no snapshots to begin replaying from. This motivates the addition of a startPosition record alongside the existing endPosition record. Instead of resetting endPosition to 0 on upgrade, startPosition would be set to endPosition and endPosition would be left unchanged.

If the disk space consumed by the growing transcript becomes an issue, at a future time we might add an operation to copy the historical portions of the transcript to an alternative (presumably cheaper) storage medium and truncate the leading portions of the active stream store accordingly. Depending on how replay from archival storage is implemented, this archiving might be entirely orthogonal to the startPosition modifications done by upgrades.