backuptar: Fix sparse file handling #221
Conversation
A recent OS change altered how sparse files are represented in backup streams. This caused backuptar to no longer work with certain files. The specific behavior that changed is as follows: - Empty sparse files (size = 0), previously did not have any data or sparse block streams in the backup stream. Now, they will have a data stream with size = 0, and no sparse block streams. - Sparse files with a single allocated range (e.g. a normal file that has the sparse attribute set) previously would not show as sparse in the backup stream. Now, they will show as sparse. The old backuptar behavior assumed that if the sparse flag was set on the data stream, then there would always be a set of sparse blocks following. These changes break this assumption, and so require special handling. It is unsupported to have a data stream, marked sparse, that contains file content AND a series of sparse block streams following. As far as I can tell this is not a valid case for backup streams. This change also cleans up some code and error messages, and expands on the test coverage for backuptar. For more information on backup stream format see: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-bkup/f67950c8-d583-469a-83dd-c4ff4cedf533 Signed-off-by: Kevin Parsons <[email protected]>
| if dataHdr != nil { | ||
| // A data stream was found. Copy the data. | ||
| if (dataHdr.Attributes & winio.StreamSparseAttributes) == 0 { | ||
| // We assume that we will either have a data stream size > 0 XOR have sparse block streams. | ||
| if dataHdr.Size > 0 || (dataHdr.Attributes&winio.StreamSparseAttributes) == 0 { |
There was a problem hiding this comment.
gofmt enforces this style. I don't understand the exact rule it's using to format, but if I take away the dataHdr.Size > 0 || then it puts a space in around the &, otherwise not.
@katiewasnothere is there something that should be expanded upon that would be helpful here? I had never touched this code either, and had to learn what it was doing. I tried to clarify the reasoning for the new behavior in my comments. |
No, it made sense! I wrote that more to say that while it looks good to me, since I'm not super familiar with the code, I could potentially be missing small details/issues. |
A recent OS change altered how sparse files are represented in backup
streams. This caused backuptar to no longer work with certain files. The
specific behavior that changed is as follows:
sparse block streams in the backup stream. Now, they will have a
data stream with size = 0, and no sparse block streams.
has the sparse attribute set) previously would not show as sparse in
the backup stream. Now, they will show as sparse.
The old backuptar behavior assumed that if the sparse flag was set on
the data stream, then there would always be a set of sparse blocks
following. These changes break this assumption, and so require special
handling.
It is unsupported to have a data stream, marked sparse, that contains
file content AND a series of sparse block streams following. As far as
I can tell this is not a valid case for backup streams.
This change also cleans up some code and error messages, and expands on
the test coverage for backuptar.
For more information on backup stream format see: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-bkup/f67950c8-d583-469a-83dd-c4ff4cedf533
Signed-off-by: Kevin Parsons [email protected]