From d02950b4c589cdb2c294f0e9e02ae5c62a712ddf Mon Sep 17 00:00:00 2001 From: "flowzone-app[bot]" <124931076+flowzone-app[bot]@users.noreply.github.com> Date: Mon, 26 Jun 2023 13:44:27 +0000 Subject: [PATCH] v20.10.37 --- .versionbot/CHANGELOG.yml | 151 ++++++++++++++++++++++++++++++++++++++ CHANGELOG.md | 6 ++ VERSION | 2 +- 3 files changed, 158 insertions(+), 1 deletion(-) diff --git a/.versionbot/CHANGELOG.yml b/.versionbot/CHANGELOG.yml index 655afde147..51c2cf2ba1 100644 --- a/.versionbot/CHANGELOG.yml +++ b/.versionbot/CHANGELOG.yml @@ -1,3 +1,154 @@ +- commits: + - subject: "Bugfix: concatReadSeekCloser.Read() with buffers of any size" + hash: 84471473ce02a0b8820e13105f845b67c19903c9 + body: > + Previously, `concatReadSeekCloser.Read()` would incorrectly return + + an `io.ErrUnexpectedEOF` if the last read from the second concatenated + + `Reader` didn't completely fill the passed buffer. + + + For instance: + + + ``` + First Reader Second Reader + |aaaaaaaaaaaaa|bbbbbbbbbbbbbbbbbbb| <--- concatReadSeekCloser + |--- previously read ---|--- buffer ---| <--- last read + |xxxx| <--- "excess" buffer + ``` + + + In this example, we have a `concatReadSeekCloser` that concatenates two + + `Reader`s (`aaa...` and `bbb...`). The last `Read()` used a buffer + + larger than the yet-to-be-read portion of the `bbb...`. So, it would + + incorrectly return an `io.ErrUnexpectedEOF`. + + + This commit makes sure that last `Read()` returns all the remaining data + + without an error. It also adds various test cases for + + `concatReadSeekCloser.Read()`, many of which would fail before this + + correction. + + + Interestingly, this bug was silently affecting us. Not in a fatal way, + + but causing deltas to be larger than necessary. Indeed, running + + `TestDeltaSize` after this commit shows that some test cases are + + producing deltas smaller than what we expected before. For posterity, + + see all the details below. + + + We use `concatReadSeekCloser`s to concatenate all layers of the basis + + image when creating the "signature" of the basis image. In this process, + + the `concatReadSeekCloser`s are wrapped around by a buffered reader with + + a buffer of 65kB. + + + If, in any read, part of this 65kB buffer was beyond the second + + concatenated reader, it would result in an `io.ErrUnexpectedEOF`. This + + would not cause the whole process to fail, but would prematurely end the + + signature generation: some of the final blocks in the basis image would + + not be added to the signature. Therefore, if those blocks appeared in + + the target image, they'd result in (larger) LITERAL, instead of + + (smaller) COPY operations. + + + For illustration, here's the delta generated for the `delta-006-008` + + test case. First before this commit: + + + ``` + OP_COPY_N1_N2 0 512 + OP_COPY_N2_N2 1536 1024 + OP_COPY_N2_N2 3584 1024 + OP_COPY_N2_N2 5632 1536 + OP_COPY_N2_N2 8192 1536 + OP_COPY_N2_N2 10752 4096 + OP_COPY_N2_N2 15872 8704 + OP_COPY_N2_N2 25600 10752 + OP_COPY_N2_N2 37376 10752 + OP_COPY_N2_N4 49152 131584 + OP_COPY_N4_N4 181760 150528 + OP_COPY_N4_N4 333312 500736 + OP_COPY_N4_N4 835072 1000960 + OP_COPY_N4_N4 1837056 1000960 + OP_COPY_N4_N4 2839040 1027584 # Here: a COPY op... + OP_LITERAL_N2 21504 # ...followed by a 21 kB LITERAL. + OP_COPY_N4_N2 2838528 512 + OP_COPY_N4_N2 2838528 512 + OP_END + ``` + + + And after this commit: + + + ``` + OP_COPY_N1_N2 0 512 + OP_COPY_N2_N2 1536 1024 + OP_COPY_N2_N2 3584 1024 + OP_COPY_N2_N2 5632 1536 + OP_COPY_N2_N2 8192 1536 + OP_COPY_N2_N2 10752 4096 + OP_COPY_N2_N2 15872 8704 + OP_COPY_N2_N2 25600 10752 + OP_COPY_N2_N2 37376 10752 + OP_COPY_N2_N4 49152 131584 + OP_COPY_N4_N4 181760 150528 + OP_COPY_N4_N4 333312 500736 + OP_COPY_N4_N4 835072 1000960 + OP_COPY_N4_N4 1837056 1000960 + OP_COPY_N4_N4 2839040 1049088 # COPY only, since we detected a longer match + OP_COPY_N4_N2 3888640 512 + OP_COPY_N4_N2 3888640 512 + OP_END + ``` + + + That 21kB LITERAL is the difference in size we saw in the test results. + footer: + Signed-off-by: Leandro Motta Barros + signed-off-by: Leandro Motta Barros + Change-type: patch + change-type: patch + author: Leandro Motta Barros + nested: [] + - subject: Minor code and comments tweaks + hash: 7dd51428c918980163fbe34511152dc72e7a3372 + body: | + Using `defer` for the sake of being more idiomatic (and maybe slightly + more reliable); plus, using the proper doc comment standards. + footer: + Signed-off-by: Leandro Motta Barros + signed-off-by: Leandro Motta Barros + Change-type: patch + change-type: patch + author: Leandro Motta Barros + nested: [] + version: 20.10.37 + title: "" + date: 2023-06-26T13:44:22.217Z - commits: - subject: Further improve resilience of image pulls hash: d864e340bfe050144252db8b0de9c66a3a40fa20 diff --git a/CHANGELOG.md b/CHANGELOG.md index db33701b0b..1f504eb20d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file automatically by Versionist. DO NOT EDIT THIS FILE MANUALLY! This project adheres to [Semantic Versioning](http://semver.org/). +# v20.10.37 +## (2023-06-26) + +* Bugfix: concatReadSeekCloser.Read() with buffers of any size [Leandro Motta Barros] +* Minor code and comments tweaks [Leandro Motta Barros] + # v20.10.36 ## (2023-05-04) diff --git a/VERSION b/VERSION index da2afb0ef8..e86fc537f3 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -20.10.36 \ No newline at end of file +20.10.37 \ No newline at end of file