Add support for packing and unpacking Android sparse images

This supports all features of Android sparse images, including holes,
and CRC32 (both full image checksum and CRC32 chunks).

Partial sparse images, like those included in GrapheneOS' new optimized
factory images, can also be packed and unpacked with these new commands,
unlike AOSP's simg2img and img2simg tools.

This new functionality is not relevant for avbroot's main use case, but
is useful for unpacking certain factory images for comparison with OTAs
during troubleshooting.

Signed-off-by: Andrew Gunnerson <[email protected]>

README.extra.md

@@ -286,7 +286,7 @@ All metadata slots in the newly packed LP image will be identical.
 ### Repacking an LP image
 
 ```bash
-avbroot lp repack [-i <input LP image>] [-i <input LP image>]... -o <output LP image> [-o <output LP image>]...
+avbroot lp repack -i <input LP image> [-i <input LP image>]... -o <output LP image> [-o <output LP image>]...
 ```
 
 This subcommand is logically equivalent to `avbroot lp unpack` followed by `avbroot lp pack`, except more efficient. Instead of unpacking and packing all partition images, the raw data is directly copied from the old LP image to the new LP image.
@@ -340,3 +340,47 @@ avbroot payload info -i <payload>
 ```
 
 This subcommand shows all of the payload header fields (which will likely be extremely long).
+
+## `avbroot sparse`
+
+This set of commands is for working with Android sparse images. All features of the file format are supported, including hole chunks and CRC32 checksums.
+
+### Unpacking a sparse image
+
+```bash
+avbroot sparse unpack -o <input sparse image> -o <output raw image>
+```
+
+This subcommand unpacks a sparse image to a raw image. If the sparse image contains CRC32 checksums, they will be validated during unpacking. If the sparse image contains holes, the output image will be created as a native sparse file.
+
+Certain fastboot factory images may have multiple sparse images, like `super_1.img`, `super_2.img`, etc., where they all touch a disjoint set of regions on the same partition. These can be unpacked by running this subcommand for each sparse image and specifying the `--preserve` option along with using the same output file. This preserves the existing data in the output file when unpacking each sparse image.
+
+### Packing a sparse image
+
+```bash
+avbroot sparse pack -i <input raw image> -o <output sparse image>
+```
+
+This subcommand packs a new sparse image from a raw image. The default block size is 4096 bytes, which can be changed with the `--block-size` option.
+
+By default, this will pack the entire input file. However, on Linux, there is an optimization where all holes in the input file, if it is a native sparse file, will be stored as hole chunks instead of `0`-filled chunks in the output sparse image.
+
+To pack a partial sparse image, such as those used in the special fastboot factory images mentioned above, pass in `--region <start> <end>`. This option can be specified multiple times to pack multiple regions.
+
+Unlike AOSP's `img2simg` tool, which never writes CRC32 checksums, this subcommand will write checksums if the input file has no holes and the entire file is being packed.
+
+### Repacking a sparse image
+
+```bash
+avbroot sparse repack -i <input sparse image> -o <output sparse image>
+```
+
+This subcommand is logically equivalent to `avbroot sparse unpack` followed by `avbroot sparse pack`, except more efficient. This is useful for roundtrip testing of avbroot's sparse file parser.
+
+### Showing sparse image metadata
+
+```bash
+avbroot sparse info -i <input sparse image>
+```
+
+This subcommand shows the sparse image metadata, including the header and all chunks.

avbroot/Cargo.toml

@@ -20,7 +20,9 @@ clap = { version = "4.4.1", features = ["derive"] }
 clap_complete = "4.4.0"
 cms = { version = "0.2.2", features = ["std"] }
 const-oid = "0.9.5"
+crc32fast = "1.4.2"
 ctrlc = "3.4.0"
+dlv-list = "0.5.2"
 flate2 = "1.0.27"
 gf256 = { version = "0.3.0", features = ["rs"] }
 hex = { version = "0.4.3", features = ["serde"] }

avbroot/src/cli/args.rs

@@ -15,7 +15,7 @@ use clap::{Parser, Subcommand, ValueEnum};
 use tracing::{debug, Level};
 use tracing_subscriber::fmt::{format::Writer, time::FormatTime};
 
-use crate::cli::{avb, boot, completion, cpio, fec, hashtree, key, lp, ota, payload};
+use crate::cli::{avb, boot, completion, cpio, fec, hashtree, key, lp, ota, payload, sparse};
 
 #[allow(clippy::large_enum_variant)]
 #[derive(Debug, Subcommand)]
@@ -30,6 +30,7 @@ pub enum Command {
     Lp(lp::LpCli),
     Ota(ota::OtaCli),
     Payload(payload::PayloadCli),
+    Sparse(sparse::SparseCli),
     /// (Deprecated: Use `avbroot ota patch` instead.)
     Patch(ota::PatchCli),
     /// (Deprecated: Use `avbroot ota extract` instead.)
@@ -134,6 +135,7 @@ pub fn main(logging_initialized: &AtomicBool, cancel_signal: &AtomicBool) -> Res
         Command::Lp(c) => lp::lp_main(&c, cancel_signal),
         Command::Ota(c) => ota::ota_main(&c, cancel_signal),
         Command::Payload(c) => payload::payload_main(&c, cancel_signal),
+        Command::Sparse(c) => sparse::sparse_main(&c, cancel_signal),
         // Deprecated aliases.
         Command::Patch(c) => ota::patch_subcommand(&c, cancel_signal),
         Command::Extract(c) => ota::extract_subcommand(&c, cancel_signal),

avbroot/src/cli/mod.rs

@@ -14,3 +14,4 @@ pub mod key;
 pub mod lp;
 pub mod ota;
 pub mod payload;
+pub mod sparse;