Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
ci: enable 'librocksdb-sys' to be able to be properly cached
Today we leverage the 'Swatinem/rust-cache' github action in order to cache rust third-party dependencies and avoid needing to rebuild them every time CI is run. Unfortunately it was identified that despite a cache hit the 'librocksdb-sys' crate was always retriggering a very costly build. The crux of the issue is mostly highlighted by this issue (rust-rocksdb/rust-rocksdb#574) on the rust-rocksdb repository. The 'librocksdb-sys' crate uses a build script to be able to build the c++ rocksdb project (tracked via a submodule in the rust-rocksdb repo) and makes use of of the cargo directive "cargo:rerun-if-changed=rocksdb/" to ensure that anytime the submodule is updated that the c++ code should be recompiled. In particular this cargo directive only uses the filesystem last-modified "mtime" timestamp to determine if the file has changed (https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorerun-if-changedpath). The 'Swatinem/rust-cache' action explicitly avoids caching the '.cargo/registry/src' directory since its faster for cargo to repopulate the unpacked crate src from their archives in '.cargo/registry/cache' (which is cached by the github action). This leads to the unintended consiquency that when cargo goes to unpack the 'librocksdb-sys' crate src, the mtime of the "rocksdb/" directory no longer matches the cached mtime that was restored from a previous build of the 'librocksdb-sys' dependency resulting in 'librocksdb-sys' being rebuilt. In order to fix this issue we can additionally cache the '.cargo/registry/src/*/librocksdb-sys-*' directory (since the github caching infrastructure preserves mtimes of cached files) and ensure that the mtime matches what cargo expects, avoiding a rebuild. In order to do this I've forked the 'Swatinem/rust-cache' action to 'bmwill/rust-cache' add added the ability to additionally specify other paths that should be cached and specified the '.cargo/registry/src/*/lib-rocksdb-sys-*' directory.
- Loading branch information