Skip to content

Commit

Permalink
feat(cache): Write temporary files in a dedicated directory (#265)
Browse files Browse the repository at this point in the history
This changes where temporary files are created to be in a dedicated
directory instead of the destination.  The final destinations are pruned
using the `symbolicator cleanup` command and this does not expect
files to be deleted by anything but itself, thus erroring if temporary
files have been moved to their final locations.

Since the new location is still within the cache directory moves from
there to their final cache location are atomic as users are expected
to keep the entire cache directory on one filesystem.
  • Loading branch information
Floris Bruynooghe committed Oct 27, 2020
1 parent 5106ff4 commit 5ad8293
Show file tree
Hide file tree
Showing 9 changed files with 285 additions and 69 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
- Fix a bug where Sentry requests were cached even though caches were disabled. ([#260](https://github.com/getsentry/symbolicator/pull/260))
- Publish Docker containers to DockerHub at `getsentry/symbolicator`. ([#271](https://github.com/getsentry/symbolicator/pull/271))
- Add `processing_pool_size` configuration option that allows to set the size of the processing pool. ([#273](https://github.com/getsentry/symbolicator/pull/273))
- Use a dedicated `tmp` sub-directory in the cache directory to write temporary files into. ([#265](https://github.com/getsentry/symbolicator/pull/265))

## 0.2.0

Expand Down
92 changes: 61 additions & 31 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ actix-web = { version = "0.7.19", features = ["tls"], default-features = false }
actix = "0.7.9"
anyhow = "1.0.32"
apple-crash-report-parser = "0.4.2"
backtrace = "0.3.53"
failure = "0.1.6"
thiserror = "1.0.20"
serde = { version = "1.0.101", features = ["derive", "rc"] }
Expand Down
13 changes: 5 additions & 8 deletions src/actors/common/cache.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
use std::collections::BTreeMap;
use std::fmt;
use std::fs;
use std::io;
use std::path::{Path, PathBuf};
use std::sync::Arc;
Expand Down Expand Up @@ -59,6 +58,10 @@ impl<T: CacheItemRequest> Cacher<T> {
current_computations: Arc::new(Mutex::new(BTreeMap::new())),
}
}

pub fn tempfile(&self) -> io::Result<NamedTempFile> {
self.config.tempfile()
}
}

impl fmt::Display for CacheKey {
Expand Down Expand Up @@ -202,13 +205,7 @@ impl<T: CacheItemRequest> Cacher<T> {
// just got pruned.
metric!(counter(&format!("caches.{}.file.miss", name)) += 1);

let temp_file = if let Some(ref path) = cache_path {
let dir = path.parent().unwrap();
tryf!(fs::create_dir_all(dir));
tryf!(NamedTempFile::new_in(dir))
} else {
tryf!(NamedTempFile::new())
};
let temp_file = tryf!(self.tempfile());

let future = request.compute(temp_file.path()).and_then(move |status| {
if let Some(ref cache_path) = cache_path {
Expand Down
Loading

0 comments on commit 5ad8293

Please sign in to comment.