fix: Fetch ELF debug files as well just to be safe if it is a Windows…

… minidump (#555) For certain executables (PE, MachO, ELF), symbolicator will now attempt to download all known debug companion file types for that group, instead of just the singular debug companion type they're commonly paired with. This is because there are certain configurations where it is possible to have something like a PE with an ELF debug companion. To be specific, if symbolicator needs to process a minidump generated by say, a MachO, it will search for corresponding PDB, dSYM, and ELF (debug) files that may match that given MachO.
getsentry · Sep 20, 2021 · 016adba · 016adba
1 parent 3013d09
commit 016adba
Show file tree

Hide file tree

Showing 12 changed files with 257 additions and 90 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -27,6 +27,7 @@
 - Symbolication tasks are being spawned on a tokio 1 Runtime now. ([#531](https://github.com/getsentry/symbolicator/pull/531))
 - Symbolicator now allows explicitly versioning its caches. It will fall back to supported outdated cache versions immediately instead of eagerly waiting for updated cache files. Recomputations of newer cache versions are being done lazily in the background, bounded by new settings `max_lazy_redownloads` and `max_lazy_recomputations` for downloaded and derived caches respectively. ([#524](https://github.com/getsentry/symbolicator/pull/524), [#533](https://github.com/getsentry/symbolicator/pull/533), [#535](https://github.com/getsentry/symbolicator/pull/535))
 - Remove actix-web in favor of axum. This changes the web framework and also completely switches to the tokio 1 Runtime. ([#544](https://github.com/getsentry/symbolicator/pull/544))
+- Search for all known types of debug companion files during symbolication, in case there exists for example an ELF debug companion for a PE. ([#555](https://github.com/getsentry/symbolicator/pull/555))
 
 ### Fixes
 

diff --git a/...or/src/services/snapshots/symbolicator__services__symbolication__tests__add_bucket-2.snap b/...or/src/services/snapshots/symbolicator__services__symbolication__tests__add_bucket-2.snap
@@ -1,6 +1,7 @@
 ---
-source: src/services/symbolication.rs
-expression: response.await.unwrap()
+source: crates/symbolicator/src/services/symbolication.rs
+expression: response.unwrap()
+
 ---
 status: completed
 stacktraces:
@@ -29,6 +30,10 @@ modules:
     image_addr: "0x100000000"
     image_size: 4096
     candidates:
+      - source: local
+        location: "http://localhost:<port>/download/50/2fc0a51ec13e479998684fa139dca7.debug"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/502F/C0A5/1EC1/3E47/9998/684FA139DCA7"
         download:
@@ -44,3 +49,4 @@ modules:
         location: "http://localhost:<port>/download/502F/C0A5/1EC1/3E47/9998/684FA139DCA7.app"
         download:
           status: notfound
+
diff --git a/.../services/snapshots/symbolicator__services__symbolication__tests__apple_crash_report.snap b/.../services/snapshots/symbolicator__services__symbolication__tests__apple_crash_report.snap
@@ -1,6 +1,7 @@
 ---
-source: src/services/symbolication.rs
-expression: response.await.unwrap()
+source: crates/symbolicator/src/services/symbolication.rs
+expression: response.unwrap()
+
 ---
 status: completed
 timestamp: 1547055742
@@ -133,6 +134,18 @@ modules:
         location: "http://localhost:<port>/download/2D90/3291/397D/3D14/BFCA/52C7FB8C5E00.app"
         download:
           status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/2d/903291397d3d14bfca52c7fb8c5e00.debug"
+        download:
+          status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/YetAnotherMac/2D903291397D3D14BFCA52C7FB8C5E000/YetAnotherMa_"
+        download:
+          status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/YetAnotherMac/2D903291397D3D14BFCA52C7FB8C5E000/YetAnotherMac"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/YetAnotherMac/2D903291397D3D14BFCA52C7FB8C5E000/YetAnotherMac.sym"
         download:
@@ -193,3 +206,4 @@ modules:
     debug_file: /Users/bruno/Documents/Unreal Projects/YetAnotherMac/MacNoEditor/YetAnotherMac.app/Contents/UE4/Engine/Binaries/ThirdParty/PhysX3/Mac/libPxFoundationPROFILE.dylib
     image_addr: "0x1131fa000"
     image_size: 28671
+
diff --git a/...rc/services/snapshots/symbolicator__services__symbolication__tests__minidump_linux-2.snap b/...rc/services/snapshots/symbolicator__services__symbolication__tests__minidump_linux-2.snap
@@ -1,19 +1,27 @@
 ---
-source: src/services/symbolication.rs
+source: crates/symbolicator/src/services/symbolication.rs
 expression: cache_entries
+
 ---
 - local_5d_7b6259552275a3c17bd4c3fd05f5a6bf40caa5
 - local_5d_7b6259552275a3c17bd4c3fd05f5a6bf40caa5_debug
 - local_5d_7b6259552275a3c17bd4c3fd05f5a6bf40caa5_src_zip
 - local_b5_381a457906d279073822a5ceb24c4bfef94ddb
 - local_b5_381a457906d279073822a5ceb24c4bfef94ddb_debug
 - local_b5_381a457906d279073822a5ceb24c4bfef94ddb_src_zip
+- local_crash_C0BCC3F19827FE653058404B2831D9E60_cras_
+- local_crash_C0BCC3F19827FE653058404B2831D9E60_crash
 - local_crash_C0BCC3F19827FE653058404B2831D9E60_crash_src_zip
 - local_crash_C0BCC3F19827FE653058404B2831D9E60_crash_sym
 - local_f1_c3bcc0279865fe3058404b2831d9e64135386c
 - local_f1_c3bcc0279865fe3058404b2831d9e64135386c_debug
 - local_f1_c3bcc0279865fe3058404b2831d9e64135386c_src_zip
+- local_ld-2_23_so_59627B5D2255A375C17BD4C3FD05F5A60_ld-2_23_s_
+- local_ld-2_23_so_59627B5D2255A375C17BD4C3FD05F5A60_ld-2_23_so
 - local_ld-2_23_so_59627B5D2255A375C17BD4C3FD05F5A60_ld-2_23_so_src_zip
 - local_ld-2_23_so_59627B5D2255A375C17BD4C3FD05F5A60_ld-2_23_so_sym
+- local_libc-2_23_so_451A38B5067979D2073822A5CEB24C4B0_libc-2_23_s_
+- local_libc-2_23_so_451A38B5067979D2073822A5CEB24C4B0_libc-2_23_so
 - local_libc-2_23_so_451A38B5067979D2073822A5CEB24C4B0_libc-2_23_so_src_zip
 - local_libc-2_23_so_451A38B5067979D2073822A5CEB24C4B0_libc-2_23_so_sym
+
diff --git a/.../src/services/snapshots/symbolicator__services__symbolication__tests__minidump_linux.snap b/.../src/services/snapshots/symbolicator__services__symbolication__tests__minidump_linux.snap
@@ -1,6 +1,7 @@
 ---
-source: src/services/symbolication.rs
-expression: response.await.unwrap()
+source: crates/symbolicator/src/services/symbolication.rs
+expression: response.unwrap()
+
 ---
 status: completed
 timestamp: 1522061032
@@ -127,6 +128,14 @@ modules:
     image_addr: "0x400000"
     image_size: 106496
     candidates:
+      - source: local
+        location: "http://localhost:<port>/download/crash/C0BCC3F19827FE653058404B2831D9E60/cras_"
+        download:
+          status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/crash/C0BCC3F19827FE653058404B2831D9E60/crash"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/crash/C0BCC3F19827FE653058404B2831D9E60/crash.sym"
         download:
@@ -178,6 +187,14 @@ modules:
         location: "http://localhost:<port>/download/b5/381a457906d279073822a5ceb24c4bfef94ddb.debug"
         download:
           status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/libc-2.23.so/451A38B5067979D2073822A5CEB24C4B0/libc-2.23.s_"
+        download:
+          status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/libc-2.23.so/451A38B5067979D2073822A5CEB24C4B0/libc-2.23.so"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/libc-2.23.so/451A38B5067979D2073822A5CEB24C4B0/libc-2.23.so.sym"
         download:
@@ -251,6 +268,14 @@ modules:
         location: "http://localhost:<port>/download/5d/7b6259552275a3c17bd4c3fd05f5a6bf40caa5.debug"
         download:
           status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/ld-2.23.so/59627B5D2255A375C17BD4C3FD05F5A60/ld-2.23.s_"
+        download:
+          status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/ld-2.23.so/59627B5D2255A375C17BD4C3FD05F5A60/ld-2.23.so"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/ld-2.23.so/59627B5D2255A375C17BD4C3FD05F5A60/ld-2.23.so.sym"
         download:
@@ -270,3 +295,4 @@ modules:
     debug_file: linux-gate.so
     image_addr: "0x7fff5aef1000"
     image_size: 8192
+
diff --git a/...rc/services/snapshots/symbolicator__services__symbolication__tests__minidump_macos-2.snap b/...rc/services/snapshots/symbolicator__services__symbolication__tests__minidump_macos-2.snap
@@ -1,14 +1,22 @@
 ---
-source: src/services/symbolication.rs
+source: crates/symbolicator/src/services/symbolication.rs
 expression: cache_entries
+
 ---
 - local_67E9_247C_814E_392B_A027_DBDE6748FCBF
 - local_67E9_247C_814E_392B_A027_DBDE6748FCBF_app
 - local_67E9_247C_814E_392B_A027_DBDE6748FCBF_src_zip
+- local_67_e9247c814e392ba027dbde6748fcbf_debug
 - local_9B2A_C56D_107C_3541_A127_9094A751F2C9
 - local_9B2A_C56D_107C_3541_A127_9094A751F2C9_app
 - local_9B2A_C56D_107C_3541_A127_9094A751F2C9_src_zip
+- local_9b_2ac56d107c3541a1279094a751f2c9_debug
+- local_crash_67E9247C814E392BA027DBDE6748FCBF0_cras_
+- local_crash_67E9247C814E392BA027DBDE6748FCBF0_crash
 - local_crash_67E9247C814E392BA027DBDE6748FCBF0_crash_src_zip
 - local_crash_67E9247C814E392BA027DBDE6748FCBF0_crash_sym
+- local_libdyld_dylib_9B2AC56D107C3541A1279094A751F2C90_libdyld_dyli_
+- local_libdyld_dylib_9B2AC56D107C3541A1279094A751F2C90_libdyld_dylib
 - local_libdyld_dylib_9B2AC56D107C3541A1279094A751F2C90_libdyld_dylib_src_zip
 - local_libdyld_dylib_9B2AC56D107C3541A1279094A751F2C90_libdyld_dylib_sym
+
diff --git a/.../src/services/snapshots/symbolicator__services__symbolication__tests__minidump_macos.snap b/.../src/services/snapshots/symbolicator__services__symbolication__tests__minidump_macos.snap
@@ -1,6 +1,7 @@
 ---
-source: src/services/symbolication.rs
-expression: response.await.unwrap()
+source: crates/symbolicator/src/services/symbolication.rs
+expression: response.unwrap()
+
 ---
 status: completed
 timestamp: 1521713398
@@ -72,6 +73,10 @@ modules:
     image_addr: "0x109b9b000"
     image_size: 69632
     candidates:
+      - source: local
+        location: "http://localhost:<port>/download/67/e9247c814e392ba027dbde6748fcbf.debug"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/67E9/247C/814E/392B/A027/DBDE6748FCBF"
         download:
@@ -80,6 +85,14 @@ modules:
         location: "http://localhost:<port>/download/67E9/247C/814E/392B/A027/DBDE6748FCBF.app"
         download:
           status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/crash/67E9247C814E392BA027DBDE6748FCBF0/cras_"
+        download:
+          status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/crash/67E9247C814E392BA027DBDE6748FCBF0/crash"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/crash/67E9247C814E392BA027DBDE6748FCBF0/crash.sym"
         download:
@@ -318,6 +331,18 @@ modules:
         location: "http://localhost:<port>/download/9B2A/C56D/107C/3541/A127/9094A751F2C9.app"
         download:
           status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/9b/2ac56d107c3541a1279094a751f2c9.debug"
+        download:
+          status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/libdyld.dylib/9B2AC56D107C3541A1279094A751F2C90/libdyld.dyli_"
+        download:
+          status: notfound
+      - source: local
+        location: "http://localhost:<port>/download/libdyld.dylib/9B2AC56D107C3541A1279094A751F2C90/libdyld.dylib"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/libdyld.dylib/9B2AC56D107C3541A1279094A751F2C90/libdyld.dylib.sym"
         download:
@@ -727,3 +752,4 @@ modules:
     debug_file: libxpc.dylib
     image_addr: "0x7fffe8134000"
     image_size: 172032
+
diff --git a/.../services/snapshots/symbolicator__services__symbolication__tests__minidump_windows-2.snap b/.../services/snapshots/symbolicator__services__symbolication__tests__minidump_windows-2.snap
@@ -1,7 +1,13 @@
 ---
-source: src/services/symbolication.rs
+source: crates/symbolicator/src/services/symbolication.rs
 expression: cache_entries
+
 ---
+- local_57_898dab25000_debug
+- local_59_0285e9e0000_debug
+- local_59_b0d8f3183000_debug
+- local_5a_49bb75c1000_debug
+- local_5a_b380779000_debug
 - local_crash_exe_5AB380779000_crash_ex_
 - local_crash_exe_5AB380779000_crash_exe
 - local_crash_pdb_3249D99D0C4049318610F4E4FB0B69361_crash_pd_
@@ -30,3 +36,4 @@ expression: cache_entries
 - local_wrpcrt4_pdb_AE131C6727A74FA19916B5A4AEF411901_wrpcrt4_pd_
 - local_wrpcrt4_pdb_AE131C6727A74FA19916B5A4AEF411901_wrpcrt4_pdb
 - local_wrpcrt4_pdb_AE131C6727A74FA19916B5A4AEF411901_wrpcrt4_sym
+
diff --git a/...rc/services/snapshots/symbolicator__services__symbolication__tests__minidump_windows.snap b/...rc/services/snapshots/symbolicator__services__symbolication__tests__minidump_windows.snap
@@ -1,6 +1,7 @@
 ---
 source: crates/symbolicator/src/services/symbolication.rs
-expression: response.await.unwrap()
+expression: response.unwrap()
+
 ---
 status: completed
 timestamp: 1521713273
@@ -169,6 +170,10 @@ modules:
     image_addr: "0x2a0000"
     image_size: 36864
     candidates:
+      - source: local
+        location: "http://localhost:<port>/download/5a/b380779000.debug"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/crash.exe/5AB380779000/crash.ex_"
         download:
@@ -259,6 +264,10 @@ modules:
     image_addr: "0x70b70000"
     image_size: 151552
     candidates:
+      - source: local
+        location: "http://localhost:<port>/download/57/898dab25000.debug"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/dbgcore.dll/57898DAB25000/dbgcore.dl_"
         download:
@@ -385,6 +394,10 @@ modules:
     image_addr: "0x75050000"
     image_size: 917504
     candidates:
+      - source: local
+        location: "http://localhost:<port>/download/59/0285e9e0000.debug"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/kernel32.dll/590285E9e0000/kernel32.dl_"
         download:
@@ -436,6 +449,10 @@ modules:
     image_addr: "0x75810000"
     image_size: 790528
     candidates:
+      - source: local
+        location: "http://localhost:<port>/download/5a/49bb75c1000.debug"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/rpcrt4.dll/5A49BB75c1000/rpcrt4.dl_"
         download:
@@ -502,6 +519,10 @@ modules:
     image_addr: "0x77170000"
     image_size: 1585152
     candidates:
+      - source: local
+        location: "http://localhost:<port>/download/59/b0d8f3183000.debug"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/ntdll.dll/59B0D8F3183000/ntdll.dl_"
         download:
@@ -522,3 +543,4 @@ modules:
         location: "http://localhost:<port>/download/wntdll.pdb/971F98E5CE6041FFB2D7235BBEB345781/wntdll.sym"
         download:
           status: notfound
+
diff --git a/...r/src/services/snapshots/symbolicator__services__symbolication__tests__remove_bucket.snap b/...r/src/services/snapshots/symbolicator__services__symbolication__tests__remove_bucket.snap
@@ -1,6 +1,7 @@
 ---
-source: src/services/symbolication.rs
-expression: response.await.unwrap()
+source: crates/symbolicator/src/services/symbolication.rs
+expression: response.unwrap()
+
 ---
 status: completed
 stacktraces:
@@ -29,6 +30,10 @@ modules:
     image_addr: "0x100000000"
     image_size: 4096
     candidates:
+      - source: local
+        location: "http://localhost:<port>/download/50/2fc0a51ec13e479998684fa139dca7.debug"
+        download:
+          status: notfound
       - source: local
         location: "http://localhost:<port>/download/502F/C0A5/1EC1/3E47/9998/684FA139DCA7"
         download:
@@ -44,3 +49,4 @@ modules:
         location: "http://localhost:<port>/download/502F/C0A5/1EC1/3E47/9998/684FA139DCA7.app"
         download:
           status: notfound
+
diff --git a/crates/symbolicator/src/sources.rs b/crates/symbolicator/src/sources.rs
@@ -458,9 +458,31 @@ impl FileType {
     #[inline]
     pub fn from_object_type(ty: ObjectType) -> &'static [Self] {
         match ty {
-            ObjectType::Macho => &[FileType::MachDebug, FileType::MachCode, FileType::Breakpad],
-            ObjectType::Pe => &[FileType::Pdb, FileType::Pe, FileType::Breakpad],
-            ObjectType::Elf => &[FileType::ElfDebug, FileType::ElfCode, FileType::Breakpad],
+            // There are instances where an application's debug files are ELFs despite the
+            // executable not being ELFs themselves. It probably isn't correct to assume that any
+            // specific debug file type is heavily coupled with a particular executable type so we
+            // return a union of all possible debug file types for native applications.
+            ObjectType::Macho => &[
+                FileType::MachCode,
+                FileType::Breakpad,
+                FileType::MachDebug,
+                FileType::Pdb,
+                FileType::ElfDebug,
+            ],
+            ObjectType::Pe => &[
+                FileType::Pe,
+                FileType::Breakpad,
+                FileType::MachDebug,
+                FileType::Pdb,
+                FileType::ElfDebug,
+            ],
+            ObjectType::Elf => &[
+                FileType::ElfCode,
+                FileType::Breakpad,
+                FileType::MachDebug,
+                FileType::Pdb,
+                FileType::ElfDebug,
+            ],
             ObjectType::Wasm => &[FileType::WasmCode, FileType::WasmDebug],
             _ => Self::all(),
         }