Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Frequent Killed: 9 with rust on M1 Mac #162795

Closed
n8henrie opened this issue Mar 4, 2022 · 21 comments
Closed

Frequent Killed: 9 with rust on M1 Mac #162795

n8henrie opened this issue Mar 4, 2022 · 21 comments
Labels
0.kind: bug Something is broken 6.topic: darwin Running or building packages on Darwin 6.topic: rust

Comments

@n8henrie
Copy link
Contributor

n8henrie commented Mar 4, 2022

Describe the bug

A clear and concise description of what the bug is.

I'm writing my first shell.nix:

{ pkgs ? import <nixpkgs> { } }:
with pkgs;
pkgs.mkShell {
  nativeBuildInputs = [
    cargo
    cargo-edit
    libiconv
    openssl
    pkg-config
    rust-analyzer
    rustc
    rustfmt
    darwin.apple_sdk.frameworks.Security
  ];
}

It runs a (currently trivial) Rust script:

use rss::Channel;

async fn get_channel() -> anyhow::Result<Channel> {
    let ua = "killed-9-example";
    let client = reqwest::Client::builder().user_agent(ua).build()?;
    let content = client
        .get("https://n8henrie.com/feed.xml")
        .send()
        .await?
        .bytes()
        .await?;
    let channel = Channel::read_from(&content[..])?;
    Ok(channel)
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let result = get_channel().await?;
    for item in result.items() {
        for cat in &item.categories {
            println!("{:?}", cat)
        }
    }
    Ok(())
}

The code runs and works as expected. Sometimes. Unfortunately, I find that when I make a few trivial changes, cargo run results in Killed: 9. cargo clean && cargo run and it usually works again.

I made an example repo https://github.com/n8henrie/killed-9-example -- though it looks like an even more trivial example reproduces the bug (example below).

Steps To Reproduce

Steps to reproduce the behavior:

$ git clone https://github.com/n8henrie/killed-9-example
$ cd ./killed-9-example
$ nix-shell
$ cargo run
... it works ... 
$ nvim -u NONE src/main.rs 
... make a trivial change ... 
$ cargo run
   Compiling killed-9-example v0.1.0 (/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.jfRdy7s1HQ/killed-9-example)
    Finished dev [unoptimized + debuginfo] target(s) in 0.58s
     Running `target/debug/killed-9-example`
Killed: 9
$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.03s
     Running `target/debug/killed-9-example`
Killed: 9
$ cargo clean
$ cargo run
... it works again ...

An even simpler example, with no Rust dependencies:

$ cargo new simpletest
     Created binary (application) `simpletest` package
$ cd simpletest/
$ cat <<'EOF' > shell.nix
{ pkgs ? import <nixpkgs> { } }:
with pkgs;
pkgs.mkShell {
  nativeBuildInputs = [
    cargo
    cargo-edit
    libiconv
    openssl
    pkg-config
    rust-analyzer
    rustc
    rustfmt
    darwin.apple_sdk.frameworks.Security
  ];
}
EOF
$ nix-shell
$ cargo run
   Compiling simpletest v0.1.0 (/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.n5qubIFyw2/simpletest)
    Finished dev [unoptimized + debuginfo] target(s) in 0.24s
     Running `target/debug/simpletest`
Hello, world!
$ nvim -u NONE src/main.rs
... trivial change, "hello world" -> "hello worlds"
$ cargo run
   Compiling simpletest v0.1.0 (/private/var/folders/kb/tw_lp_xd2_bbv0hqk4m0bvt80000gn/T/tmp.n5qubIFyw2/simpletest)
    Finished dev [unoptimized + debuginfo] target(s) in 0.14s
     Running `target/debug/simpletest`
Killed: 9
$ echo $?
137

Expected behavior

A clear and concise description of what you expected to happen.

cargo run works without Killed: 9

Screenshots

If applicable, add screenshots to help explain your problem.

I think the above examples should suffice

Notify maintainers

@Mic92 @LnL7 @zowoq

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"aarch64-darwin"`
 - host os: `Darwin 21.3.0, macOS 12.2.1`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.5pre20211007_844dd90`
 - channels(n8henrie): `""`
 - channels(root): `"darwin, nixpkgs-22.05pre358510.d34f7085a90"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`
@n8henrie n8henrie added the 0.kind: bug Something is broken label Mar 4, 2022
@zowoq zowoq added 6.topic: darwin Running or building packages on Darwin 6.topic: rust labels Mar 4, 2022
@Mic92
Copy link
Member

Mic92 commented Mar 6, 2022

Can you replace rustc with rustup and still reproduce the issue (when the same rust version is used)?

@n8henrie
Copy link
Contributor Author

n8henrie commented Mar 7, 2022

I thought it might be related to my editor interacting with rust-analyzer somehow, but it looks like that's not the case:

$ nix-shell --pure
[nix-shell:~/git/killed-9-example]$ cat src/main.rs
fn main() {
    println!("Hello, world!");
}
[nix-shell:~/git/killed-9-example]$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.00s
     Running `target/debug/killed-9-example`
Hello, world!

[nix-shell:~/git/killed-9-example]$ sed -i 's/world/you/' src/main.rs

[nix-shell:~/git/killed-9-example]$ cat src/main.rs
fn main() {
    println!("Hello, you!");
}

[nix-shell:~/git/killed-9-example]$ cargo run
   Compiling killed-9-example v0.1.0 (/Users/n8henrie/git/killed-9-example)
    Finished dev [unoptimized + debuginfo] target(s) in 0.16s
     Running `target/debug/killed-9-example`
Killed: 9

Can you replace rustc with rustup and still reproduce the issue

Yes:

Before:

[nix-shell:~/git/killed-9-example]$ rustc --version
rustc 1.58.1

After:

$ sed -i 's/rustc/rustup/' shell.nix
$ echo '1.58.1' > rust-toolchain
$ nix-shell --pure
[nix-shell:~/git/killed-9-example]$ rustc --version
rustc 1.58.1 (db9d1b20b 2022-01-20)

[nix-shell:~/git/killed-9-example]$ cargo run
   Compiling killed-9-example v0.1.0 (/Users/n8henrie/git/killed-9-example)
    Finished dev [unoptimized + debuginfo] target(s) in 0.85s
     Running `target/debug/killed-9-example`
Hello, you!

[nix-shell:~/git/killed-9-example]$ sed -i 's/you/world/' src/main.rs

[nix-shell:~/git/killed-9-example]$ cat src/main.rs
fn main() {
    println!("Hello, world!");
}

[nix-shell:~/git/killed-9-example]$ cargo run
   Compiling killed-9-example v0.1.0 (/Users/n8henrie/git/killed-9-example)
    Finished dev [unoptimized + debuginfo] target(s) in 0.21s
     Running `target/debug/killed-9-example`
Killed: 9

@Mic92
Copy link
Member

Mic92 commented Mar 8, 2022 •
edited
Loading

Ok. Is this not be a bug of the library/program rather than rustc/cargo? It does not look like it was cargo that was killed, but the program. You could try to attach lldb and report the backtrace to the crate author.

@n8henrie
Copy link
Contributor Author

n8henrie commented Mar 8, 2022

Prior to your edit:

Could this not be a bug of the library rather than rustc/cargo? You could try to attach lldb and report the backtrace to the crate author.

There are no external libraries or dependencies in the updated examples I've posted above.

Post edit:

It does not look like it was cargo that was killed, but the program. You could try to attach lldb and report the backtrace to the crate author.

This is the default "hello world" provided by cargo new -- there is no crate author to report to.

$ ls -A
shell.nix
$ \cat shell.nix
{ pkgs ? import <nixpkgs> { } }:
with pkgs;
pkgs.mkShell {
  nativeBuildInputs = [
    cargo
    cargo-edit
    libiconv
    openssl
    pkg-config
    rust-analyzer
    rustup
    rustfmt
    darwin.apple_sdk.frameworks.Security
  ];
}
$ nix-shell --pure
[nix-shell:]$ cargo new foo
     Created binary (application) `foo` package
[nix-shell:]$ cd foo
[nix-shell:foo]$ cargo run
   Compiling foo v0.1.0 (foo)
    Finished dev [unoptimized + debuginfo] target(s) in 1.01s
     Running `target/debug/foo`
Hello, world!
[nix-shell:foo]$ sed -i 's/world/you/' src/main.rs
[nix-shell:foo]$ cargo run
   Compiling foo v0.1.0 (foo)
    Finished dev [unoptimized + debuginfo] target(s) in 0.19s
     Running `target/debug/foo`
Killed: 9

@n8henrie
Copy link
Contributor Author

n8henrie commented Mar 8, 2022

Also, I don't see this behavior outside of nix-shell:

$ cargo new foo
     Created binary (application) `foo` package
$ cd foo/
$ cargo run
   Compiling foo v0.1.0 (foo)
    Finished dev [unoptimized + debuginfo] target(s) in 0.74s
     Running `target/debug/foo`
Hello, world!
$ sed -i 's/world/you/' src/main.rs
$ cargo run
   Compiling foo v0.1.0 (foo)
    Finished dev [unoptimized + debuginfo] target(s) in 0.10s
     Running `target/debug/foo`
Hello, you!

@Mic92
Copy link
Member

Mic92 commented Mar 8, 2022

I see. I cannot help here. I don't have the hardware to reproduce the bug.

@sternenseemann
Copy link
Member

log(1) could help you determine what exactly gets killed and why.

@n8henrie
Copy link
Contributor Author

Thank you for the suggestion. I'll take a look and report back soon.

@n8henrie
Copy link
Contributor Author

The following lines look suspicious, but this is above my level of expertise. I don't understand why changing a single line in the source code would produce this error in nix-shell but not in MacOS.

kernel: CODE SIGNING: cs_invalid_page(0x104ccc000): p=23225[killed-9-example] final status 0x23020200, denying page sending SIGKILL
kernel: CODE SIGNING: process 23225[killed-9-example]: rejecting invalid page at address 0x104ccc000 from offset 0x38000 in file "/

Full output:

$ log show --last 10m --debug --predicate 'eventMessage contains "killed-9-example"'
Timestamp                       Thread     Type        Activity             PID    TTL  
2022-03-19 08:45:39.172822-0600 0x1a566    Default     0x0                  0      0    kernel: (AppleMobileFileIntegrity) AMFI: '/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example' has no CMS blob?
2022-03-19 08:45:39.172829-0600 0x1a566    Default     0x0                  0      0    kernel: (AppleMobileFileIntegrity) AMFI: '/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example': Unrecoverable CT signature issue, bailing out.
2022-03-19 08:45:39.173474-0600 0x1a568    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) allowed via dev tool environment(/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:39.173979-0600 0x1a568    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Library (/nix/store/sv54j4kykmp0v3izfnd8ycl8kli4l2ij-libiconv-50/lib/libiconv.dylib) allowed in Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) via dev tool environment (/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:39.174004-0600 0x1a568    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Library (/nix/store/sv54j4kykmp0v3izfnd8ycl8kli4l2ij-libiconv-50/lib/libiconv.dylib) allowed in Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) via dev tool environment (/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:39.174091-0600 0x1a568    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Library (/nix/store/sv54j4kykmp0v3izfnd8ycl8kli4l2ij-libiconv-50/lib/libiconv-nocharset.dylib) allowed in Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) via dev tool environment (/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:39.174114-0600 0x1a568    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Library (/nix/store/sv54j4kykmp0v3izfnd8ycl8kli4l2ij-libiconv-50/lib/libiconv-nocharset.dylib) allowed in Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) via dev tool environment (/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:39.174186-0600 0x1a568    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Library (/nix/store/sv54j4kykmp0v3izfnd8ycl8kli4l2ij-libiconv-50/lib/libcharset.1.0.0.dylib) allowed in Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) via dev tool environment (/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:39.174211-0600 0x1a568    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Library (/nix/store/sv54j4kykmp0v3izfnd8ycl8kli4l2ij-libiconv-50/lib/libcharset.1.0.0.dylib) allowed in Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) via dev tool environment (/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:54.948309-0600 0x1a86f    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) allowed via dev tool environment(/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:54.948758-0600 0x1a86f    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Library (/nix/store/sv54j4kykmp0v3izfnd8ycl8kli4l2ij-libiconv-50/lib/libiconv.dylib) allowed in Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) via dev tool environment (/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:54.948782-0600 0x1a86f    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Library (/nix/store/sv54j4kykmp0v3izfnd8ycl8kli4l2ij-libiconv-50/lib/libiconv.dylib) allowed in Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) via dev tool environment (/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:54.948861-0600 0x1a86f    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Library (/nix/store/sv54j4kykmp0v3izfnd8ycl8kli4l2ij-libiconv-50/lib/libiconv-nocharset.dylib) allowed in Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) via dev tool environment (/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:54.948879-0600 0x1a86f    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Library (/nix/store/sv54j4kykmp0v3izfnd8ycl8kli4l2ij-libiconv-50/lib/libiconv-nocharset.dylib) allowed in Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) via dev tool environment (/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:54.948946-0600 0x1a86f    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Library (/nix/store/sv54j4kykmp0v3izfnd8ycl8kli4l2ij-libiconv-50/lib/libcharset.1.0.0.dylib) allowed in Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) via dev tool environment (/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:54.948964-0600 0x1a86f    Debug       0x0                  0      0    kernel: (AppleSystemPolicy) Library (/nix/store/sv54j4kykmp0v3izfnd8ycl8kli4l2ij-libiconv-50/lib/libcharset.1.0.0.dylib) allowed in Process (/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example) via dev tool environment (/System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal)
2022-03-19 08:45:54.949439-0600 0x1a86f    Default     0x0                  0      0    kernel: CODE SIGNING: cs_invalid_page(0x104ccc000): p=23225[killed-9-example] final status 0x23020200, denying page sending SIGKILL
2022-03-19 08:45:54.949452-0600 0x1a86f    Default     0x0                  0      0    kernel: CODE SIGNING: process 23225[killed-9-example]: rejecting invalid page at address 0x104ccc000 from offset 0x38000 in file "/Users/n8henrie/git/killed-9-example/target/debug/killed-9-example" (cs_mtime:1646679111.830753257 != mtime:1647701154.916071530) (signed:1 validated:1 tainted:1 nx:0 wpmapped:0 dirty:0 depth:0)
2022-03-19 08:45:54.949513-0600 0x1a86f    Default     0x0                  0      0    kernel: killed-9-example[23225] Corpse allowed 1 of 5
2022-03-19 08:45:55.486546-0600 0xa06f     Default     0x0                  1238   0    ReportCrash: Formulating fatal 309 report for corpse[23225] killed-9-example
2022-03-19 08:45:55.536960-0600 0x1a883    Default     0x0                  23268  0    osanalyticshelper: (OSAnalytics) creating type 309 as /Users/n8henrie/Library/Logs/DiagnosticReports/.killed-9-example-2022-03-19-084555.ips
2022-03-19 08:45:55.650391-0600 0x1a883    Default     0x0                  23268  0    osanalyticshelper: (OSAnalytics) Saved type '309(<private>)' report (1 of max 25) at /Users/n8henrie/Library/Logs/DiagnosticReports/killed-9-example-2022-03-19-084555.ips
2022-03-19 08:45:55.650962-0600 0x1a883    Default     0x0                  23268  0    osanalyticshelper: xpc log creation type 309 result success: /Users/n8henrie/Library/Logs/DiagnosticReports/killed-9-example-2022-03-19-084555.ips
2022-03-19 08:45:55.651118-0600 0xa06f     Default     0x0                  1238   0    ReportCrash: (OSAnalytics) client log create type 309 result success: /Users/n8henrie/Library/Logs/DiagnosticReports/killed-9-example-2022-03-19-084555.ips
2022-03-19 08:45:55.651171-0600 0xa06f     Default     0x0                  1238   0    ReportCrash: no MetricKit for process killed-9-example type 309 bundleId (null)
2022-03-19 08:45:55.651405-0600 0xa06f     Default     0x0                  1238   7    ReportCrash: (CoreAnalytics) [com.apple.CoreAnalytics.stability-event:event-send] Sending event: com.apple.stability.crash {"exceptionCodes":"0x0000000000000032, 0x0000000104ccc000(\n    50,\n    4375494656\n)EXC_BAD_ACCESSSIGKILL (Code Signature Invalid)UNKNOWN_0x32 at 0x0000000104ccc000","incidentID":"1BCF2A6D-3234-4DF2-8E02-524D84272CF3","logwritten":1,"process":"killed-9-example","responsibleApp":"Terminal","terminationReasonExceptionCode":"0x2","terminationReasonNamespace":"CODESIGNING"}
2022-03-19 08:45:55.651663-0600 0x19e4c    Default     0x0                  643    7    analyticsd: [com.apple.CoreAnalytics.stability-event:event-recv] Received event: com.apple.stability.crash {"exceptionCodes":"0x0000000000000032, 0x0000000104ccc000(\n    50,\n    4375494656\n)EXC_BAD_ACCESSSIGKILL (Code Signature Invalid)UNKNOWN_0x32 at 0x0000000104ccc000","incidentID":"1BCF2A6D-3234-4DF2-8E02-524D84272CF3","logwritten":1,"process":"killed-9-example","responsibleApp":"Terminal","terminationReasonExceptionCode":"0x2","terminationReasonNamespace":"CODESIGNING"}
2022-03-19 08:45:55.656449-0600 0x19e4c    Default     0x0                  643    7    analyticsd: [com.apple.CoreAnalytics.stability-event:event-aggregated] Aggregated. Transform: StabilityCrashNumerator3WithBundleVersion Dirty: 1 Event: com.apple.stability.crash {"exceptionCodes":"0x0000000000000032, 0x0000000104ccc000(\n    50,\n    4375494656\n)EXC_BAD_ACCESSSIGKILL (Code Signature Invalid)UNKNOWN_0x32 at 0x0000000104ccc000","incidentID":"1BCF2A6D-3234-4DF2-8E02-524D84272CF3","logwritten":1,"process":"killed-9-example","responsibleApp":"Terminal","terminationReasonExceptionCode":"0x2","terminationReasonNamespace":"CODESIGNING","timestamp":1647701155628861}
2022-03-19 08:45:55.656556-0600 0x19e4c    Default     0x0                  643    7    analyticsd: [com.apple.CoreAnalytics.stability-event:event-aggregated] Aggregated. Transform: StabilityCrashNumerator3WithIncidentID Dirty: 1 Event: com.apple.stability.crash {"exceptionCodes":"0x0000000000000032, 0x0000000104ccc000(\n    50,\n    4375494656\n)EXC_BAD_ACCESSSIGKILL (Code Signature Invalid)UNKNOWN_0x32 at 0x0000000104ccc000","incidentID":"1BCF2A6D-3234-4DF2-8E02-524D84272CF3","logwritten":1,"process":"killed-9-example","responsibleApp":"Terminal","terminationReasonExceptionCode":"0x2","terminationReasonNamespace":"CODESIGNING","timestamp":1647701155628861}
2022-03-19 08:45:55.656654-0600 0x19e4c    Default     0x0                  643    7    analyticsd: [com.apple.CoreAnalytics.stability-event:event-aggregated] Aggregated. Transform: StabilityCrashNumerator3 Dirty: 1 Event: com.apple.stability.crash {"exceptionCodes":"0x0000000000000032, 0x0000000104ccc000(\n    50,\n    4375494656\n)EXC_BAD_ACCESSSIGKILL (Code Signature Invalid)UNKNOWN_0x32 at 0x0000000104ccc000","incidentID":"1BCF2A6D-3234-4DF2-8E02-524D84272CF3","logwritten":1,"process":"killed-9-example","responsibleApp":"Terminal","terminationReasonExceptionCode":"0x2","terminationReasonNamespace":"CODESIGNING","timestamp":1647701155628861}

@a-h
Copy link
Contributor

a-h commented Mar 26, 2022

I have an M1 Mac, so I tried out your example repo, and couldn't reproduce the problem.

https://asciinema.org/a/gHz8aFVsGOGaG8xbB9txIHIB4

@n8henrie
Copy link
Contributor Author

n8henrie commented Mar 27, 2022 •
edited
Loading

@a-h thanks for chiming in! That's good to know, thanks for trying to reproduce.

EDIT: Just to be sure, did you try it out a few times? It doesn't happen every time.

@n8henrie
Copy link
Contributor Author

@a-h Also -- multi-user install?

$ nix --version
nix (Nix) 2.5pre20211007_844dd90

@a-h
Copy link
Contributor

a-h commented Mar 28, 2022 •
edited
Loading

I was on nix (Nix) 2.3.16 (multi-user) but I decided to update to test with later versions.

I removed Nix from my Mac, and reinstalled which brought in nix (Nix) 2.7.0 and also brought in a later version of rust (rustc 1.58.1) presumably due to updating channels, since once I started using it, I could run your example without first dropping the rust code in the toml file down to 2018.

However, still no problems for me. I tried it at least 10 times. I don't think it would make any difference about exactly which Mac I'm using, but I'm on an M1 Max MacBook Pro.

I probably don't use Nix in the way it seems to be designed - to create individual, minimal, reproducible dev environments. I'm usually the only person on a team that's using it, so I mostly use it to configure my system as a global level, and use a shell.nix from time-to-time. However, my entire config is here: https://github.com/a-h/dotfiles in case there's anything useful in there for you.

You don't have a virus scanner running do you? Killing a program randomly like that sounds like the sort of thing it might do.

@jasonrm
Copy link
Contributor

jasonrm commented Mar 28, 2022

@n8henrie you might try updating your nix channels. I has similar issues with MacOS killing cargo cbuild due to code signing issues on the nixos-21.11 channel, but a more recent unstable worked.

Unfortunately I couldn't get an example to fail with the nixpkgs-22.05pre358510.d34f7085a90 channel you are using, but updating might be worth a try.

test default.nix 
let
  bootstrap = import <nixpkgs> { };
  stable = import
    (bootstrap.fetchFromGitHub {
      owner = "NixOS";
      repo = "nixpkgs";
      # latest nixos-21.11 from https://channels.nix.gsc.io/nixos-21.11/history
      rev = "d89f18a17e51532ed5f4d45297b0ddf11e46b9c8";
      sha256 = "sha256-CbXZkiVTA0ByV/yncFHCAGIa/AfcSb0cjqzEa56rO+Y=";
      # sha256 = bootstrap.lib.fakeSha256;
    })
    { };
  unstable = import
    (bootstrap.fetchFromGitHub {
      owner = "NixOS";
      repo = "nixpkgs";
      # latest nixpkgs-unstable from https://channels.nix.gsc.io/nixpkgs-unstable/history
      rev = "30d3d79b7d3607d56546dd2a6b49e156ba0ec634";
      sha256 = "sha256-Ctij+dOi0ZZIfX5eMhgwugfvB+WZSrvVNAyAuANOsnQ=";
      # sha256 = bootstrap.lib.fakeSha256;
    })
    { };
  old-unstable = import
    (bootstrap.fetchFromGitHub {
      owner = "NixOS";
      repo = "nixpkgs";
      # latest nixpkgs-unstable from https://channels.nix.gsc.io/nixpkgs-unstable/history
      rev = "d34f7085a90431b36547226735905f0bb69cbafa";
      sha256 = "sha256-QqsfaWnJB4OT1ngvNnI3+dI3ByURgYNeC9vRq3AyfKw=";
      # sha256 = bootstrap.lib.fakeSha256;
    })
    { };

  mkExample = pkgs: pkgs.callPackage "${stable.path}/pkgs/tools/video/rav1e/default.nix" {};
in
{
  stable = mkExample stable;
  unstable = mkExample unstable;
  old-unstable = mkExample old-unstable;
}
❯ nix-build -A old-unstable
/nix/store/5wf8jyxc2bv6b226926x15cffizpn69s-rav1e-0.4.1

❯ nix-build -A unstable
/nix/store/jlh463m86ij3v27nr0qyy60p7r0gn29x-rav1e-0.4.1

❯ nix-build -A stable
# fails
nix build log 
  ❯ nix log /nix/store/100cwmf9s70bsys8i44s0m3vmwsjc6kr-rav1e-0.4.1.drv | cat
  @nix { "action": "setPhase", "phase": "unpackPhase" }
  unpacking sources
  unpacking source archive /nix/store/4wy9h45nmp4gaz6ag47k5p7ckmns1vfn-rav1e-0.4.1.tar.gz
  source root is rav1e-0.4.1.tar.gz
  Executing cargoSetupPostUnpackHook
  unpacking source archive /nix/store/7493flr50r8v22s3n487lyxzh21d9yfr-rav1e-0.4.1-vendor.tar.gz
  Finished cargoSetupPostUnpackHook
  @nix { "action": "setPhase", "phase": "patchPhase" }
  patching sources
  Executing cargoSetupPostPatchHook
  Validating consistency between /private/tmp/nix-build-rav1e-0.4.1.drv-0/rav1e-0.4.1.tar.gz//Cargo.lock and /private/tmp/nix-build-rav1e-0.4.1.drv-0/rav1e-0.4.1-vendor.tar.gz/Cargo.lock
  Finished cargoSetupPostPatchHook
  @nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" }
  updateAutotoolsGnuConfigScriptsPhase
  @nix { "action": "setPhase", "phase": "configurePhase" }
  configuring
  @nix { "action": "setPhase", "phase": "buildPhase" }
  building
  Executing cargoBuildHook
  ++ env CC_aarch64-apple-darwin=/nix/store/c2zmgzjaz0zf30in2fqr4qhzypc148kl-clang-wrapper-11.1.0/bin/cc CXX_aarch64-apple-darwin=/nix/store/c2zmgzjaz0zf30in2fqr4qhzypc148kl-clang-wrapper-11.1.0/bin/c++ CC_aarch64-apple-darwin=/nix/store/c2zmgzjaz0zf30in2fqr4qhzypc148kl-clang-wrapper-11.1.0/bin/cc CXX_aarch64-apple-darwin=/nix/store/c2zmgzjaz0zf30in2fqr4qhzypc148kl-clang-wrapper-11.1.0/bin/c++ cargo build -j 20 --target aarch64-apple-darwin --frozen --release
     Compiling autocfg v1.0.1
     Compiling libc v0.2.92

     <snip>

     Compiling thiserror v1.0.24
     Compiling v_frame v0.2.1
     Compiling av-metrics v0.6.2
  warning: unused attribute `allow`
     --> src/context/partition_unit.rs:462:5
      |
  462 |     #[allow(dead_code)]
      |     ^^^^^^^^^^^^^^^^^^^
      |
      = note: `#[warn(unused_attributes)]` on by default
  note: the built-in attribute `allow` will be ignored, since it's applied to the macro invocation `assert`
     --> src/context/partition_unit.rs:463:5
      |
  463 |     assert!(bsize.is_sqr());
      |     ^^^^^^

  warning: `#[macro_use]` only has an effect on `extern crate` and modules
     --> src/context/cdf_context.rs:464:1
      |
  464 | #[macro_use]
      | ^^^^^^^^^^^^

  warning: `rav1e` (lib) generated 2 warnings
      Finished release [optimized + debuginfo] target(s) in 26.14s
  /nix/store/g307yhp2l4i43l9zz0fc120m37c123yc-stdenv-darwin/setup: line 95: 26970 Killed: 9               cargo cbuild --release --frozen --prefix=/nix/store/1rcl2xvngzzv89j5wrhscdkzypjk2brn-rav1e-0.4.1 --target aarch64-apple-darwin
MacOS Crash Report 
  -------------------------------------
  Translated Report (Full Report Below)
  -------------------------------------

  Incident Identifier: 49FC5CD0-CAA4-44DD-B291-A298B0C73763
  CrashReporter Key:   07DB2E62-1362-CDA7-09E4-3330BC346916
  Hardware Model:      Mac13,2
  Process:             cargo-cbuild [79411]
  Path:                /Volumes/VOLUME/*/cargo-cbuild
  Identifier:          cargo-cbuild
  Version:             ???
  Code Type:           ARM-64 (Native)
  Role:                Unspecified
  Parent Process:      bash [78082]
  Coalition:           org.nixos.nix-daemon [5202]
  Responsible Process: nix [19123]

  Date/Time:           2022-03-26 18:07:09.3915 -0700
  Launch Time:         2022-03-26 18:07:09.3711 -0700
  OS Version:          macOS 12.3 (21E230)
  Release Type:        User
  Report Version:      104

  Exception Type:  EXC_BAD_ACCESS (SIGKILL (Code Signature Invalid))
  Exception Subtype: UNKNOWN_0x32 at 0x0000000104730000
  Exception Codes: 0x0000000000000032, 0x0000000104730000
  VM Region Info: 0x104730000 is in 0x104730000-0x1050b8000;  bytes after start: 0  bytes before end: 9994239
        REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
        UNUSED SPACE AT START
  --->  mapped file                 104730000-1050b8000    [ 9760K] r-x/r-x SM=COW  ...ct_id=5fed3b7
        mapped file                 1050b8000-105130000    [  480K] rw-/rw- SM=COW  ...ct_id=5fed3b7
  Exception Note:  EXC_CORPSE_NOTIFY
  Termination Reason: CODESIGNING 2 

  Highlighted by Thread:  0

  Backtrace not available

  No thread state (register information) available

  Binary Images:
                 0x0 - 0xffffffffffffffff ??? (*) <00000000-0000-0000-0000-000000000000> ???

  Error Formulating Crash Report:
  _dyld_process_info_create failed with 6
  dyld_process_snapshot_get_shared_cache failed
  Failed to create CSSymbolicatorRef - corpse still valid ¯\_(ツ)_/¯
  thread_get_state(PAGEIN) returned 0x10000003: (ipc/send) invalid destination port
  thread_get_state(EXCEPTION) returned 0x10000003: (ipc/send) invalid destination port
  thread_get_state(FLAVOR) returned 0x10000003: (ipc/send) invalid destination port
Versions 
❯ nix --version
nix (Nix) 2.6.0

❯ sw_vers
ProductName:    macOS
ProductVersion: 12.3
BuildVersion:   21E230

❯ nix-daemon --version
nix-daemon (Nix) 2.6.0

❯ nix-channel --list
home-manager https://github.com/nix-community/home-manager/archive/master.tar.gz
nixpkgs https://github.com/NixOS/nixpkgs/archive/98bb5b77c8c6666824a4c13d23befa1e07210ef1.tar.gz

❯ sudo -i nix-channel --list
nixpkgs https://github.com/NixOS/nixpkgs/archive/98bb5b77c8c6666824a4c13d23befa1e07210ef1.tar.gz

98bb5b7 is an older nixos-unstable. Why 98bb5b7? Because newer broke something else, lol (due to a different issue, maybe #160876)

@thefloweringash
Copy link
Member

thefloweringash commented Mar 29, 2022 •
edited
Loading

I can reproduce the problem on recent master (f505412), but not in a way that makes it easy to debug.

[nix-shell:~/src/killed-9-example]$ i=0; while /run/current-system/sw/bin/git checkout src/main.rs && sed -i "s/world/$((i++))/" src/main.rs && cargo run; do :; done

[... snip runs 0 to 467 ...]

Updated 1 path from the index
   Compiling killed-9-example v0.1.0 (/Users/lorne/src/killed-9-example)
    Finished dev [unoptimized + debuginfo] target(s) in 0.17s
     Running `target/debug/killed-9-example`
Hello, 468!
Updated 1 path from the index
   Compiling killed-9-example v0.1.0 (/Users/lorne/src/killed-9-example)
    Finished dev [unoptimized + debuginfo] target(s) in 0.16s
     Running `target/debug/killed-9-example`
Killed: 9

But this only seems to be a transient error, and the resulting binary is valid if I try to run it again:

[nix-shell:~/src/killed-9-example]$ target/debug/killed-9-example
Hello, 469!

There's also been a case where the result cached on hydra is persistently invalid, but succeeds after rebuilding. Maybe it's unrelated to my reproduction. In the case where it's easily reproducible, is the resulting binary properly ad-hoc signed?

❯ nix build github:NixOs/nixpkgs/nixos-21.11#cargo-c && ./result/bin/cargo-cbuild
zsh: killed     ./result/bin/cargo-cbuild
❯ codesign -vv ./result/bin/cargo-cbuild
./result/bin/cargo-cbuild: invalid signature (code or signature have been modified)
In architecture: arm64

❯ codesign -vv target/debug/killed-9-example
target/debug/killed-9-example: valid on disk
Versions 
❯ nix --version
nix (Nix) 2.7.0

❯ sw_vers
ProductName:	macOS
ProductVersion:	12.3
BuildVersion:	21E230

❯ nix-daemon --version
nix-daemon (Nix) 2.7.0

@a-h
Copy link
Contributor

a-h commented Mar 29, 2022 •
edited
Loading

Ah, I wasn't persistent enough. I did think about scripting it, but didn't follow through.

Using your little script, I was also able to reproduce the error after 843 times...

Hello, 843!
Updated 1 path from the index
   Compiling killed-9-example v0.1.0 (/Users/adrian/github.com/killed-9-example)
    Finished dev [unoptimized + debuginfo] target(s) in 0.17s
     Running `target/debug/killed-9-example`
Killed: 9

It seems unlikely that this is a problem with Nix though.

Installing cargo etc. using an alternative mechanism, then running the same repeated test would find out. Because if it happens using a different installation mechanism, it can't be anything to do with Nix.

@n8henrie
Copy link
Contributor Author

@a-h

You don't have a virus scanner running do you?

Nope. Again, thanks for your efforts here.

@jasonrm

you might try updating your nix channels

No luck. Could using nix-darwin be an issue here?

$ sudo -i nix-channel --list
darwin https://github.com/LnL7/nix-darwin/archive/master.tar.gz
nixpkgs https://nixos.org/channels/nixpkgs-unstable
$ sudo -i nix-channel --update
unpacking channels...
$ nix-shell --pure
[nix-shell:~/git/killed-9-example]$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/killed-9-example`
Killed: 9

Tried cargo clean and a few reruns, no luck.

@thefloweringash

Odd, it's basically instant for me, and resulting binary is no good if I try to re-execute. Looks like codesigning worked.

$ git checkout src/main.rs
Updated 0 paths from the index
$ nix-shell --pure
[nix-shell:~/git/killed-9-example]$ cargo clean
[nix-shell:~/git/killed-9-example]$ i=0; while cargo run; do sed -i "s/world[0-9]*/world$((i++))/" src/main.rs; done
   Compiling killed-9-example v0.1.0 (/Users/n8henrie/git/killed-9-example)
    Finished dev [unoptimized + debuginfo] target(s) in 0.50s
     Running `target/debug/killed-9-example`
Hello, world!
   Compiling killed-9-example v0.1.0 (/Users/n8henrie/git/killed-9-example)
    Finished dev [unoptimized + debuginfo] target(s) in 0.19s
     Running `target/debug/killed-9-example`
Killed: 9
[nix-shell:~/git/killed-9-example]$ ./target/debug/killed-9-example
Killed: 9
[nix-shell:~/git/killed-9-example]$ codesign
bash: codesign: command not found
[nix-shell:~/git/killed-9-example]$ exit
exit
$ codesign -vv ./target/debug/killed-9-example
./target/debug/killed-9-example: valid on disk
./target/debug/killed-9-example: satisfies its Designated Requirement

I don't seem to have cargo-cbuild, and my cargo and rustc are not codesigned. Should they be?

[nix-shell:~/git/killed-9-example]$ command -v cargo-cbuild
[nix-shell:~/git/killed-9-example]$ command -v cargo
/nix/store/sakd69ff7clis6b5w46cxjblgkl7r904-cargo-1.58.1/bin/cargo
[nix-shell:~/git/killed-9-example]$ command -v rustc
/nix/store/sfia0kg9dz6ii1i20jh14ijdpxky98sm-rustup-1.24.3/bin/rustc
[nix-shell:~/git/killed-9-example]$ exit
exit
$ codesign -vv /nix/store/sfia0kg9dz6ii1i20jh14ijdpxky98sm-rustup-1.24.3/bin/rustc
/nix/store/sfia0kg9dz6ii1i20jh14ijdpxky98sm-rustup-1.24.3/bin/rustc: code object is not signed at all
$ codesign -vv /nix/store/sakd69ff7clis6b5w46cxjblgkl7r904-cargo-1.58.1/bin/cargo
/nix/store/sakd69ff7clis6b5w46cxjblgkl7r904-cargo-1.58.1/bin/cargo: code object is not signed at all

@a-h

Because if it happens using a different installation mechanism, it can't be anything to do with Nix.

Well I'll be darned, you're right -- same crash outside of nix. Sorry to waste so much time! I've never seen it outside of nix so I was surprised.

Ends up, I think it's perhaps a problem with 1.58.1-aarch64-apple-darwin, which I pinned above but didn't commit to the repo (until just now).

I've now added the toolchain file and an example crasher script to the repo, though I'm also closing the issue as it looks like a rust problem and not nix.

Thanks so much to all for your help and again sorry for the time spent!

@n8henrie
Copy link
Contributor Author

Well, it works outside of nix now (crashme.sh runs without crashing), likely since it's using 1.59, but still crashes in nix-shell, likely because it's still at 1.58.1 there. ¯\_(ツ)_/¯

$ rustc --version
rustc 1.59.0 (9d1b2106e 2022-02-23)
$ nix-shell --pure

[nix-shell:~/git/killed-9-example]$ rustc --version
rustc 1.58.1

[nix-shell:~/git/killed-9-example]$ cat rust-toolchain.toml
[toolchain]
# channel = "1.58.1"
channel = "stable"

[nix-shell:~/git/killed-9-example]$ ./crashme.sh
Updated 1 path from the index
   Compiling killed-9-example v0.1.0 (/Users/n8henrie/git/killed-9-example)
    Finished dev [unoptimized + debuginfo] target(s) in 0.32s
     Running `target/debug/killed-9-example`
Hello, world!
   Compiling killed-9-example v0.1.0 (/Users/n8henrie/git/killed-9-example)
    Finished dev [unoptimized + debuginfo] target(s) in 0.15s
     Running `target/debug/killed-9-example`
./crashme.sh: line 10: 75979 Killed: 9               cargo run

Some seemingly related issues:

I suppose I should still close, with the only pending issue being to wait for nixpkgs-unstable to update to 1.59.0?

@sternenseemann
Copy link
Member

#165406 has the rustc update iirc.

@n8henrie n8henrie mentioned this issue May 3, 2022
Closed
@n8henrie
Copy link
Contributor Author

n8henrie commented May 3, 2022 •
edited
Loading

I'm wondering if this may have been related to using michaeleisel/zld#110 the whole time; today I started getting the same crashes outside of nix, using rustc 1.60.0.

I've updated my example repo to a flake: https://github.com/n8henrie/killed-9-example

Interestingly, I can run nix develop -i in that repo and reproduce the crash -- but when I remove the zld relevant config from ~/.cargo/config, the crash also resolves in the nix "environment" (I guess not a "nix shell" anymore? what's the terminology here?).

Why is that? I had thought that nix develop -i should be reasonably close to nix-shell --pure, and that my ~/.cargo/config shouldn't even be picked up.

Should I be using something else for highly reproducibility / pure-ness with nix flakes?

@n8henrie
Copy link
Contributor Author

zld 1.3.4 has been released and includes a fix for this issue: https://github.com/michaeleisel/zld/releases/tag/1.3.4

@winterqt winterqt mentioned this issue Nov 11, 2023
Open
@n8henrie n8henrie mentioned this issue Mar 18, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
0.kind: bug Something is broken 6.topic: darwin Running or building packages on Darwin 6.topic: rust
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@jasonrm @thefloweringash @Mic92 @a-h @n8henrie @sternenseemann @zowoq