Build fails in OpenBSD

[anoduck]

@karlicoss I am a fan of your projects, so I just wanted to give this one a try. It is no biggie, it is not compatible with my system. More or less, just giving you a heads-up.

OpenBSD is kinda funky when it comes to cross compilation of rust resources with python. In order to compile solely rust packages, root privileges are required. The maintainer has been tinkering with rust to make it more compatible with OpenBSD, so this might change, but as of now cross compiling rust packages with python is a no-go.

Doing without, and making do, are all part of the game when running OpenBSD. I have toughened my skin to where it is a part of life.

[karlicoss]

Hey! The project itself isn't using rust so must be some dependencies -- and to be honest, have no idea which of them are using rust -- can you perhaps post more information what are you trying to install and which dependencies are failing? Thanks :)

[anoduck]

I apologize, I should have included this when I created the issue, and I am thankful you asked. It appears my first assumption was incorrect, maturin is doing fine. The build is generating an error during installation of dependencies for watchfiles-0.18.1. The dependency in question is cfg-if.

Below is my output of the error:

× Building wheel for watchfiles (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [56 lines of output]
      Running `maturin pep517 build-wheel -i /usr/local/bin/python3.10 --compatibility off`
      📡 Using build options bindings from pyproject.toml
         Compiling target-lexicon v0.12.3
         Compiling libc v0.2.124
         Compiling autocfg v1.1.0
         Compiling proc-macro2 v1.0.37
         Compiling once_cell v1.10.0
         Compiling cfg-if v1.0.0
      error: could not compile `cfg-if` due to 3 previous errors
      warning: build failed, waiting for other jobs to finish...
      error: could not compile `libc` due to 4 previous errors
      💥 maturin failed
        Caused by: Failed to build a native library through cargo
        Caused by: Cargo build finished with "exit status: 101": `"cargo" "rustc" "--release" "--target" "x86_64-apple-darwin" "--message-format" "json" "--lib" "--" "-C" "link-arg=-undefined" "-C" "link-arg=dynamic_lookup" "-C" "link-args=-Wl,-install_name,@rpath/_rust_notify.abi3.so"`
      📦 Including license file "LICENSE"
      🍹 Building a mixed python/rust project
      🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
      🐍 Not using a specific python interpreter
      error[E0463]: can't find crate for `core`
        |
        = note: the `x86_64-apple-darwin` target may not be installed
        = help: consider downloading the target with `rustup target add x86_64-apple-darwin`
      
      
      error[E0463]: can't find crate for `compiler_builtins`
      
      
      error[E0463]: can't find crate for `core`
        |
        = note: the `x86_64-apple-darwin` target may not be installed
        = help: consider downloading the target with `rustup target add x86_64-apple-darwin`
      
      
      error[E0463]: can't find crate for `compiler_builtins`
      
      
      error: aborting due to 2 previous errors
      
      
      For more information about this error, try `rustc --explain E0463`.
      
      error[E0432]: unresolved import `ffi::c_void`
          --> /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.124/src/unix/mod.rs:1498:17
           |
      1498 |         pub use ::ffi::c_void;
           |                 ^^^^^^^^^^^^^
      
      
      error: aborting due to 3 previous errors
      
      
      Some errors have detailed explanations: E0432, E0463.
      
      For more information about an error, try `rustc --explain E0432`.
      
      Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/usr/local/bin/python3.10', '--compatibility', 'off'] returned non-zero exit status 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for watchfiles
Successfully built httptools uvloop
Failed to build watchfiles
ERROR: Could not build wheels for watchfiles, which is required to install pyproject.toml-based projects

First thing noticed is it appears to by identifying my system as darwin and not berkeley unix. Second, I discovered cfg-if-1.0.0 is already installed on my system. Why it was not used I do not know, but since it was previously installed successfully, leads me to believe the issue is with watchfiles and not cfg-if. So, I am going to open an issue there.

[karlicoss]

Thanks for the info, looks like watchfiles is an optional dependency of uvicorn https://github.com/encode/uvicorn/blob/master/pyproject.toml#L44

Promnesia uses these optional dependencies as uvicorn[standard] here -- they give a performance boost normally.
Can you try changing uvicorn[standard] to uvicorn locally and see if your build works now and check that promnesia is happy?

If that works, perhaps the right thing to do would be to add some conditional logic to the setup file -- there are two options here

• either add it to promnesia (basically "if openbsd, then instann uvicorn, otherwise uvicorn[standard]")
• or to uvicorn itself, "if openbsd, then it should skip watchfiles" -- similar to uvloop here: https://github.com/encode/uvicorn/blob/master/pyproject.toml#L43

I'm more inclined to the latter since it would be more specific, but not sure if uvicorn developers would be happy with it -- you'll probably want to open an issue in their repo first.

[karlicoss]

ah sorry, also just realised I missed your last sentence 😂 yeah, if the issue with watchfiles itself and we can fix it there it would be perfect

[anoduck]

Well, I'll be a speckled hare. Moving from uvicorn[standard] to just uvicorn did the trick. I have to admit, I am rather impressed. You know your stuff, and that buys me time until watchfiles can be worked out.

I have had to allow firefox access to promnesia's db in firefox's unveil file, and will probably have to write a script to start promnesia when I start up my desktop, but for now she is clocking away.

[anoduck]

Alright, ran into a forsaken grep issue. I didn't know grep differed so much between OSs. I know enough python (still learning) to basically replicate and alter your modifications for grep in windows to have it use ggrep in openbsd. This catch happens A-LOT. So, I will fork and create a pull request when done.

[karlicoss]

@anoduck if available for openbsd I recommend you to install ripgrep -- much less headache :)