Cannot find type NSUInteger in this scope

[troelsfr]

Minimum reproducible example

Checkout the latest version of the repository on a Mac using Apple Silicon. Run:

cargo run --release --features metal --bin mistralrs-server

Error

The main error is related to the fork of candle-core, line 211: https://github.com/EricLBuehler/candle/blob/main/candle-core/src/quantized/metal.rs

Here is the output:

 % cargo run --release --features metal --bin mistralrs-server
   Compiling candle-core v0.6.1 (https://github.com/EricLBuehler/candle.git?rev=8a99f7c#8a99f7cf)
   Compiling try_match v0.4.2
   Compiling monostate v0.1.13
   Compiling sparsevec v0.2.0
   Compiling uuid v0.8.2
   Compiling system-configuration v0.6.1
   Compiling hyper-tls v0.6.0
   Compiling pyo3 v0.22.2
   Compiling ureq v2.10.1
   Compiling rayon-cond v0.3.0
   Compiling onig v6.4.0
   Compiling rust-embed-impl v8.5.0
error[E0412]: cannot find type `NSUInteger` in this scope
   --> /Users/tfr/.cargo/git/checkouts/candle-c6a149c3b35a488f/8a99f7c/candle-core/src/quantized/metal.rs:211:65
    |
211 |         let size = (self.count * self.dtype.size_in_bytes()) as NSUInteger;
    |                                                                 ^^^^^^^^^^ not found in this scope
    |
help: consider importing this type alias
    |
1   + use metal::NSUInteger;
    |

   Compiling darling v0.11.0
   Compiling pyo3-macros v0.22.2
   Compiling derive_builder v0.20.1
   Compiling nix v0.29.0
   Compiling utoipa-gen v4.3.0
   Compiling crossterm v0.25.0
   Compiling rust-embed-utils v8.5.0
   Compiling esaxx-rs v0.1.10
   Compiling matchers v0.1.0
   Compiling memoffset v0.9.1
   Compiling dirs v5.0.1
   Compiling cfgrammar v0.13.7
error[E0609]: no field `count` on type `&metal::QMetalStorage`
   --> /Users/tfr/.cargo/git/checkouts/candle-c6a149c3b35a488f/8a99f7c/candle-core/src/quantized/metal.rs:211:26
    |
211 |         let size = (self.count * self.dtype.size_in_bytes()) as NSUInteger;
    |                          ^^^^^ unknown field
    |
    = note: available fields are: `dtype`, `device`, `buffer`

   Compiling pyo3-ffi v0.22.2
error[E0599]: no method named `size_in_bytes` found for enum `quantized::GgmlDType` in the current scope
   --> /Users/tfr/.cargo/git/checkouts/candle-c6a149c3b35a488f/8a99f7c/candle-core/src/quantized/metal.rs:211:45
    |
211 |         let size = (self.count * self.dtype.size_in_bytes()) as NSUInteger;
    |                                             ^^^^^^^^^^^^^ method not found in `GgmlDType`
    |
   ::: /Users/tfr/.cargo/git/checkouts/candle-c6a149c3b35a488f/8a99f7c/candle-core/src/quantized/mod.rs:144:1
    |
144 | pub enum GgmlDType {
    | ------------------ method `size_in_bytes` not found for this enum

error[E0609]: no field `count` on type `&metal::QMetalStorage`
   --> /Users/tfr/.cargo/git/checkouts/candle-c6a149c3b35a488f/8a99f7c/candle-core/src/quantized/metal.rs:223:44
    |
223 |         Ok(read_to_vec::<u8>(&buffer, self.count))
    |                                            ^^^^^ unknown field
    |
    = note: available fields are: `dtype`, `device`, `buffer`

   Compiling rustls-pemfile v2.1.3
   Compiling Inflector v0.11.4
   Compiling str_inflector v0.12.0
   Compiling strum_macros v0.26.4
   Compiling sharded-slab v0.1.7
   Compiling futures-executor v0.3.30
   Compiling minijinja v2.2.0
   Compiling itertools v0.12.1
   Compiling tracing-log v0.2.0
   Compiling serde_path_to_error v0.1.16
Some errors have detailed explanations: E0412, E0599, E0609.
For more information about an error, try `rustc --explain E0412`.
error: could not compile `candle-core` (lib) due to 4 previous errors
warning: build failed, waiting for other jobs to finish...

Other information

The error was not present in v0.2.4. This can be verified by checking out the version and rerunning the command.

Latest commit or version

This was done using the latest version of the repo.

[vlbosch]

I am running into the same error with the latest commit eadebac

[ken0x0a]

=> EricLBuehler/candle#27

[EricLBuehler]

@ken0x0a @vlbosch @troelsfr I opened #792 which should make some progress to solving the issue, can you please try it out?

[harrychin]

The build failed for me locally: cargo install --git https://github.com/EricLBuehler/mistral.rs mistralrs-server --branch metal_build_fix_nsu --features "metal"

error[E0004]: non-exhaustive patterns: DType::I16 not covered

I made a PR to fix the issue in candle-core you can pull in @EricLBuehler which fixed my local build.

[EricLBuehler]

@harrychin thanks for the PR! I merged it and #792 is updated, can you please confirm that it works there?

[harrychin]

@EricLBuehler done, thank you!

[EricLBuehler]

@harrychin thank you for testing! I merged #792, so the fix should be available on master.

@troelsfr I think this should be fixed, can you please confirm?

[EricLBuehler]

Closing as @harrychin has reported this is fixed, please feel free to reopen!

[troelsfr]

@EricLBuehler sorry for the delay in responding and thanks for the quick turn around time on this issue.

I can confirm it works on my end! Thanks a lot