Core ML model generation stuck

[servin]

Core ML generate models never finishes on my M1 Pro Macbook , any suggestions?

(venv10) servin@Ericks-MacBook-Pro whisper.cpp % ./models/generate-coreml-model.sh large
100%|█████████████████████████████████████| 2.87G/2.87G [06:32<00:00, 7.86MiB/s]
ModelDimensions(n_mels=80, n_audio_ctx=1500, n_audio_state=1280, n_audio_head=20, n_audio_layer=32, n_vocab=51865, n_text_ctx=448, n_text_state=1280, n_text_head=20, n_text_layer=32)
/Users/servin/Documents/GitHub/whisper.cpp/venv10/lib/python3.10/site-packages/whisper/model.py:166: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert x.shape[1:] == self.positional_embedding.shape, "incorrect audio shape"
/Users/servin/Documents/GitHub/whisper.cpp/venv10/lib/python3.10/site-packages/whisper/model.py:97: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  scale = (n_state // self.n_head) ** -0.25
Converting PyTorch Frontend ==> MIL Ops: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉| 2611/2612 [00:00<00:00, 3211.35 ops/s]
Running MIL frontend_pytorch pipeline: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 30.83 passes/s]
Running MIL default pipeline: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 57/57 [00:18<00:00,  3.06 passes/s]
Running MIL backend_mlprogram pipeline: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 278.68 passes/s]

[servin]

servin@192 ~ % system_profiler SPSoftwareDataType SPHardwareDataType
Software:

    System Software Overview:

      System Version: macOS 13.3.1 (a) (22E772610a)
      Kernel Version: Darwin 22.4.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: xxxxxx
      User Name: xxxxx (servin)
      Secure Virtual Memory: Enabled
      System Integrity Protection: Enabled
      Time since boot: 4 hours, 54 minutes

Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro18,1
      Model Number: MK1E3E/A
      Chip: Apple M1 Pro
      Total Number of Cores: 10 (8 performance and 2 efficiency)
      Memory: 16 GB
      System Firmware Version: 8422.100.650
      OS Loader Version: 8422.100.650
      Serial Number (system): R004F2GTJJ
      Hardware UUID: xxxxxx
      Provisioning UDID: xxxxxxx
      Activation Lock Status: Enabled

[frahlg]

I am experiencing a similar problem. The ANECompilerService appears to be running continuously, with 100% CPU utilization on one core, even after 9 hours. Is it possible that the process is just taking an unusually long time? Has anyone found a solution or workaround for this issue?

[frahlg]

And my is more or less the same as yours.

Software:

System Software Overview:

  System Version: macOS 13.3.1 (a) (22E772610a)
  Kernel Version: Darwin 22.4.0
  Boot Volume: Macintosh HD
  Boot Mode: Normal
  Computer Name: xxxx
  User Name: .....
  Secure Virtual Memory: Enabled
  System Integrity Protection: Enabled
  Time since boot: 5 days, 32 minutes

Hardware:

Hardware Overview:

  Model Name: MacBook Pro
  Model Identifier: MacBookPro18,3
  Model Number: MKGP3KS/A
  Chip: Apple M1 Pro
  Total Number of Cores: 8 (6 performance and 2 efficiency)
  Memory: 16 GB
  System Firmware Version: 8422.100.650
  OS Loader Version: 8422.100.650
  Serial Number (system): xxxxxx
  Hardware UUID: xxxx
  Provisioning UDID: xxxx
  Activation Lock Status: Disabled

[jamexhuang]

Same problem as yours.

Software:

  System Version: macOS 13.3.1 (22E261)
  Kernel Version: Darwin 22.4.0
  Boot Volume: Macintosh HD
  Boot Mode: Normal
  Computer Name: xxxxxx
  User Name: xxxxxx
  Secure Virtual Memory: Enabled
  System Integrity Protection: Enabled
  Time since boot: 17 hours

Hardware:

  Model Name: MacBook Pro
  Model Identifier: MacBookPro18,1
  Model Number: xxxxxx
  Chip: Apple M1 Pro
  Total Number of Cores: 10 (8 performance and 2 efficiency)
  Memory: 16 GB
  System Firmware Version: 8422.100.650
  OS Loader Version: 8422.100.650
  Serial Number (system): xxxxxx
  Hardware UUID: xxxxxx
  Provisioning UDID: xxxxxx
  Activation Lock Status: Enabled

``

[frahlg]

I had mine running for 18h before I gave up and stopped the process. If anyone are able to get this to work, or have a precompiled version, I would be happy for some advice or a link for downloading the large model.

[krutilin]

Same issue. 13.3.1 (22E261) Apple M1 Pro 16GB.

[krutilin]

btw there are already generated models:

./models/download-ggml-model.sh medium

and it works out of the box.

[frahlg]

Thanks I am aware, but this is about compilation of the CoreML-models. Quite intrigued about the performance.

[servin]

I was able to download and compile tiny and base models. So everything points to the size of the model and the ram required

[guris12]

can you please share the steps ?

[ArtemBernatskyy]

is there some way to download precompiled models for large/medium with CoreML? I tried on MBP 2021 (32RAM) but stuck... Thx!

[ridane]

same issue on : 13.3.1 - m1 max - 64 ram 😕

[dietoff]

Same issue - but it seems to eventually work for me - it is just very slow. Try first with base.en - it should be done within minutes. medium took several hours for me.

[ArtemBernatskyy]

BTW, found that someone already generated #566 (comment)

[krutilin]

It worked for me too. But you need to wait, about 3-5 hours for medium model. And then some huge time for the first run. But after all is quite fast with CoreML.