Torch bin not packaged with flet build macos

[torablien]

When packaging an app for macOS (arm) with torch, the app builds but when launched, it gives a white screen with the error RuntimeError: Unable to find torch_shm_manager at /Users/admin/Library/Caches/flet_testing-1.0.0-1/app/__pypackages__/torch/bin/torch_shm_manager

If you manually copy the bin directory from your local torch install (e.g. flet-testing/venv/lib/python3.12/site-packages/torch/bin to Library/Caches/flet_testing-1.0.0-1/app/__pypackages__/torch) then the app runs as expected.

Code example to reproduce the issue:

import flet as ft
import torch

def main(page: ft.Page):
    page.add(ft.Text("Hello World!"))

ft.app(main)

App displays a blank white screen for 10 seconds (splash screen loading?) and then shows a white screen with the following error:

Traceback (most recent call last):
  File "<string>", line 43, in <module>
  File "<frozen runpy>", line 229, in run_module
  File "<frozen runpy>", line 88, in _run_code
  File "/var/folders/q1/dcc1rnq13f9fhs4hxd03jqy80000gn/T/serious_python_tempgFfj8Q/main.py", line 5, in <module>
  File "/var/folders/q1/dcc1rnq13f9fhs4hxd03jqy80000gn/T/serious_python_tempgFfj8Q/__pypackages__/torch/__init__.py", line 1427, in <module>
  File "/var/folders/q1/dcc1rnq13f9fhs4hxd03jqy80000gn/T/serious_python_tempgFfj8Q/__pypackages__/torch/__init__.py", line 1416, in manager_path
RuntimeError: Unable to find torch_shm_manager at /Users/admin/Library/Caches/flet_testing-1.0.0-1/app/__pypackages__/torch/bin/torch_shm_manager

Describe the results you expected:

App runs and shows Hello World

Additional information you deem important (e.g. issue happens only occasionally):

Once bin is copied over, it works as expected. Although, IIUC flet build does not use pyinstaller, this might be helpful: pytorch/pytorch#1898

Flet version (pip show flet):

% pip show flet
Name: flet
Version: 0.22.0
Summary: Flet for Python - easily build interactive multi-platform apps in Python
Home-page: 
Author: Appveyor Systems Inc.
Author-email: [email protected]
License: Apache-2.0
Location: /Users/admin/Code/flet-testing2/venv/lib/python3.12/site-packages
Requires: cookiecutter, fastapi, flet-runtime, packaging, qrcode, uvicorn, watchdog
Required-by: 

Give your requirements.txt file (don't pip freeze, instead give direct packages):

flet
torch

Operating system:

M1 Pro Mac - OS X Sonoma 14.1.2

Additional environment details:

Python 3.12.2

[FeodorFitsner]

Could you share your full build log please?

[FeodorFitsner]

Nevermind, I see "bin" directories are stripped off from the resulting installation.

[ap4499]

Is there a fix for this, without manually manipulating the caches folder?

In the Flet build.py, I can't see where the bin directories are being filtered out, or an argument to specify specific folders to include.

I need to distribute an executable, and I am struggling to come up with an elegant way of doing it (ie. zero user interaction).

[torablien]

Revisiting this and seems still reproducible in 0.24.1. Any suggestions on workarounds?

Nevermind, I see "bin" directories are stripped off from the resulting installation.

Is this something we could monkey patch as a workaround?

[ap4499]

Revisiting this and seems still reproducible in 0.24.1. Any suggestions on workarounds?

Nevermind, I see "bin" directories are stripped off from the resulting installation.

Is this something we could monkey patch as a workaround?

Not an official fix.. and I see that the Flet team are working on a version of Flet build that doesn't use the compiler..

But that aside, I have monkey patched. Part of the issue is that even if you include the missing bin in a folder (say resources) it is still stripped out.

To get around this, I have zipped it. Then, at runtime, I unzip it, and patch the cache using the unzipped bin file. (It has just occurred to me that it might have been possible to just name the folder something other than bin...)

Off-topic, but I have had to use the same approach to get the Pytorch optimizers working, and I used cx_Freeze to generate the missing pyc/py files and then monkey patch as you call it.

Can confirm that patching both at runtime works okay. I patch at the entry point to the app, before even processing imports.

[torablien]

Revisiting this and seems still reproducible in 0.24.1. Any suggestions on workarounds?

Nevermind, I see "bin" directories are stripped off from the resulting installation.

Is this something we could monkey patch as a workaround?

Not an official fix.. and I see that the Flet team are working on a version of Flet build that doesn't use the compiler..

But that aside, I have monkey patched. Part of the issue is that even if you include the missing bin in a folder (say resources) it is still stripped out.

To get around this, I have zipped it. Then, at runtime, I unzip it, and patch the cache using the unzipped bin file. (It has just occurred to me that it might have been possible to just name the folder something other than bin...)

Off-topic, but I have had to use the same approach to get the Pytorch optimizers working, and I used cx_Freeze to generate the missing pyc/py files and then monkey patch as you call it.

Can confirm that patching both at runtime works okay. I patch at the entry point to the app, before even processing imports.

Thanks, I also see the optimizer issues - do you happen to have a code pointer to your patch? Would love to give it a try!

[ap4499]

Here is a link to the patch: https://github.com/ap4499/PyTorchFlet_Patch/blob/main/main.py

In the folder are the files that you will need to patch Torch (and as it turns out, I had just renamed bin to bin_c to stop it being stripped out).

• Bin: as above.
• Torch.zip: There are surplus files, I created it using a separate build using cx_Freeze and extracted Torch from the built app. When patching, only a handful of files are copied over, so it could be slimmed down, but I haven't yet done this.
• It is likely that patching with the torch.zip would also resolve the bin issue in a single swipe.. as Bin will also be missing, and it is contained within torch.zip.
• It is a .zip to stop the Flet Build from pulling out any files that we want in there.

The class in main.py is the part that manages the unzip and then copies across and files that aren't found in the location, from the source. It is just to the application cache. Again, the Bin patching is likely unnecessary given the torch.zip process.

Apologies it is a bit unrefined - I've carved out that class from my code, and with the patching, I stumbled through it and haven't yet cleaned up the code yet.

Also, if you are signing, notarising and/or publishing to Mac AppStore, could you let me know how you get on? I'm finding it a real struggle to get it past notarization!

[torablien]

Here is a link to the patch: https://github.com/ap4499/PyTorchFlet_Patch/blob/main/main.py

In the folder are the files that you will need to patch Torch (and as it turns out, I had just renamed bin to bin_c to stop it being stripped out).

• Bin: as above.
• Torch.zip: There are surplus files, I created it using a separate build using cx_Freeze and extracted Torch from the built app. When patching, only a handful of files are copied over, so it could be slimmed down, but I haven't yet done this.
• It is likely that patching with the torch.zip would also resolve the bin issue in a single swipe.. as Bin will also be missing, and it is contained within torch.zip.
• It is a .zip to stop the Flet Build from pulling out any files that we want in there.

The class in main.py is the part that manages the unzip and then copies across and files that aren't found in the location, from the source. It is just to the application cache. Again, the Bin patching is likely unnecessary given the torch.zip process.

Apologies it is a bit unrefined - I've carved out that class from my code, and with the patching, I stumbled through it and haven't yet cleaned up the code yet.

Also, if you are signing, notarising and/or publishing to Mac AppStore, could you let me know how you get on? I'm finding it a real struggle to get it past notarization!

Thanks, I will give this a try! Were there a set of steps you followed when using cx_freeze? I will try following your approach and if I make any refinements, I'm happy to share it with you. No worries on the unrefined state - I get the gist with a quick scan so it's definitely readable. I appreciate you taking the effort to extract it out, this is helpful.

I don't have any immediate plans to publish to Mac App Store as this is an internal app right now but I have read that not notarizing can lead to other permissions issues so I may have to attempt that as well. If I do, I'll definitely let you know.

[ap4499]

Steps to get the Torch patch are:

I attempted to create patching files with both PyInstaller, and also the raw Torch files, but neither seemed to work (although I didn't spend much time debugging why this was - other users on MacOS had reported issues with Torch.Compile() when using PyInstaller. I believe it uses a type of JIT which is what causes the issue, hence trying the raw Torch files).

To create the patching file, I created a blank project and .venv, installed PyTorch (default installation for Apple/MPS from the PyTorch website). I am pretty sure that I used 2.3.1 (as at the time I did it, 2.4.0 had C++ runtime issues on Windows, which are patched in 2.4.1).

Then from there, just built it - with the only library, other than defaults, being PyTorch.

Then, on Mac:

1. Located the .app archive that is created by cx_Freeze.
2. Opened the contents of it (right click).
3. Located where the PyTorch files were.
4. Extracted them via copy/paste. This is then the .zip file that is contained in the repo above.

[FeodorFitsner]

So, bin folder has been "historically" removed during the packaging as it's not expected to be able to launch any executable on mobile phones.
Now I see it makes sense to include "bin" while packaging for desktops plus adding its path to system path, so any tools can be called right away. Do I understand right?

[torablien]

So, bin folder has been "historically" removed during the packaging as it's not expected to be able to launch any executable on mobile phones. Now I see it makes sense to include "bin" while packaging for desktops plus adding its path to system path, so any tools can be called right away. Do I understand right?

Yes, though it may not be necessary to always add to the system path as monkey patching bin for the torch case seems to work without editing the system path.

As @ap4499 has pointed out, it does seem like other necessary files are also not being included which is causing issues like with pytorch optimizers. Given that their patch of a simple 1:1 copy from the built output from cx_freeze seems to fix the issue (reproable and also fixes for me), it may make sense to generally avoid removing parts while packaging unless absolutely necessary. I've also run into somewhat similar issues with other packages. For example, the transformers package complains that the __init__.py file is missing (though the __init__.pyc is present) but if I just copy that from my venv directly into Library/Caches/app-name-1.0.0-1/app/__pypackages__/transformers, everything seems to work. A monkey patch routine on launch works but is, of course, less than ideal.

I don't know the history and am definitely not a packaging expert - was the historical motivation behind that to reduce package size and avoid crashes caused by potentially attempting to launch on mobile? Do you foresee any major issues by reducing the pruning for desktops?

[FeodorFitsner]

Got it. We will provide --compile option in the next release of packaging, to overcome .py/.pyc issue.

[ap4499]

Got it. We will provide --compile option in the next release of packaging, to overcome .py/.pyc issue.

Thanks @FeodorFitsner.

Could I ask where in your code the bin/.py/.pyc are being stripped out? Presumably it's in serious_python?

[FeodorFitsner]

Sure, it's here: https://github.com/flet-dev/serious-python/blob/main/src/serious_python/bin/package_command.dart#L329-L331

[ap4499]

@torablien

Unrelated to Flet dev for the most part. But I have signed, notarized, and cleared Gatekeeper.

No luck on the AppStore.. but I've seen many people struggle with it.. and I've given up.

#4031