Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Errors when follow README for 0.1.0 release #147

Closed
heyong4725 opened this issue Nov 16, 2022 · 0 comments · Fixed by #149
Closed

Errors when follow README for 0.1.0 release #147

heyong4725 opened this issue Nov 16, 2022 · 0 comments · Fixed by #149
Assignees
@haixuanTao

Comments

@heyong4725
Copy link
Collaborator

heyong4725 commented Nov 16, 2022 •
edited by phil-opp
Loading

Env: MacOS

I followed README to test v0.1.0 release, and have following errors, please take a look @haixuanTao

(base) ~/GitHub/dora-rs/bin/test1 $ dora start dataflow.yml
dyld[35552]: Library not loaded: /Library/Frameworks/Python.framework/Versions/3.11/Python
  Referenced from: <49D90856-1186-3169-BACA-0F128A85669B> /Users/yonghe/GitHub/dora-rs/bin/dora-runtime
  Reason: tried: '/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file), '/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file), '/System/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file, not in dyld cache)
dyld[35551]: Library not loaded: /Library/Frameworks/Python.framework/Versions/3.11/Python
  Referenced from: <49D90856-1186-3169-BACA-0F128A85669B> /Users/yonghe/GitHub/dora-rs/bin/dora-runtime
  Reason: tried: '/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file), '/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file), '/System/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file, not in dyld cache)
Traceback (most recent call last):
  File "/Users/yonghe/GitHub/dora-rs/bin/test1/./node_1/node_1.py", line 4, in <module>
    from dora import Node
ModuleNotFoundError: No module named 'dora'
  2022-11-16T17:37:32.302484Z ERROR dora_coordinator: error occured in dataflow `48b40fe9-3a44-4a63-b952-122583f5fb4c`

Caused by:
   0: failed to run dataflow at /Users/yonghe/GitHub/dora-rs/bin/test1/dataflow.yml
   1: custom node failed
   2: runtime node op_1 failed (unknown exit code)

Location:
    binaries/coordinator/src/run/runtime.rs:55:17
    at binaries/coordinator/src/lib.rs:90

48b40fe9-3a44-4a63-b952-122583f5fb4c
(base) ~/GitHub/dora-rs/bin/test1 $ ls ../
dataflow.yml                 dora-coordinator             dora-v0.1.0-x86_64-macOS.zip test1
dora                         dora-runtime                 iceoryx
(base) ~/GitHub/dora-rs/bin/test1 $ python --version
Python 3.9.7
(base) ~/GitHub/dora-rs/bin/test1 $ pip --version
pip 21.2.4 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)
(base) ~/GitHub/dora-rs/bin/test1 $
@haixuanTao haixuanTao self-assigned this Nov 16, 2022
haixuanTao added a commit that referenced this issue Nov 21, 2022
@haixuanTao
DRAFT: Fixing Python linking error
992b825 
This commit is an initial draft at fixing #147. The error is due to the
fact that pyo3 has linked the libpython from the compilation and not
trying to use libpython that is available in `LD_LIBRARY_PATH`.

The current only solution to disable the linking is to use the `extension-module` flag.
This requires to make the python `runtime-node` packaged in a python library.
The python `runtime-node` should also be fully compatible with the other operators in case we want hybrid runtime.

The issue that i'm facing is that. Due to the packaging, I have to deal with the `GIL` that is present from the start of
`dora-runtime` node. This however makes the process single threaded wich is impossible.

So, I have to disable it, but when I do, I have a race condition:
```bash
Exception ignored in: <module 'threading' from '/usr/lib/python3.8/threading.py'>
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 1373, in _shutdown
    assert tlock.locked()
AssertionError:
```
The issue is the same as PyO3/pyo3#1274

To fix this issue, I'm going to look again at the different step to make this work.

But this is my current investigation.
This was referenced Nov 21, 2022
Merged
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@haixuanTao haixuanTao

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixing Python linking error dora-rs/dora
2 participants
@haixuanTao @heyong4725