Windows: Import error after pip install #813
Comments
|
Hi @Tilix4 ! |
|
Hi @apetrynet I've read this thread, but I cannot find I'm installing with pip's |
|
What command line did you use to do the pip install? I notice that you used --target, but it's hard to guess what's going on without the whole command line. |
|
I noticed blender in there. Could it be related to #667? |
|
My command line is very basic: @apetrynet I've also tried with the Blender python, but I have the same issue with the windows python only, then Blender is not a topic for now ;) |
|
I am getting the same error message under different circumstances. I'm using OpenTimelineIO 0.13.0 in a Python script, and using PyInstaller to bundle it up as an .exe for use around the studio. When bundled, running the .exe on a different machine throws this error: As far as I can tell (and I'm not confident about this), either PyInstaller or the frozen OpenTimelineIO code isn't coping properly with the relative path inherent in the line Short of an update to OpenTimelineIO, might anyone be able to suggest an adjustment to this opentime.py file that would make it PyInstaller-friendly please? I'm not familiar with the structure of the code within site-packages/opentimelineio. |
|
@donquibeats pyinstaller can be a bit tricky. I think there is a way to unzip the contents of a pyinstaller archive, are you able to find the DLLs in there? Also, which python version are you using to build the pyinstaller package? |
|
Thanks for the responses. This is a Python 3.7 project, and PyInstaller creates a folder containing the .exe and a bunch of associated DLL's, but opentime.py and opentimelineio.py are not among them. However, copying these DLL's manually from cxx-libs/bin into the same folder as the .exe doesn't fix the problem, unfortunately. I'm not currently aware of how I would test whether the DLL's are bundled inside the .EXE. Strangely, if we target OpenTimelineIO 0.11.0 then it works fine. This error only occurs if we target 0.12.x or 0.13.0. We can make it work if we accept the older version of OTIO (via pypi). |
0.11.0 was the last pure python release which is probably why it works out of the box. Other than that I'm afraid I can't be of any more help on this issue. |
|
Thanks. Does the fact that 0.11.0 works give any indicators, either of what the problem might be, or of a possible workaround that we could use? For our current needs 0.11.0 is OK, but obviously we wouldn't want to be stuck on the older version forever. |
|
Please excuse me if this is obvious to you :) I believe @meshula is cooking up a cleaner install OTIO and the python bindings that hopefully will help. |
|
That makes sense, thanks very much for the explanation. In the short term, when I get time I will try again to use 0.13.0 via PyInstaller by embedding the two DLL files manually and adapting the import code, to see if I can put them somewhere that a PyInstaller-bundled Python can see them. |
|
Hey @donquibeats, here are a couple useful references for this stuff in pyinstaller:
Forgive me if I'm covering stuff you already know. In my past struggles with PyInstaller, I remember these two bits being key for solving the problem I had. |
|
Have you tried adding your PYTHONPATH to your PATH variable? A similar thread about USD pyd loading was resolved by doing so. |
|
Firstly, really sorry it took me a month to even see these replies. I got pulled onto a different bit of the project and have only just looped round to this question. I remain stuck in the same position as before though, in fact it's a little worse. I am trying to do AAF and XML exports from an Otio object, but in OTIO 0.11.0 it throws errors that don't get thrown by 0.13.0. However, I can't successfully use PyInstaller to compile an .EXE using 0.13.0 (the error I reported in a previous post on this thread), but I can compile fine if I use 0.11.0. So I'm stuck. From the suggestion by @reinecke I've just run We do use .spec files for the PyInstaller compilation, and have added_files parameters where I could manually tell PyInstaller to bring in specific files, but the problem is I don't understand which files are missing and need to be brought in. I'm not sure how to apply the suggestion from @meshula since PyInstaller is meant to handle all the Python bundling and path pointing, and the rest of the Python libraries work fine, this remains a problem specific to OpenTimelineIO version 0.12.0 or above. Any further suggestions would be very welcome. 'opentimelineio', |
|
@meshula Re-opening this discussion. Doing Full message in: |
|
More info: I see in your log that pybind11 is failing to find Python... |
meshula
added
build
|
I also tested with: #886 I get the same error. |
But two lines above it says |
|
yes, that's related. It's Python's libs it didn't find |
yields There must be something different about where the Python libraries are located for Blender. Do you know where they are? |
|
Following the instructions on the blender website, I see the libraries land in c:\blender-git\lib\win64-vc15\python\37\libs and 39\libs to make this work, there probably needs to be a way to pass an argument to pip to tell it where this directory is, and then pass the location from pip to cmake, such that find_package(Python) can subsequently discover the libraries. We are working through enabling CI for Windows builds at the moment. I think it makes sense to first get our CI working, then address specific build issues such as this one as a second step. |
|
Further reference: Installing into Blender is documented in @Tilix4 's project: https://gitlab.com/Tilix4/vse-io |
|
There isn't any way to tell CMake to take the current running python? Because I've launched pip using the Blender's internal python, CMake doesn't handle any reference to it?
Actually I wrote this before testing it, and I raised the issue after my tests. Then this is not a way to make it work... |
|
CMake is properly finding the runtime components. It isn't finding the python.lib for linking with C++. That's the failure. Blender puts the python linking libraries in a different place than the CMake Python find script expects. That's why I'm saying we would need to tell CMake where the Blender build puts the linking library in particular. The rest is fine. |
|
Hey, I am experiencing similar issues to some comments in the thread. Any idea on how can I solve this? |
|
Hi @meshula, Any news about this? |
|
Hi there, thanks for the follow up. In the last month or so, we've put some effort into making the Python set up easier to manage. None of us currently build Blender, so we haven't tested that yet. If you don't specify OTIO_PYTHON_INSTALL_DIR or a CMAKE_INSTALL_PREFIX, the OTIO build system will attempt to set up python according to the cmake standard command: find_package(Python REQUIRED COMPONENTS Interpreter Development)So at this point, my expectation would be that if you don't override those two variables, and you do follow standard Blender/cmake guidance for find_package(Python), things should work. I am being a little vague and optimistic about standard guidance on Blender/cmake, because I hope there is some, but haven't found it yet. If you know where that is, we could do a test and verify if my expectation makes sense. |
|
Thanks for this information. I'm not sure but maybe there is a misunderstanding. I'm not talking about building Blender with OTIO, but installing OTIO into the embedded python of Blender. Something like: Then to try it you may download any windows portable Blender version and try this command. It tested today and it still fails. |
|
To clarify, I have been focused on the cmake side of the problem. Pip uses cmake to install otio. otio uses cmake's find_package to discover details about Python's installation. Therefore to install into Blender, there has to be some way to tell cmake to use Blender's Python, instead of your system blender. According to what I can find on stackexchange, running pip from blender's python should be sufficient to satisfy find_package, as per the command line you proposed. According to Blender's issues pages, there was a bug in some versions of cmake 3.17 that prevented find_package(python) from working, but it eventually got fixed, so presumably cmake 3.18 + should be expected to work, but I imagine that's not the problem you are encountering. So, the possible problems are - 1 - our setup.py does not interpret information received from pip properly |
|
Hi @meshula, is there any update about it? Starting from now I'll spend some time trying to fix this issue and being finally able to pip install OTIO into blender's python. |
|
@meshula, first step following https://opentimelineio.readthedocs.io/en/latest/tutorials/quickstart.html#to-build-otio-for-both-c-and-python-development I get this error on both linux and windows: Traceback (most recent call last):
File "C:\Users\Felix\AppData\Local\Programs\Python\Python37\lib\distutils\fancy_getopt.py", line 233, in getopt
opts, args = getopt.getopt(args, short_opts, self.long_opts)
File "C:\Users\Felix\AppData\Local\Programs\Python\Python37\lib\getopt.py", line 93, in getopt
opts, args = do_longs(opts, args[0][2:], longopts, args[1:])
File "C:\Users\Felix\AppData\Local\Programs\Python\Python37\lib\getopt.py", line 157, in do_longs
has_arg, opt = long_has_args(opt, longopts)
File "C:\Users\Felix\AppData\Local\Programs\Python\Python37\lib\getopt.py", line 174, in long_has_args
raise GetoptError(_('option --%s not recognized') % opt, opt)
getopt.GetoptError: option --cxx-install-root not recognized
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "setup.py", line 36, in <module>
_dist.parse_command_line()
File "C:\Users\Felix\AppData\Roaming\Python\Python37\site-packages\setuptools\dist.py", line 499, in parse_command_line
result = _Distribution.parse_command_line(self)
File "C:\Users\Felix\AppData\Local\Programs\Python\Python37\lib\distutils\dist.py", line 483, in parse_command_line
args = self._parse_command_opts(parser, args)
File "C:\Users\Felix\AppData\Roaming\Python\Python37\site-packages\setuptools\dist.py", line 814, in _parse_command_opts
nargs = _Distribution._parse_command_opts(self, parser, args)
File "C:\Users\Felix\AppData\Local\Programs\Python\Python37\lib\distutils\dist.py", line 578, in _parse_command_opts
(args, opts) = parser.getopt(args[1:])
File "C:\Users\Felix\AppData\Local\Programs\Python\Python37\lib\distutils\fancy_getopt.py", line 235, in getopt
raise DistutilsArgError(msg)
distutils.errors.DistutilsArgError: option --cxx-install-root not recognized |
|
Thanks for the report. Unfortunately, the documentation wasn't updated when we recently updated the pip set up to interact more naturally with cmake. The cxx-install-root option is no longer wired up. Could you try again with |
|
Thanks, it worked. I'm digging into pip issue. |
|
Daily report: The weels building It seems to be an issue in the These directories don't exist under Blenders python: Setting by hand directly in the code Which looks like a similar issue, where the Trying to disable the Hope the information is useful, will continue my cluedo tomorrow. |
|
Great progress, congratulations. As part of your process, I would recommend looking at your CMakeCache.txt. You can discover (1) what are the names of variables actually in use, and (2) how do the things you set relate to where they landed in the CMakeCache. It's a shame the Python ecosystem is so complex, but I guess it's to be expected after decades of experimentation by the community. |
|
Okay, I managed to install into Blender's python by setting Now I have to find a way to correctly target these resources from Blender's python. Do you have any input about pybind which could provide it on its own? |
|
By "correctly target these resources" do you mean setting up Blender to also see OpenTimelineIO? I think you can't point outside of Blender - https://blender.stackexchange.com/questions/172421/pythonpath-ignored - but there are a bunch of solutions - https://stackoverflow.com/questions/11161901/how-to-install-python-modules-in-blender Unfortunately Python on Windows is very prickly compared to other platforms, due to I suppose philosophical differences by the maintainers of different platforms. For example, on Windows, there is a not well documented rule that debug builds of Python need all modules suffixed with a lower case d. There are also difficulties with dynamic library search paths that do not apply on other platforms. You can see similar struggles here PixarAnimationStudios/OpenUSD#1478 and here PixarAnimationStudios/OpenUSD#1420 .... |
|
No, I mean to tell the builder to find the system python instead of the current running python for the library and the include directories (which will force to have python3 installed to pip install into blender's python). The other possibility would be to embed all these dependencies into OTIO package but it feels heavy. |
|
NB: I'm facing the exact same issues with the https://github.com/pybind/cmake_example trying to pip install it into blender's python. I'll try to make it work and then report the modifications into OTIO's. |
Bug Report
-- edit: removed the checkboxes so that GitHub is no longer confused about open tasks --
Describe the bug here. If it is a build issue, please note whether the issue was encountered via
cmake,pip, ortox.To Reproduce
Expected Behavior
The import statement to work correctly.
Logs
The text was updated successfully, but these errors were encountered: