PTH files support #537
Comments
|
Additional information, gif of my repro:
|
|
To verify with the new LS |
|
adding a path file in the site-packages is a workaround. But language server should be able to recognize the package through init.py. Hopefully this is going to be added. |
|
PTH files are for the module to be resolvable regardless of where it is executed. While also not having to manage PYTHONPATH. It is a perfect solution for working on a standalone module nested inside a repo's other stuff that is relied on by other unaffiliated code. |
|
OK, this is about |
MikhailArkhipov
changed the title
|
I started using a single PTH file with multiple paths in about August. It was working in CPython and in VS Code. In about November or December something changed and VS Code stopped resolving my module (although CPython is still able to resolve them). I finally found the workaround as specified in the defect. Use 1 PTH file for each path you want to resolve, then both systems work just fine. Relative paths inside the module stopped resolving in VS Code around the same time and still do not resolve, which is a different problem. https://docs.python.org/3/library/site.html
|
|
@jaredcm - was it LS or Jedi |
|
I've been using LS exclusively from when I started using VS Code. |
|
We don't explicitly support pth files yet, but note that if you have a different directory which contains your code (like This should fix the original issue reported here, given the screenshots. LS version 0.3.1+ should fix the issue with editable installs (which pth files are used in), and is available in the daily channel by setting: "python.analysis.downloadChannel": "daily" |
|
Thanks for the update. The original problem was that PTH files with multiple entries stopped working. I resolved my original problem by putting 1 path per PTH file. LanguageServer seemed okay with it and has been working fine for the past 6 months. I put a comment in #1171 that language server is hanging when pth files are pointing to the same directories as the workspace. I just tried the daily (0.3.12) it still hangs. I'm unsure if this recent breakage is due to any fixes as part of the defect. My PTH configuration works in Python REPL, IPython with all of the Python linting tools, and VSCode with Jedi also works. I switched to Jedi after hitting #1171 a week or so ago. I tried switching back to LanguageServer today because of an unrelated issue in Jedi davidhalter/jedi#1299. /Users/{user}/Documents/Code/stress_test is the root of my workspace. Starting Microsoft Python language server. |
|
So long as you set "python.autoComplete.extraPaths": ["./lib"] |
|
Ahhh, that worked for the majority of paths, thank you very much. Is there a way to surface this error, for better end-user understanding? I wasn't getting any warnings just a hung language server. I see now that the analyzed module is reporting the expected import path. I also see that when using extra paths the PTH paths are removed from system and put into user. I do have one more issue. I have a package in the root of the git repo/VS workspace which cannot be resolved, even if I put '.' or the full path in the setting. For this one I do actually get the unresolved-import-warning and language server does analyze the rest of the packages. [Info - 2:45:04 PM] Interpreter search paths: Could it be the duplicate in the Interpreter search path? |
|
We're working on a potential way to surface the misconfiguration (or some auto config), but it's not a simple task to recognize the difference between "you just need this config" with "you genuinely have a bad import". From the analysis' point of view, all it knows is that it went to find a module and it wasn't there. Anything past that is going to have to be heuristics. The workspace root should be removed from the interpreter path list, yes. I'm going to patch that now. |
|
This actually fixes a couple of other bugs; thank you for showing your logs! 😄 |
|
Ok, thanks. Couldn't it be that anything in the interpreter paths with a common VSCode workspace path be moved to user? |
|
Can you be more specific? Many (perhaps most) users will do something like |
|
Oh, okay. I hadn't thought of the interpreter being in the workspace. Then I guess surfacing an error is the best recourse. What was throwing me was, as mentioned above, I wasn't getting import errors, language server was just hanging, so the extraPaths link didn't appear on the surface to be relevant. I don't know if one is already available but, a screencast on how language server works and how to debug it would be useful as well. Thanks for the help. |
|
The hanging is a bug, so I wouldn't worry about that specifically (I didn't realize the connection until #1175 had a repro), but working on better displaying what we expect to be configured (and eliminating that) is something to work on, yes. |
|
As part of testing your fix I switched my PTH configurator to multi-path in a single file, like I had it when I originally reported this defect. It works now with the extraPaths setting. Everything in VS seems to resolve properly and my intellisense has the correct information for the import including decorated properties that return on-demand instantiated classes. I think we can close this ticket. Thank you once again. |
|
I'm going to leave this open for now (if that's alright); I think we're using this issue to track specifically supporting reading the I'd be really curious as to why multiple pth files would make a difference; but I know that the |
|
Oh, that is fine. I don't know why there was a difference, either. Originally I was using multiple paths per file and it worked in LS. Then LS stopped resolving my packages, thus this defect. The workaround was to use 1 path per file and then LS was able to resolve again. Then we had the hang, and the extraPaths setting fixed that. |
|
I can repro additional problems when a .pth file has multiple lines, running Python 3.7.3 and LS 0.3.6.1 on Windows. From the preceding thread it's not clear whether my scenario is supported, so please tell me if I should open an issue. My variation is that I'm using a venv - my problems only occur when selecting it. One of the symptoms is that if my .pth file has two lines, the first one being blank (empty), or commented with #, LS will not load: upon selecting the venv I see "! Select Python Interpreter" on the status line. Another symptom is that if the .pth contains two valid lines, I get "unresolved reference" on "import sys". I can also repro a "unresolved reference" problem with a single-line .pth file. This scenario is a bit different so I'll hold its details for now. Please advise what you need. |
|
Interpreter selection ("Select Python Interpreter") is a VS Code Python extension issue. The language server only uses the interpreter that the extension says to use. If one isn't selected, then we will likely fail to find any stdlib stuff (including sys), since we won't where the python install/binary is. If you can't select an interpreter, then I'd suggest opening an issue on the extension: https://github.com/Microsoft/vscode-python As for |
|
so a problem that I've run into when using a You can see these extra lines rather simply: If the vscode extension can ignore these lines than |
|
@jakebailey not sure if we can ignore those stdout lines, if we could that would be extremely helpful |
|
This is the language server repo, but the errors above appear to be errors from the Python extension. If you can't activate your environment due to that I would open a bug with the reproducer there: https://github.com/microsoft/vscode-python Though I'm not sure how much can be done if we the Python executable itself isn't doing what it's instructed due to a misconfiguration... |
@jaredcm commented on Mon Dec 17 2018
Environment data
Expected behaviour
Adding custom locations to
.pthfile should be resolved by the Python intellisense.Actual behaviour
I have a custom package that is in a subdirectory of my project. I want Python to resolve it as if it were an installed module.
The project is setup as such.
os_path
-|+- project_dir
--|+- bin
---|+- test.py
--|+- lib
---|+- custom_module
----|+- __init__.py
In the python 3.7 site-packages directory I have a
.pthfile with something like the following contents.{os_dir}/{project_dir}
{os_dir}/{project_dir}/lib
In Python this resolves without issue and so I can do.
If I create a test.py file in the project's bin directory like so:
In VSCode I get unresolved import:
{
"resource": "/{os_path}/{project_dir}/bin/test.py",
"owner": "generated_diagnostic_collection_name#0",
"code": "unresolved-import",
"severity": 4,
"message": "unresolved import 'custom_module'",
"source": "Python",
"startLineNumber": 2,
"startColumn": 8,
"endLineNumber": 2,
"endColumn": 21
}
If I switch the import to lib.{custom_module} then the module resolution works. I'm guessing this is because VSCode adds the workspace to the PYTHONPATH.
Steps to reproduce:
.pthfile and add the full OS path to the lib directory.Logs
Output for
Pythonin theOutputpanel (View→Output, change the drop-down the upper-right of theOutputpanel toPython)Log (Window)
Output from
Consoleunder theDeveloper Toolspanel (toggle Developer Tools on underHelp)@jaredcm commented on Mon Dec 17 2018
I tried to create a brand new project and recreate this. The custom module is resolved without issue. When I try and import modules from my initial project I still cannot resolve them. In both projects I get an error message in the workload console like:
@jaredcm commented on Mon Dec 17 2018
Found the issue. Looks to be a corrupted .mypy_cache. I removed the cache directory from my initial project and the imports started working in both projects.
@jaredcm commented on Mon Dec 17 2018
Looks like I was wrong. I restarted my computer and now my custom module cannot be resolved again. I remember I had been messing about with my pth file. Normally I have 4 paths in it which are all loaded into Python without issue. While filling out this bug report, I had removed all but 1, which was when VS Code started working. I had to add the other 3 paths back again later for my scripts to function properly. Which is why it stopped working and I didn't notice until VSCode was restarted.
project.pth =
os_path/project
os_path/project/lib
os_path/project/tests
os_path/project/unit_tests
I've done some testing:
It looks like VSCode gets confused when pth files have more than one entry.
@jaredcm commented on Mon Dec 17 2018
New issue:
In custom_module I have many sub_modules and they have other sub_modules. Within each of the sub_modules I use relative imports to load their child classes. The sub_module directory has a __init__,py with the import statement of "from .rest import Rest". This should resolve to a child directory named 'rest' that has a __init__.py file where the Rest class is defined. The full resolution from the root module to the imported class is 'custom_module.sub_module.rest.Rest'
The error message is:
{
"resource": "~/project/lib/custom_module/sub_module/__init__.py",
"owner": "generated_diagnostic_collection_name#0",
"code": "unresolved-import",
"severity": 4,
"message": "unresolved import 'lib.custom_modulerest'",
"source": "Python",
"startLineNumber": 6,
"startColumn": 6,
"endLineNumber": 6,
"endColumn": 11
}
Also notice that the message is not showing the correct module import path. The lib directory is not a module as it does not have a __init__.py file. If I change the path of the relative import to a full module import of 'from custom_module.sub_module.rest import Rest', then the error goes away. The path in the message is also missing the current module. It's almost like it the path it is looking for is shifted too far up the tree. Instead of looking in custom_module/sub_module/rest, I think it is looking for lib/custom_module/rest which doesn't exist.
Developer Console output
@d3r3kk commented on Thu Jan 17 2019
Thanks for bringing this up @jaredcm, an interesting problem!
I didn't know about
.pthfiles until this issue popped up so thank you for that.I can repro this issue with the latest stable VS Code (1.30.2) + latest extension (2018.12.1), using the Jedi language server (
"python.jediEnabled": true").Workaround: The Microsoft Python Language Server (as of version 0.1.72.0, or greater)
seems to handle this case(see below). Have you attempted to use this instead of Jedi? See this issue for details on how to enable/update the Microsoft Python Language Server.Adding more than one entry.
Actually, on second look I have found that this is not handled properly by the Microsoft Python Language Server as I'd originally stated (at least version 0.1.72.0 doesn't) as I'd left out the multiple lines in the
.pthpart after I got things up and running.Indeed, if you add a
.pthfile to yoursite-packageswith > 1 path, the issue does repro.Here's the sample workspace:
3727_module_resolution.zip
Here's the sample
.pthfile contents that I placed in thesite-packages. I created a.venvin my workspace and copied this file to${workspaceFolder}/.venv/Lib/site-packages/test_3727.pth.The text was updated successfully, but these errors were encountered: