Jedi doesn't complete inside modules with a name equal to their containing package.

[Alexandre-Silva]

Say we have a project with the following structure:

$tree
.
├── a.py
└── b
    ├── __init__.py
    ├── b.py
    └── c.py

If we try to get completions from inside file b/b.py, jedi fails to do so.
For example, in the following test script we see in which situations we can get completions.

import sys
import jedi

sys.path.append("/tmp/test")

source = """
from a import A
from b.b import B
from b.c import C
A().
B().
C().
"""

fmt = "{:15} {:20} {:15}"
print(fmt.format('file', 'line to complete', 'len(jedi.Script(...).completions())'))
print()

for p in ('test.py', 'b/b.py', 'b/test.py'):
    for line_num in (5, 6, 7):
        line = source.splitlines()[line_num - 1]
        script = jedi.Script(source, line_num, len(line), '/tmp/test/' + p)
        print(fmt.format(p, line, len(script.completions())))
    print()

Which outputs:

file            line to complete     len(jedi.Script(...).completions())

test.py         A().                              22
test.py         B().                              22
test.py         C().                              22

b/b.py          A().                              22
b/b.py          B().                               0
b/b.py          C().                               0

b/test.py       A().                              22
b/test.py       B().                               0
b/test.py       C().                              22

As we can see, we can't get completions in module 'b/b.py' for other modules in package 'b'.
Additionaly, modules from package 'b' such as 'b/c.py' cant get completions for 'b/b.py'.

Python 3.5.2
Linux 4.7.2-1-ARCH GNU/Linux
jedi 0.9.0 (from emacs anaconda-mode v0.1.5)

[joshpeng]

Similar experience here too:
Python 2.7.12
Win 7 x64
jedi 0.9.0 (from Visual Studio Code's Python extension)

$tree
.
├── a
    ├──__init__.py
    ├──a.py
    ├──c.py
└── b
    ├── __init__.py
    ├── b.py

When in b/b.py, trying to import a/a.py via this from a.a import A yields no autocomplete.
However importing a/c.py with from a.c import C will give you autocompletes.

What is really odd about the first scenario of from a.a import A is that it actually works for a few moments. After trying to pull autocomplete a few times though then it fails.

[davidhalter]

This is kind of a known issue. It's hard for Jedi to know where to start searching for those name. I guess we could search both directories for results (or clearly start defining a project path).

It's possible to do this correctly by using something like Script(sys_path=['/path/to/a']). However this is a bit annoying to define.

[joshpeng]

from a.c import C works though which would suggest Jedi is somehow already searching in directory a. I'm sure there is good reason from a.a import A works differently, but just in layman's terms, it seems that the two should behave the same. Thanks.

[davidhalter]

Ok. I understand and think you're probably right.

[sliwinski-milosz]

Would be great if we will be able to define project path for jedi as because of above import issues Visual Studio Code and Atom "Go to definition" doesn't work well in mentioned cases.

Any chance for implementing project path parameter for jedi?

[davidhalter]

You cannot specify a project path, but you can specify the sys.path, which is probably good enough for you.

[sliwinski-milosz]

Hmm could you please help me to configure it?

This is my problem:

$tree
.
└── a
    ├── __init__.py
└── tests
    └── a
        ├── __init__.py
        ├── some_file.py
    ├── __init__.py

So whenever in tests/a/some_file.py I do: import a I would like to be able to go to the definition of a which is in project directory. Currently it goes to tests/a/__init__.py.

I made some changes in Jedi code (diff):

1. I removed path to tests from sys_path.
2. Added some logging.

After that I checked sys_path and this is what I get (everything is in virtual env):

Jedi just started: 
source: None
line: 1
column: 8
path: /Users/user/workspace/python/jedi_issue/tests/a/__init__.py
source_path: None
source_encoding: None
sys_path: None

Jedi at the end of init: 
source: b'import a\n'
line: 1
column: 8
path: /Users/user/workspace/python/jedi_issue/tests/a/__init__.py
source_path: None
source_encoding: None
sys_path: [
    "/Users/user/.virtualenvs/jedi_issue/lib/python3.5/site-packages",
    "/Users/user/.virtualenvs/jedi_issue/bin",
    "/Users/user/.virtualenvs/jedi_issue/bin/..",
    "/Users/user/workspace/python/jedi_issue",
    "/Users/user/.vscode/extensions/donjayamanne.python-0.5.9/pythonFiles",
    "/Users/user/.virtualenvs/jedi_issue/lib/python35.zip",
    "/Users/user/.virtualenvs/jedi_issue/lib/python3.5",
    "/Users/user/.virtualenvs/jedi_issue/lib/python3.5/plat-darwin",
    "/Users/user/.virtualenvs/jedi_issue/lib/python3.5/lib-dynload",
    "/usr/local/Cellar/python3/3.5.2_1/Frameworks/Python.framework/Versions/3.5/lib/python3.5",
    "/usr/local/Cellar/python3/3.5.2_1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/plat-darwin",
    "/Users/user/.virtualenvs/jedi_issue/lib/python3.5/site-packages"
]
parser.module: <Module: /Users/user/workspace/python/jedi_issue/tests/a/__init__.py@1-2>
goto_definitions: helpers.sorted_definitions(set(defs)): [<Definition module a>]

Based on above sys_path it should be able to open a which is in project directory. But somehow it opens tests/a/__init__.py. When I delete tests/a/__init__.py then it works fine.

Could you please tell me which path should I add to sys_path or how to solve my issue?

[davidhalter]

Can you please just send me a simple jedi.Script call with results and what should not be the case? This is way too much information,

[sliwinski-milosz]

Above message is not that long if you will skip logs from Jedi.

I can reproduce issue in Visual Studio Code but I don't know how it calls jedi that is why I started to log variables inside jedi script.

Could you please try to understand what is going on based on my previous message? Otherwise I will have to go deeper into understanding how Visual Studio Code + Jedi work.

Briefly issue is about not being able to go to the definition of module based on project path - it goes to the current open module instead if both module names are the same.

[davidhalter]

Please dig deeper, I don't have the time to look into the issues too much. There's 90 other open issues.

[davidhalter]

For the original issue: Please recheck. I think I've fixed it. The priorities of paths are now a bit more clear and I don't think it's still an issue. At least I've been working on very similar cases.

Reopening if it's still an issue.

[Alexandre-Silva]

@davidhalter , I've re-executed the initial test script and I'm getting the following for jedi-0.11.1:

file            line to complete     len(jedi.Script(...).completions())

test.py         A().                              23
test.py         B().                              23
test.py         C().                              23

b/b.py          A().                              23
b/b.py          B().                               0
b/b.py          C().                              23

b/test.py       A().                              23
b/test.py       B().                               0
b/test.py       C().                              23

Notice how it works as expected, with the exception of b/test.py. Meaning, completions for files inside the same module with repeated names still fail. It is, however, an improvement over what happened in jedi-0.9.0 (when this bug reported was first opened).

However, re-executing in jedi-0.12.0 (latest commit from master) results in the following:

file            line to complete     len(jedi.Script(...).completions())

test.py         A().                              23
test.py         B().                              23
test.py         C().                              23

b/b.py          A().                               0
b/b.py          B().                               0
b/b.py          C().                               0

b/test.py       A().                               0
b/test.py       B().                               0
b/test.py       C().                               0

So, it appears some wonky stuff is still happening.

[davidhalter]

Thanks for checking it again. I think it is already a lot better even if your test doesn't reflect it.

However there are still two issues:

1. You are using b/b.py as a path. This gets cached with a different content than the actual content. that doesn't work. I have therefore renamed it to b/x.py. See results below.
2. Currently you need to have a setup.py / .git / .hg file in your base directory to "mark it as a base directory. Otherwise the directory of the file is chosen as a directory. This will change as soon as the project API is released (it already exists), where users will be able to specify a base path. However that will happen sometime after RFC: Environments (Virtualenv Improvements) #1053. I have an idea how to make it better until that's possible.

$ touch setup.py
$ python3 run.py
file            line to complete     len(jedi.Script(...).completions())

test.py         A().                              22
test.py         B().                              22
test.py         C().                              22

b/x.py          A().                              22
b/x.py          B().                              22
b/x.py          C().                              22

b/test.py       A().                              22
b/test.py       B().                              22
b/test.py       C().                              22

[davidhalter]

I now changed some things about detecting __init__.pys and now it should work for you :)

[Alexandre-Silva]

Confirmed, it works as expected. @davidhalter thanks.