Skip to content

Commit

Permalink
Fix resolution of ParamSpec in circular dependencies
Browse files Browse the repository at this point in the history
What's happening here?
1. We load module `bar`, by requesting it from the loader.
1. Before resolving types, we need to load its dependencies. So we load module `foo`.
1. We'd like to load `foo`'s dependencies, but that's `bar` and that's a circle, so we start resolving types in `foo`.
1. When resolving external types we create alias an `foo.bar`, copying the unresolved signature from `bar.bar`.
1. When resolving local types, `_P` is not resolvable within `foo`.
1. We run `visitors.AdjustTypeParameters`, which seems to require local type resolution, and runs afterwards. It declares `_P` in `foo`. Now `_P.args` would be resolvable but we already finished resolving local types and we don't do it again.
1. We resolve types in `bar`, validate `bar`'s AST and return.
1. We load module `foo`, by requesting it from the loader.
1. It's cached. We just need to validate it. We realize that types aren't resolved, yet, e.g. `_P.args`. We used to do only external type resolution again, which did not resolve ParamSpec, and we eventually bailed. This change also adds local type resolution again, which does resolve ParamSpec.

PiperOrigin-RevId: 684071752
  • Loading branch information
frigus02 authored and copybara-github committed Oct 10, 2024
1 parent 65ad8fb commit 56b0e73
Show file tree
Hide file tree
Showing 2 changed files with 42 additions and 0 deletions.
5 changes: 5 additions & 0 deletions pytype/load_pytd.py
Original file line number Diff line number Diff line change
Expand Up @@ -737,6 +737,11 @@ def finish_and_verify_ast(self, mod_ast):
self._modules[k].ast
)
mod_ast = self._resolve_external_types(mod_ast)
# Circular imports can leave type params (e.g. ParamSpecArgs)
# unresolved. External type parameters are added to the AST in
# visitors.AdjustTypeParameters, after resolving local types. But those
# are needed to resolve e.g. `_P.args` references.
mod_ast = self._resolver.resolve_local_types(mod_ast)
self._resolver.verify(mod_ast)
return mod_ast

Expand Down
37 changes: 37 additions & 0 deletions pytype/load_pytd_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,43 @@ class Popen: ...
)
)

def test_circular_dependency_with_type_param(self):
with test_utils.Tempdir() as d:
d.create_file(
"bar.pyi",
"""
from typing import Callable, ParamSpec
from foo import Foo
_P = ParamSpec("_P")
class Bar:
foo: Foo | None
def bar(obj: Callable[_P, None], /, *args: _P.args, **kwargs: _P.kwargs) -> Bar: ...
""",
)
d.create_file(
"foo.pyi",
"""
from bar import bar as _bar
class Foo: ...
bar = _bar
""",
)
loader = load_pytd.Loader(
config.Options.create(
module_name="base",
python_version=self.python_version,
pythonpath=d.path,
)
)
bar = loader.import_name("bar")
foo = loader.import_name("foo")
self.assertTrue(bar.Lookup("bar.bar"))
self.assertTrue(foo.Lookup("foo.bar"))

def test_cache(self):
with test_utils.Tempdir() as d:
d.create_file("foo.pyi", "def get_bar() -> bar.Bar: ...")
Expand Down

0 comments on commit 56b0e73

Please sign in to comment.