Pyright cannot resolve numpy functions

[robertocarta]

Describe the bug
I'm still getting the problem described in #295 with pyright 1.1.42. Not sure if I was supposed to open a new issue, feel free to close it if you want to reopen the old one

Screenshots or Code

VS Code extension or command-line
Running pyright in coc-pyright with neovim in MacOS

[erictraut]

Are you using a type stub for numpy? If so, this is a bug in your type stub.

If you don't have a type stub and have enabled Pyright's "Use Library Code For Types" setting, then this is expected. There are limits to what a type checker can extract from the library implementation itself. In the case of numpy, some functions are implemented in native libraries, so there's no available type information.

For more information about type stubs and why they are so important for type checking, please refer to this documentation.

[oblitum]

Hi @erictraut. I tried pyright through coc-pyright and I also found issues with numpy functions, it's my first attempt moving to it from MPLS. Pyright could complete lots of functions, but some in specific it was unable to, for example, the numpy.abs function. I was able to get it suggested if I didn't generate type stubs, but it didn't come with any signatureHelp, just the abs string was available at numpy.. I then generated the type stubs, which worked for what seems lots of symbols, but then, for some reason, abs stopped from being even available, after generated stubs.

I also noticed that after stub generation, go-to-definition for most functions now offer two places to go, the original module location, and the ./typings location. Regarding completion, now the ./typings function prototypes seem to get priority and are being offered for signatureHelp, but these generated stubs often contain less information than what's present in the original lib, when it's available, for example I get lots of ... in function arguments that comes from generated stubs, while those same arguments in the original prototype actually contains information like =None, etc.

[erictraut]

The behaviors you describe above are all expected. Once a type stub is present, Pyright assumes that it accurately and completely describes the interface contract for the module. If classes, methods or functions are missing from the type stub, the type checker will have no idea that they are missing.

The "generate type stub" feature provides you with a first draft. The stubs it generates are almost always incomplete. Creating a complete, high-quality stub for a library as large and complex as numpy is a big undertaking — something that I wouldn't recommend you do by yourself. It's something the numpy community should create and maintain.

[oblitum]

@erictraut I just at least supposed it could improve at stub generation, there's some cases where it strips information from prototypes and replace it with ... where it would be much more useful if, say, the default argument value was left there as-is instead. Also (I'm not sure whether behavior is the same in other clients) but by go-to-definition I really most certainly will not be caring about the stub definition, so I think there could be at least an option to ignore ./typings for go-to-def, with it being ON by default.

[jakebailey]

The PEPs say that stubs shouldn't contain default values. Stubs can be separate from the code they describe, and if they contained the default values, they they may be misleading if the library they describe changes the default value. E.g., if I read the signature and it says a=1, but it turns out the stub hasn't been updated, then my code is subtly wrong. (IMO this is one reason annotations in the real code are nicer.)

I can see value in a toggle to say "don't jump to stubs" as we have some basic support for finding the corresponding "real" code, but I doubt we'd want to make it the default option in either case. See also #795, where we find that we don't have a way to tell the client "choose this one first".

[oblitum]

Thanks for the information @jakebailey.

[oblitum]

It's something the numpy community should create and maintain

Just checked that numpy is already typed on master and has stub files, it just needs a new release :)

[jakebailey]

I'm actually curious if they are distributing those or not in their packages... As far as I can tell they were committed before their previous release, but Pylance users are still reporting that some members are missing (which should have been covered by the stubs).

[oblitum]

It was indeed merged before the last release, but the last release commit isn't including any of the merged .pyi files in the source tree, so I concluded that even though it was merged to master before, it is not present in the last release.

[jakebailey]

That's good news, at least. I'll be patiently waiting for it... 😄

[oblitum]

For your info, I just tested numpy from master and abs is suggested! But sadly it's still coming without signatureHelp, but interestingly, the alternative absolute was also suggested together, and this one contains the full information.

[oblitum]

Another alternative as suggested here seems even better at the moment. The stubs are quite less noisy and abs is readily available, and it doesn't require numpy from master :)

[jakebailey]

I find that very surprising, because that stub project is focusing on a lot more than capturing just what numpy does now, and is interested in doing shapes and metadata, things that aren't supported besides via their mypy plugin. My impression was that they weren't going to be usable.

[oblitum]

I was afraid of that but on a first impression, the experience is looking a bit better. Saying that because the signatureHelp from numpy master (besides missing that abs, I guess it'll get fixed eventually) is very, hmmm, technical, meaning, the type info for signatures is so detailed that is can be overwhelming, there's much Bool|Int|Array|etc for each argument which creates a rather long signature, it may be correct/exact, but kinda noisy.

[jakebailey]

My first impression might have been wrong, looking more closely. Things like ndarray are generic, but only over their dtypes, which are bounded. So these might be pretty usable, but potentially incomplete (by their admission).