Pylance large project issues

[rkrishnasanka]

Environment data

• Language Server version: Pylance language server 2020.11.0 (pyright 9d8e431b)
• OS and version: VSCode Remote Server (Ubuntu 18.04) / Editor (MacOS)
• Python version (& distribution if applicable, e.g. Anaconda): pyenv python 3.8.0

Expected behaviour

Pylance was working great at the beginning of my project, syntax highlight, pyright annotations, etc. However, as the project grew in size it's been struggling to keep up. It seems like even with basic type checking enabled, it's completely choking now.

Actual behaviour

Logs

XXX

[judej]

@rkrishnasanka, thanks for the report. Could you please add the tracelogs?

https://github.com/microsoft/pylance-release/blob/master/TROUBLESHOOTING.md#filing-an-issue

thanks

[jakebailey]

It's been over a month since we asked for logs; if you can provide them at the latest version we can reopen this to take a look, but without them there isn't much we can do to pin down the performance issues.

[rkrishnasanka]

Sorry about the delay. I had to work on another project for sometime and I couldn't exactly replicate the same error but here are the traces for another example where vscode just started slogging to keep up.

pylance-traces.txt

[jakebailey]

What is distBlockListener.py? That file consistently takes 20 seconds to check, just by itself. Is there anything abnormal about that file? Is it large? What does it import?

[jakebailey]

Better still if the source is available.

[rkrishnasanka]

Here's the project that I was using.
https://github.com/CIDARLAB/pyLFR/tree/dev

Here's the distBlockListener.py https://github.com/CIDARLAB/pyLFR/blob/dev/lfr/distBlockListener.py

The base-base-base class for this is auto-generated using ANTLR (https://www.antlr.org/)

[erictraut]

Thanks @rkrishnasanka, that's really helpful.

After a cursory investigation, I've discovered so far that the type analyzer is struggling with some aspect of the BitVector.py module (which is used in distBlockListener.py). We'll continue to investigate further.

[rkrishnasanka]

Thanks @erictraut ! That's good to know.

[erictraut]

I've found the root cause of the performance issue. The file in question (distBlockListener.py) now analyzes within about 150ms, whereas it used to take more than 20000ms.

This fix will be in the next release of Pylance.

[rkrishnasanka]

@erictraut I'm curious, what was the underlying cause?

[erictraut]

When pyright attempts to analyze the type of a node in the parse tree, it sometimes needs to speculatively evaluate other nodes. Once those nodes' types are known definitively, they are cached, but during "speculative evaluation", they are not cached because the resulting type might not be correct. The code in the distBlockListener.py contained some nested function calls that represent some worst-case combinatorics of this speculative execution.

Here is a stripped-down example:

list(
    map(
        int,
        list(
            "".join(
                map(lambda x: x, list(map(chr, list(b""))))
            )
        ),
    )
)

The list constructor has two overloads. The map function has 6 overloads. The join function has 2 overloads. And the lambda requires a minimum of 3 passes to resolve. Each of those represents a speculative evaluation, but because they were nested, the types of the entire nested call chain needed to be resolved through brute force since none of the intermediate results were cached. That represents 2 x 6 x 2 x 2 x 3 x 6 x 2 x 6 x 2 = ~20736 type evaluation operations. By adding an "speculative execution cache" that cached partially-evaluated types and re-evaluated them only in certain circumstances, I was able to make the cost mostly linear rather than exponential, resulting in a total of ~100 type evaluation operations.

[rkrishnasanka]

Ah, That makes sense, I was looking at some of the issues linked to this and your comments on pyright but I wasn't sure if I followed it correctly. Thanks for the explanation!

[jakebailey]

This issue has been fixed in version 2021.1.0, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/main/CHANGELOG.md#202110-6-january-2021