Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to accept file list from the standard input #5342

Closed
vors opened this issue Jun 19, 2023 · 9 comments
Closed

Option to accept file list from the standard input #5342

vors opened this issue Jun 19, 2023 · 9 comments
Labels
addressed in next version Issue is fixed and will appear in next published version enhancement request New feature or request

Comments

@vors
Copy link

vors commented Jun 19, 2023

Is your feature request related to a problem? Please describe.

We have a relatively large monorepo. I'd like to implement a functionality that quickly runs type checking only on MODIFIED (compare to the merge-base with mainline branch) files. This is exactly the same approach that we take with linter. It can be easily done now, however when we pass the list of files to pyright through the command line we quickly run into the command line buffer limit of files. In order to mitigate this problem the proposal is to allow passing the list of files though the stdin.

Describe the solution you'd like
One common way to do it is to allow pyright - to mean "read filepaths from command line".

Additional context
N/A

@vors vors added the enhancement request New feature or request label Jun 19, 2023
@erictraut
Copy link
Collaborator

I recommend against type checking only modified source files. Modifications to one file can affect type checking of any files that depend upon it (i.e. import directly or transitively from it). Pyright is designed to be fast enough that you can run it on a large code base without needing to resort to workarounds like this. Out of curiosity, how big is your source base, and how long does it take to run pyright fully?

erictraut pushed a commit that referenced this issue Jun 20, 2023
…ames to be passed via stdin if `-` option is used in the command line. This addresses #5342.
@erictraut
Copy link
Collaborator

I've added the requested enhancement because I think it may be useful in some cases, but as I mentioned above, I recommend against using the approach you outlined above.

@erictraut erictraut added the addressed in next version Issue is fixed and will appear in next published version label Jun 20, 2023
@vors
Copy link
Author

vors commented Jun 20, 2023

That's a fair point, our codebase is large enough that

  1. It's not practical to create virtual environment for the full codebase, so you will not get access to the full 3rd party libs you need for the full type check.
  2. It takes on the order of 1000s to run pyright on it as a whole.

I think we can catch 95% of errors in this fast-path. And the rest 5% will catch CI tests.

jbradaric added a commit to jbradaric/pyright-inlay-hints that referenced this issue Jun 20, 2023
commit 63f5658
Author: Erik De Bonte <[email protected]>
Date:   Tue Jun 20 10:50:18 2023 -0700

    Support PEP 712's new attribute assignment conversion (microsoft#5343)

commit 9cbe8c0
Author: Eric Traut <[email protected]>
Date:   Tue Jun 20 09:17:40 2023 -0700

    Fixed a bug that caused an incorrect false positive error for an `assert_type` call involving a `Callable[[], X]` type. Pyright was generating a signature with a positional-only separator in this case. This addresses python/typeshed#10325 (comment). (microsoft#5345)

    Co-authored-by: Eric Traut <[email protected]>

commit d12ad6e
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 23:59:08 2023 -0700

    Updated typeshed stubs to the latest version.

commit eff69f5
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 22:35:56 2023 -0700

    Fixed a bug that caused incorrect type inference for parameters in unannotated methods within child classes who derive from a generic parent class. This addresses a bug reported on stack overflow: https://stackoverflow.com/questions/76466874/trouble-specifying-type-parameter-when-inheriting-from-generic-class.

commit 43c5fae
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 20:17:00 2023 -0700

    Finished cleanup of test cases.

commit b0d4080
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 19:15:53 2023 -0700

    Next batch of test case cleanup.

commit efea9b9
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 18:26:49 2023 -0700

    Changed behavior of non-ClassVar variables within a protocol definition. Previously, an error was reported when such variables were accessed from the class (as opposed to an instance of the class). Mypy (which was the reference implementation for PEP 544) does not report an error here. This addresses microsoft/pylance-release#4389.

commit 5568b4d
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 18:00:31 2023 -0700

    Enhanced command-line version of pyright to allow file or directory names to be passed via stdin if `-` option is used in the command line. This addresses microsoft#5342.

commit cd257f7
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 15:38:13 2023 -0700

    Next batch of test case cleanup.

commit 0c5c7b6
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 14:23:50 2023 -0700

    Next batch of text case cleanup.

commit 899799d
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 13:33:41 2023 -0700

    Finished cleaning up "genericTypesX.py" test files.

commit 1111f1b
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 11:49:56 2023 -0700

    Started to untangle the hairball related to the "genericTypesX.py" test cases.

commit 67394ad
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 10:21:36 2023 -0700

    Continued cleanup of test cases.

commit 96e03aa
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 09:39:09 2023 -0700

    Did a cleanup pass for the dataclass test cases. Renamed and reordered test cases for maintainability.

commit b7df200
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 08:20:54 2023 -0700

    Fixed bug that resulted in a false positive error when defining a new type alias using the `TypeAliasType` constructor that defines no new type parameters but references an outer-scoped type parameter in the type alias definition. This addresses microsoft#5341.

commit d816d00
Author: Eric Traut <[email protected]>
Date:   Mon Jun 19 00:00:07 2023 -0700

    Started to do a pass over the test cases to make them more consistent.

commit 23bcbce
Author: Eric Traut <[email protected]>
Date:   Sun Jun 18 17:42:25 2023 -0700

    Fixed bug in code flow engine that led to incorrect type evaluation of a variable in a nested loop. This addresses https://github.com/microsoft/pylance-release/issues/4509.

commit 056415f
Author: Eric Traut <[email protected]>
Date:   Sun Jun 18 16:40:00 2023 -0700

    Fixed a bug in the control flow debugging code that prints the control flow graph. It was not correctly handling one of the node types which led to incomplete graphs.

commit 643bb1d
Author: Eric Traut <[email protected]>
Date:   Sun Jun 18 13:20:49 2023 -0700

    Improved type inference for lambdas in the case where a parameter includes a default value and the expected type doesn't include that parameter. This improvement was suggested in the [mypy issue tracker](python/mypy#15459). (microsoft#5337)

    Co-authored-by: Eric Traut <[email protected]>

commit 8ce23eb
Author: Eric Traut <[email protected]>
Date:   Sun Jun 18 12:55:22 2023 -0700

    Improved `reportUnnecessaryCast` so it works with types other than cl… (microsoft#5336)

    Improved `reportUnnecessaryCast` so it works with types other than class instances. This addresses microsoft#5333.

commit 52c8cac
Author: Eric Traut <[email protected]>
Date:   Sun Jun 18 12:41:24 2023 -0700

    Changed type printer (the component that renders types into text) to use the lowercase `type[x]` instead of `Type[x]`. It has now been four years since PEP 585 deprecated the use of the upper-case version, so most developers should be getting comfortable with the lowercase version at this point.

commit e3080b1
Author: Eric Traut <[email protected]>
Date:   Sun Jun 18 10:15:58 2023 -0700

    Fixed a bug that led to a false positive error under certain circumstances when a literal type argument was used in conjunction with a protocol that used a covariant type parameter and an implementation of that protocol that used an invariant type parameter. This addresses microsoft#5282. (microsoft#5332)

    Co-authored-by: Eric Traut <[email protected]>

commit 2b0f8d2
Author: Eric Traut <[email protected]>
Date:   Sun Jun 18 09:46:18 2023 -0700

    Improved hover text to display the calculated variance for a PEP 695-style class-scoped type variable when the user hovers over the type parameter in the type param list.

commit 02b7769
Author: Eric Traut <[email protected]>
Date:   Sun Jun 18 00:17:53 2023 -0700

    Fixed a false positive error arising from the use of a binary expression for a base class in a class declaration statement. This addresses microsoft#5326. (microsoft#5331)

    Co-authored-by: Eric Traut <[email protected]>

commit 2de35e3
Author: Eric Traut <[email protected]>
Date:   Sun Jun 18 00:00:06 2023 -0700

    Added documentation about higher-order functions.

commit 42a37f4
Author: Eric Traut <[email protected]>
Date:   Sat Jun 17 23:38:50 2023 -0700

    Re-enabled a test case that was previously disabled because it was broken.

commit 7ea11a1
Author: Eric Traut <[email protected]>
Date:   Sat Jun 17 23:17:32 2023 -0700

    Minor code cleanup — rename constant for clarity and refactor validation function. No functional change.

commit 53cb3f9
Author: Eric Traut <[email protected]>
Date:   Sat Jun 17 23:00:49 2023 -0700

    Improved consistency of parameter ordering internally to type evaluator. No functional change.

commit 9bf8231
Author: Eric Traut <[email protected]>
Date:   Sat Jun 17 22:48:18 2023 -0700

    Fixed a bug that led to incorrect type evaluation when passing a generic class (with a constructor that includes class-scoped TypeVars) as an argument for a callable parameter. The class was being specialized prematurely (with type arguments set to `Unknown`) before the constraint solver was able to solve the higher-order function's type variables. This addresses microsoft#5324. (microsoft#5328)

    Co-authored-by: Eric Traut <[email protected]>

commit 47cd514
Author: Eric Traut <[email protected]>
Date:   Sat Jun 17 22:21:52 2023 -0700

    Changed auto-variance algorithm to ignore `__new__` and `__init__` methods for purposes of calculating the variance of a TypeVar. This mirrors the behavior of mypy. (microsoft#5327)

    Co-authored-by: Eric Traut <[email protected]>

commit 3021b9c
Author: Eric Traut <[email protected]>
Date:   Sat Jun 17 21:52:57 2023 -0700

    Minor code cleanup — removed dead code and converted lambda to function. No functional change.

commit 327ce37
Author: Eric Traut <[email protected]>
Date:   Sat Jun 17 20:30:36 2023 -0700

    Added test case for microsoft#5027.

commit 0559382
Author: Eric Traut <[email protected]>
Date:   Sat Jun 17 14:19:12 2023 -0700

    Fixed false negative when a literal and non-literal are assigned to the same TypeVar in an invariant context. This addresses microsoft#5321. (microsoft#5323)

    Co-authored-by: Eric Traut <[email protected]>

commit 2829429
Author: Eric Traut <[email protected]>
Date:   Sat Jun 17 12:33:13 2023 -0700

    Fixed bug that led to an incorrect type evaluation for nested call expressions where an inner call expression used a ParamSpec. This addresses microsoft#5281. (microsoft#5322)

    Co-authored-by: Eric Traut <[email protected]>
@erictraut
Copy link
Collaborator

This is included in pyright 1.1.315, which I just published.

@vors
Copy link
Author

vors commented Jun 21, 2023

💪 thank you!

@vors
Copy link
Author

vors commented Jun 21, 2023

Hm... so I quickly tried it out and I think it still tries to type-check the whole world, even if I run it like this

echo 'foo.py' | pyright -

@erictraut
Copy link
Collaborator

Ah darn. After manually testing the new feature, I added a small change that included a bug. You'll need to wait for the next pyright release for the feature.

@vors
Copy link
Author

vors commented Jun 21, 2023

Of course, no rush! Thank you so much.

@kyle-query
Copy link

Confirmed this is working now. However it would be easier to use if pyright would split on line breaks rather than spaces, because commands like ls or find typically print one file per line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
addressed in next version Issue is fixed and will appear in next published version enhancement request New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants