Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider recursively merging sources/requirements for Python #14561

Open
stuhood opened this issue Feb 22, 2022 · 0 comments
Open

Consider recursively merging sources/requirements for Python #14561

stuhood opened this issue Feb 22, 2022 · 0 comments
Labels
backend: Python Python backend-related issues

Comments

@stuhood
Copy link
Member

stuhood commented Feb 22, 2022

Sources, requirements, and dists are currently computed per-root-Python-target (e.g. python_tests). But this does not allow for reuse of any calculations involved in merging those items. For example, the build_local_dists @rule re-requests the transitive targets, computes provided wheels, and then subtracts sources from the transitive source set... and the subtraction and subsetting is not shared across consumers.

If we instead modeled the building of a Python execution context (PEX) as the merging of a data structure containing information about the sources/requirements/dists (similar to the merging of a classpath for compilation, and thus subject to the optimization in #13112), subgraphs of that data structure could be reused across consumers.

Implementing Python-execution-context-creation recursively would be particularly helpful for mypy, which could likely use its own caches if it was executed recursively.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend: Python Python backend-related issues
Projects
None yet
Development

No branches or pull requests

2 participants