Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track memory in C-only threads as belonging to Python code launched them #72

Closed
itamarst opened this issue Aug 22, 2020 · 1 comment · Fixed by #76
Closed

Track memory in C-only threads as belonging to Python code launched them #72

itamarst opened this issue Aug 22, 2020 · 1 comment · Fixed by #76
Labels
enhancement New feature or request

Comments

@itamarst
Copy link
Collaborator

BLAS (via NumPy), Blosc (via Zarr), etc. will launch short-run threads. They only run Python code. Right now Fil sets env variables to not launch threads. It would be better to assign their allocations to Python code that launched them—a good heuristic is "thread that is only C code gets assigned to Python code that launched it".

Benefits:

  1. Fil will run code the same as regular code; right now it tries to disable threading for things it knows about.
  2. It will track memory for things Fil doesn't know about, including custom C code.
@itamarst itamarst added the enhancement New feature or request label Aug 22, 2020
@itamarst
Copy link
Collaborator Author

Implementation idea: new threads get launched with default callstack ID of the code that launched them. Python thread initialization resets the callstack.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant