Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid unconditional imports of pandas,pyarrow from rust #2727

Closed
timkpaine opened this issue Aug 30, 2024 · 4 comments
Closed

Avoid unconditional imports of pandas,pyarrow from rust #2727

timkpaine opened this issue Aug 30, 2024 · 4 comments
Labels
enhancement Feature requests or improvements

Comments

@timkpaine
Copy link
Member

Right now we do some unconditional imports of pandas/pyarrow from rust:

let pyarrow = PyModule::import_bound(py, "pyarrow")?;

let pandas = PyModule::import_bound(py, "pandas")?;

Ideally, we wouldn't incur import costs just for this check, instead we can check if e.g. pandas in sys.modules. If it is, then the import is already paid for, if it isn't, then what we are receiving can't be a pandas Dataframe

@timkpaine timkpaine added the enhancement Feature requests or improvements label Aug 30, 2024
@timkpaine
Copy link
Member Author

xref: #2723

@texodus
Copy link
Member

texodus commented Sep 9, 2024

... if it isn't, then what we are receiving can't be a pandas Dataframe

What if I do this:

import sys
import pandas
import perspective

df = pandas.DataFrame({"a": [1,2,3]})
del sys.modules["pandas"]

perspective.table(df)

@timkpaine
Copy link
Member Author

Deleting from sys modules is explicitly documented in the standard library docs as having the potential for causing problems, so no need to handle explicitly (I don't know why you would delete from sys modules unless you're trying to cause problems, this example is a degenerate case pretty clearly).

@texodus
Copy link
Member

texodus commented Sep 23, 2024

Fixed in #2754, released in v3.1.0

@texodus texodus closed this as completed Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Feature requests or improvements
Projects
None yet
Development

No branches or pull requests

2 participants