-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
more upstream-dev cftime failures #3751
Comments
Part of the hazard of using a It looks like pandas-dev/pandas#31511 was the cause of the issue:
A way to fix this upstream would be to make sure that I think it's possible we could work around this in xarray. It comes down to properly recognizing what to do when you subtract a generic |
Thanks for narrowing it down @spencerkclark . Let's see what @TomAugspurger thinks about an xarray workaround vs a pandas fix. |
FWIW, I think @jbrockmendel is still progressing on an "extension index" interface where you could have a custom dtype / Index subclass that would be properly supported. Long-term, that's the best solution. Short-term, I'm less sure what's best. |
any idea what |
Nice -- I look forward to being able to try that out!
@jbrockmendel agreed that it's unclear -- we probably should have written the code for that method in a clearer way. I think it's mainly used for subtracting a single
|
when the tests fail,
I think the issue here is that |
Yes, I noted that in my original post (sorry if that wasn't clear). |
no, that's my bad, it is pretty clear but I seem to have skipped over it |
Another kind of failure came up in the context of indexing a Series with a Example failure
Admittedly I think most people probably use a CFTimeIndex within xarray data structures, but it would be nice to maintain some ability to use it in pandas data structures too. This issue stems from the changes made in pandas-dev/pandas#31399. I think the problem is that
Could there be a simple upstream fix for this? |
Yah, pandas recently added a check for |
@jbrockmendel @TomAugspurger it turns out that fixing indexing with the "nearest" method without overriding private methods of |
Definitely. Is it still the case that the identified problems would all be solved by having |
Thanks @jbrockmendel -- I think there are two separate issues:
|
@spencerkclark can you open an issue on the pandas tracker about this and ping me there; I dont want this to fall off my radar |
Thanks @jbrockmendel -- I'll try to do that this weekend. |
Thanks so much @jbrockmendel for looking into the Once pandas-dev/pandas#32684 is merged, we should be able to un-xfail the Series |
We're making progress on pandas-dev/pandas#32684, I'd also like to see if we can do an in-pandas fix to avoid the need to override more things here (i.e. #3764). Seems like that will lead to fewer headaches long-term. Since pandas-dev/pandas#31511, the method has been updated to remove an np.asarray call, and I'm curious if that is enough to make #3764 unnecessary. The only remaining difference AFAICT is |
Thanks @jbrockmendel; it's great to see that pandas-dev/pandas#32684 was merged. Regarding #3764, I gave things a try with pandas master and removing our overrides to Example failure
In my testing, I can only get things to work if the argument to |
Can you open a pandas PR for the cftime-nearest-fix branch, and add a test in tests/test_downstream.py for the problematic behavior? That branch won't be merged as-is, but it will be easier to discuss options in-line. |
Thanks @jbrockmendel. I didn't realize you had a few downstream tests; that's great. See pandas-dev/pandas#32905. |
…ydata/xarray#3751) (#34025) Co-authored-by: Spencer Clark <[email protected]>
MeeseeksMachine (10): Backport PR #32833: DOC: FutureWarning in Sphinx build when calling read_parquet (#32847) Backport PR #32827: DOC: Fixed contributors for bugfix releases (#32928) Backport PR #33566: CI: Fix jedi deprecation warning for 0.17.0 on IPython (#33972) Backport PR #33080: CI troubleshoot azure (#33973) Backport PR #33309: DOC: include Offset.__call__ to autosummary to fix sphinx warning (#33975) Backport PR #31146: Remove possibly illegal test data (#33976) Backport PR #33241: tostring->tobytes (#33993) Backport PR #33954: DOC: update conf.py to use new numpy doc url (#33996) Backport PR #33102: PERF: fix performance regression in memory_usage(deep=True) for object dtype (#33157) Backport PR #33968: CI: Bump numpydev URL (#34250) MomIsBestFriend (1): Backport PR #32840 on branch 1.0.x (DOC: use new pydata-sphinx-theme name) (#32848) Pandas Development Team (1): RLS: 1.0.4 Simon Hawkins (24): CI: test_unsupported_other fails on pyarrow 0.17 (#33990) CI/DEP: Use numba.extending.is_jitted for numba version > 0.49.0 (#33994) DOC: start 1.0.4 (#33970) BLD: recursive inclusion of DLLs in package data (#33246) (#33995) Backport PR #33462 on branch 1.0.x (BUG: None converted to NaN after groupby first and last) (#33998) Backport PR #33761 on branch 1.0.x (REGR: fix DataFrame reduction with EA columns and numeric_only=True) (#34000) Backport PR #33292 on branch 1.0.x (REGR: Fix bug when replacing categorical value with self) (#34004) Backport PR #32870 on branch 1.0.x (DOC: Remove latest whatsnew from header) (#34003) Backport PR #33629 on branch 1.0.x (BUG: Fix Categorical use_inf_as_na bug) (#34001) release note for #33102 (#34005) Backport PR #33513 on branch 1.0.x (BUG: Fix Categorical.min / max bug) (#34022) Backport PR #32905 on branch 1.0.x (Fix to _get_nearest_indexer for pydata/xarray#3751) (#34025) Backport PR #33693 on branch 1.0.x (BUG: Fix memory issues in rolling.min/max) (#34027) Backport PR #33089 on branch 1.0.x (BUG: Don't cast nullable Boolean to float in groupby) (#34023) Backport PR #33645, #33632 and #34087 on branch 1.0.x (#34173) Backport PR #34048 on branch 1.0.x (Bug in DataFrame.replace casts columns to ``object`` dtype if items in ``to_replace`` not in values) (#34115) REGR: exceptions not caught in _call_map_locations (#34113) Backport PR #33983 on branch 1.0.x (BUG: Use args and kwargs in Rolling.apply) (#34190) Backport PR #34049 on branch 1.0.x (Bug in Series.groupby would raise ValueError when grouping by PeriodIndex level) (#34247) Backport PR #34053 on branch 1.0.x (more informative error message with np.min or np.max on unordered Categorical) (#34246) Backport PR #32479 on branch 1.0.x (BUG: Fix issue with datetime[ns, tz] input in Block.setitem) (#34369) Backport PR #33644 on branch 1.0.x (BUG: Groupby quantiles incorrect bins) (#34382) DOC: intersphinx inventory link for statsmodels (#34424) DOC: 1.0.4 release notes and date (#34425)
https://dev.azure.com/xarray/xarray/_build/results?buildId=2116&view=logs&jobId=2280efed-fda1-53bd-9213-1fa8ec9b4fa8&j=2280efed-fda1-53bd-9213-1fa8ec9b4fa8&t=175181ee-1928-5a6b-f537-168f7a8b7c2d
46 failed tests but they all seem to result from the same TypeError
The text was updated successfully, but these errors were encountered: