Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pyarrow: Check compatibility of pyarrow-backed pandas objects with numeric dtypes #2774

Merged
merged 16 commits into from
Dec 16, 2023
Merged

pyarrow: Check compatibility of pyarrow-backed pandas objects with numeric dtypes #2774

merged 16 commits into from
Dec 16, 2023

Commits on Oct 29, 2023

  1. Ensure that pyarrow backed pandas.Series can be read 

    Install pyarrow as an optional dependency, and check that pandas.Series objects backed by pyarrow dtypes (e.g. 'uint8[pyarrow]') can be read by virtualfile_from_vectors.
    @weiji14
    weiji14 committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    de36c6c View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2023

  1. Merge branch 'main' into pyarrow-compat

    @weiji14
    weiji14 committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    7a43e7b View commit details
    Browse the repository at this point in the history

  2. Fix check on whether pyarrow is installed 

    Remove duck-typing that didn't work, and use a proper try-except `import pyarrow`.
    @weiji14
    weiji14 committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    06739ef View commit details
    Browse the repository at this point in the history

Commits on Dec 3, 2023

  1. Merge branch 'main' into pyarrow-compat

    @weiji14
    weiji14 committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    88ce36d View commit details
    Browse the repository at this point in the history

  2. Ensure that pygmt.info can work with pyarrow int64/float64 dtypes 

    Check that pandas.Series and pandas.DataFrame objects backed by pyarrow dtypes (e.g. 'int64[pyarrow]' and 'float64[pyarrow]') can be read by pygmt.info. Using pandas.util._test_decorators_skip_if_no pytest mark to simplify the pytest parametrize.
    @weiji14
    weiji14 committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    83ed447 View commit details
    Browse the repository at this point in the history

  3. Add xfail test for test_geopandas_plot_int_dtypes casting to pyarrow int 

    Geopandas doesn't support casting to pyarrow dtypes like 'int32[pyarrow]' and 'int64[pyarrow]' yet, but adding an xfail test so that we don't forget to test in the future.
    @weiji14
    weiji14 committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    ad99cab View commit details
    Browse the repository at this point in the history

  4. Clarify reason for test_geopandas_plot_int_dtypes xfail 

    Actually, casting to pyarrow integer dtypes work, but writing to the temporary OGR_GMT file doesn't.
    @weiji14
    weiji14 committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    a5284fc View commit details
    Browse the repository at this point in the history

  5. Add optional pyarrow dependency to ci_test_dev and ci_tests_legacy 

    Ensure that previous and future versions of GMT are compatible with PyArrow too.
    @weiji14
    weiji14 committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    8b28ba4 View commit details
    Browse the repository at this point in the history

  6. Add note about support of PyArrow dtypes to doc/install.rst 

    Mention that PyGMT does have some initial support of Pandas objects backed by PyArrow-dtype arrays, but only uint/int/float dtypes for now.
    @weiji14
    weiji14 committed Dec 3, 2023
    Configuration menu
    Copy the full SHA
    95d2b8a View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2023

  1. Replace pandas.util._test_decorators with inline pytest.mark.skipif mark 

    Implementing the td.skip_if_no pytest mark inline instead of using the private function from pandas.
    @weiji14
    weiji14 committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    5adcba6 View commit details
    Browse the repository at this point in the history

Commits on Dec 16, 2023

  1. Merge branch 'main' into pyarrow-compat

    @weiji14
    weiji14 committed Dec 16, 2023
    Configuration menu
    Copy the full SHA
    3ca0288 View commit details
    Browse the repository at this point in the history

  2. Apply suggestions from code review 

    Co-Authored-By: Dongdong Tian <[email protected]>
    @weiji14 @seisman
    weiji14 and seisman committed Dec 16, 2023
    Configuration menu
    Copy the full SHA
    6d50a23 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into pyarrow-compat

    @weiji14
    weiji14 committed Dec 16, 2023
    Configuration menu
    Copy the full SHA
    1783c7c View commit details
    Browse the repository at this point in the history

  4. Refactor to use pygmt.helpers.testing.skip_if_no

    @weiji14
    weiji14 committed Dec 16, 2023
    Configuration menu
    Copy the full SHA
    62d22cb View commit details
    Browse the repository at this point in the history

  5. Use importlib.util.find_spec instead of try-except block 

    Cleaner way to check if pyarrow is installed or not.
    @weiji14
    weiji14 committed Dec 16, 2023
    Configuration menu
    Copy the full SHA
    a275825 View commit details
    Browse the repository at this point in the history

  6. Update doc/install.rst 

    Co-authored-by: Dongdong Tian <[email protected]>
    @weiji14 @seisman
    weiji14 and seisman committed Dec 16, 2023
    Configuration menu
    Copy the full SHA
    71946f8 View commit details
    Browse the repository at this point in the history