Skip to content

Commit

Permalink
Requirements document for the dataframe interchange protocol (#35)
Browse files Browse the repository at this point in the history
* Add a summary document for the dataframe interchange protocol

Summarizes the various discussions about and goals/non-goals
and requirements for the `__dataframe__` data interchange
protocol.

The intended audience for this document is Consortium members
and dataframe library maintainers who may want to support this
protocol.

The aim is to keep updating this till we have captured all
the requirements and answered all the FAQs, so we can actually
design the protocol after and verify it meets all our requirements.

Closes gh-29

* Process some review comments

* Process a few more review comments.

* Link to Release callback semantics in Arrow C Data Interface docs

* Add design requirements for column selection and df metadata

* Edit the nested/heterogeneous dtypes non-requirement

* Add requirements for chunking and memory layout description

Also address some smaller review comments.

* Add TBD notes on dataframe-array connection and from_dataframe

Also add more details on the Arrow C Data Interface.

* Address review comments

* Add details on implementation options

* Add details about the C implementation

* Add an image of the dataframe model and its memory layout.

* Add link to discussion on array-dataframe connection

* Some more updates for review comments

* Update table to indicate Arrow does support categoricals.

* Add section on dtype format strings

* Reflow some lines

* Add a requirement on semantic meaning of NaN/NaT, and timezone detail

* Textual tweak: say columns in a data frame are ordered

* Update requirements document for recent decisions/insights
  • Loading branch information
rgommers authored Jun 25, 2021
1 parent 6af8c2a commit 2e3914f
Show file tree
Hide file tree
Showing 4 changed files with 998 additions and 0 deletions.
Binary file added protocol/conceptual_model_df_memory.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 2e3914f

Please sign in to comment.