Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JavaScript] Allow index access for Table, RecordBatch, and Vector #34936

Open
shuhaowu opened this issue Apr 6, 2023 · 0 comments · May be fixed by #34939
Open

[JavaScript] Allow index access for Table, RecordBatch, and Vector #34936

shuhaowu opened this issue Apr 6, 2023 · 0 comments · May be fixed by #34939

Comments

@shuhaowu
Copy link

shuhaowu commented Apr 6, 2023

Describe the enhancement requested

Certain codebases that previously uses row-oriented way to access data may wish to migrate to Arrow to save serialization and deserialization cost, and to be able to gain access to fast column-oriented operations. As it stands, Arrow is sort of a drop-in replacement to row-oriented data such as a JavaScript Array of objects. This is great to incrementally migrate legacy codebases to Arrow, as it is frequently infeasible to rewrite the application to use the column-oriented data access patterns. For most data, JavaScript-object-compatible and row-oriented access is already provided via the StructRowProxy. However, if the structs themselves include a Vector, existing code will break as it assumes the Vector object to behave like a JavaScript array, which it does not due to the lack of index access. An example of such a data structure is as follows:

[
  {x: 1, y: [1, 2]},
  {x: 2, y: [2, 3]},
]

In this case, with the Arrow JS library as it is, the API consumer is unable to get individual element of the y array via table[i].y[j]. Instead, the API consumer must use the API table.get(i).y.get(j). In the situation where we are migrating a legacy code base to Arrow, this requires a large refactor of the entire codebase, which is infeasible in a short time. This negates the advantage of using Arrow as a drop-in replacement and prevents incremental migration of code to Arrow.

Arrow should provide index access for the Vector object, as well as Table and RecordBatch for backward compatibility with JavaScript arrays.

Component(s)

JavaScript

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant