You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Certain codebases that previously uses row-oriented way to access data may wish to migrate to Arrow to save serialization and deserialization cost, and to be able to gain access to fast column-oriented operations. As it stands, Arrow is sort of a drop-in replacement to row-oriented data such as a JavaScript Array of objects. This is great to incrementally migrate legacy codebases to Arrow, as it is frequently infeasible to rewrite the application to use the column-oriented data access patterns. For most data, JavaScript-object-compatible and row-oriented access is already provided via the StructRowProxy. However, if the structs themselves include a Vector, existing code will break as it assumes the Vector object to behave like a JavaScript array, which it does not due to the lack of index access. An example of such a data structure is as follows:
[
{x: 1, y: [1, 2]},
{x: 2, y: [2, 3]},
]
In this case, with the Arrow JS library as it is, the API consumer is unable to get individual element of the y array via table[i].y[j]. Instead, the API consumer must use the API table.get(i).y.get(j). In the situation where we are migrating a legacy code base to Arrow, this requires a large refactor of the entire codebase, which is infeasible in a short time. This negates the advantage of using Arrow as a drop-in replacement and prevents incremental migration of code to Arrow.
Arrow should provide index access for the Vector object, as well as Table and RecordBatch for backward compatibility with JavaScript arrays.
Component(s)
JavaScript
The text was updated successfully, but these errors were encountered:
Describe the enhancement requested
Certain codebases that previously uses row-oriented way to access data may wish to migrate to Arrow to save serialization and deserialization cost, and to be able to gain access to fast column-oriented operations. As it stands, Arrow is sort of a drop-in replacement to row-oriented data such as a JavaScript Array of objects. This is great to incrementally migrate legacy codebases to Arrow, as it is frequently infeasible to rewrite the application to use the column-oriented data access patterns. For most data, JavaScript-object-compatible and row-oriented access is already provided via the
StructRowProxy
. However, if the structs themselves include aVector
, existing code will break as it assumes theVector
object to behave like a JavaScript array, which it does not due to the lack of index access. An example of such a data structure is as follows:In this case, with the Arrow JS library as it is, the API consumer is unable to get individual element of the
y
array viatable[i].y[j]
. Instead, the API consumer must use the APItable.get(i).y.get(j)
. In the situation where we are migrating a legacy code base to Arrow, this requires a large refactor of the entire codebase, which is infeasible in a short time. This negates the advantage of using Arrow as a drop-in replacement and prevents incremental migration of code to Arrow.Arrow should provide index access for the
Vector
object, as well asTable
andRecordBatch
for backward compatibility with JavaScript arrays.Component(s)
JavaScript
The text was updated successfully, but these errors were encountered: