saving awkward array

[yehabdelaziz]

Hello everyone,

I am having trouble saving some awkward arrays. I tried saving them in the awkward.to_parquet but it is giving me this error:

AttributeError: 'Record' object has no attribute 'form'

This is the code that I ran with:

import awkward as ak
import collections

# Example Awkward Arrays
px = ak.Array([[1.0, 2.0], [3.0], [], [4.0, 5.0, 6.0]])
py = ak.Array([[0.5, 1.5], [2.5], [], [3.5, 4.5, 5.5]])
pz = ak.Array([[0.2, 1.2], [2.2], [], [3.2, 4.2, 5.2]])
e  = ak.Array([[0.1, 1.1], [2.1], [], [3.1, 4.1, 5.1]])

# Create an OrderedDict with Awkward Arrays
data = collections.OrderedDict({
    "px": px,
    "py": py,
    "pz": pz,
    "e": e
})

# Save to a Parquet file
ak.to_parquet(data, "output.parquet")

# Load from Parquet file
loaded_data = ak.from_parquet("output.parquet")
print(loaded_data)

[jpivarski]

The natural argument type for ak.to_parquet is an ak.Array or ak.Record, not collections.OrderedDict.¹

What's supposed to happen in this case is that the collections.OrderedDict (collection satisfying the Mapping protocol) would get promoted to a ak.Record, which worked. What didn't work is that we checked for a top-level record by seeing if the original data is an instance of the ak.Record, which isn't true. Instead, we should look at the promoted version of the input, rather than the original version. That will be fixed in #3175.

For your script, you probably want to do

data = ak.Record({
    "px": px,
    "py": py,
    "pz": pz,
    "e": e
})

which leaves the px, py, pz, e as separate ragged arrays, similar to collections.OrderedDict or use ak.zip

data = ak.zip({
    "px": px,
    "py": py,
    "pz": pz,
    "e": e
})

which makes a ragged arrays of records instead. (To see the difference, print the types and notice where the "var *" is.) Furthermore, you might want this Lorentz vector to have Vector methods, so do

import vector
vector.register_awkward()

data = ak.zip({
    "px": px,
    "py": py,
    "pz": pz,
    "e": e
}, with_name="Momentum4D")

and then you can do things like data.pt (even after saving and loading from Parquet, as long as vector.register_awkward() has been called in the new Python session).

Footnotes

1. Also, since Python 3.6 or so, all dicts are now ordered, making that collection type obsolete. Nobody needs to use it anymore; I think it's only in the standard library because removing it would break old code. ↩