You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Not needed for BlockCombineAll, since a large state is not accumulated there.
After single Fetch call an array of UnboxedValue in returned, each element of which is a wrapper around the arrow datum (one column).
The state is stored either in the RH hash table or in Arena, but with pointers in RH. Payload comes immediately after the keys.
When spilling, we will need to go through the datum and check each key separately - whether we will spill it or not. In general, the algorithm may be the same as in WideLastCombine.
mkql_block_add.cpp will use the methods added here: #10726
During spilling methods Serialize/Deserialize should be used like it's done here:
Input data is always serializable. The output is not always, so it will be necessary to separately add a serialization/deserialization function similar to #7300
As part of this task, the spilling should be turned off and on by changing the code.
The text was updated successfully, but these errors were encountered:
ydb/ydb/library/yql/minikql/comp_nodes/mkql_block_agg.cpp
Line 1674 in 08196b8
Not needed for BlockCombineAll, since a large state is not accumulated there.
After single Fetch call an array of UnboxedValue in returned, each element of which is a wrapper around the arrow datum (one column).
The state is stored either in the RH hash table or in Arena, but with pointers in RH. Payload comes immediately after the keys.
When spilling, we will need to go through the datum and check each key separately - whether we will spill it or not. In general, the algorithm may be the same as in WideLastCombine.
mkql_block_add.cpp will use the methods added here: #10726
During spilling methods Serialize/Deserialize should be used like it's done here:
ydb/ydb/library/yql/minikql/comp_nodes/mkql_block_agg.cpp
Line 1535 in 7a1e697
Input data is always serializable. The output is not always, so it will be necessary to separately add a serialization/deserialization function similar to #7300
As part of this task, the spilling should be turned off and on by changing the code.
The text was updated successfully, but these errors were encountered: