How do I make MERGE use deletion vectors? #3796
-
Delta 3.2.1, Spark 3.5.3 bare metal cluster, data in S3 My 1TB table has deletion vectors enabled:
When I perform a MERGE operation on it (when matched update a single field, when not matched insert all), I get the same job in Spark as when I run the same MERGE without deletion vectors: The docs say deletion vectors are not used when "a DML command with deletion vectors disabled (by a command flag or a table property) is run on the table", but how can I tell if my command has deletion vectors disabled? And how do I enable it on a command? UPD: I tested the table with DELETE, and it generated deletion vectors. Then I tried an UPDATE, and got |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
I checked the result of the UPDATE and it generated deletion vectors despite saying "rewriting". The same must apply to MERGE |
Beta Was this translation helpful? Give feedback.
I checked the result of the UPDATE and it generated deletion vectors despite saying "rewriting". The same must apply to MERGE