You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, when user makes multiple tables point to same location, running removeOrphanFiles on one of them will destroy the data of other tables.
Its a known issue, the the concept of table ownership is discussed at: #4159. Looks like there are two general directions, either
The user explicitly chooses to share table locations among tables (to take more advantage of "write.object-storage.enabled"), and then instead of using table-level removeOrphanFiles, uses some warehouse-level remove orphan file (not currently in Iceberg OSS project)
We discourage sharing of table locations for all other users.
This issue is to achieve the second one. Initial suggestions (can be changed)
Create/Alter table can check if there's other tables in catalog with the same location
Create/Alter table can check if there's any existing files in location.
This check can be configured at catalog level (?), in case the first option is achieved above and user wants to put tables in same location.
Though its not a perfect fix, it is a useful check to catch increasing problem, and it matches Trino-side fix, eg: trinodb/trino#12749
Query engine
None
The text was updated successfully, but these errors were encountered:
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
Feature Request / Improvement
Currently, when user makes multiple tables point to same location, running removeOrphanFiles on one of them will destroy the data of other tables.
Its a known issue, the the concept of table ownership is discussed at: #4159. Looks like there are two general directions, either
This issue is to achieve the second one. Initial suggestions (can be changed)
Though its not a perfect fix, it is a useful check to catch increasing problem, and it matches Trino-side fix, eg: trinodb/trino#12749
Query engine
None
The text was updated successfully, but these errors were encountered: