fix: fuse_snapshot
only show the latest snapshot for external table
#15721
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/
Summary
Before this PR, during the traversal of the table history, the application-level data operator is used. However, this approach fails to load the snapshot history of external tables because their storage parameters differ from those of the application-level data operator.
The confusing part is that during the process of writing a new snapshot, we also generate the snapshot cache. Therefore, on the node where data is written to the external table, despite the aforementioned issue, the traversal of the table history snapshot still works because the snapshot can be retrieved from the cache.
However, on other nodes that do not have the snapshot cache mentioned above, the issue persists. This results in
fuse_snapshot
only being able to return the most recent snapshot (the corresponding table snapshot cache is populated during table instantiation).fix
refactors:
rename
TableContext::get_data_operator
toget_application_level_data_operator
clarify that the data operator got from ctx is NOT Table's operator
de-dup code by introducing
shared_table::save_share_table_info
tweak CI script to test with table meta cache disabled
Fixes bug: table function
fuse_snapshot
only show the latest snapshot for external table #15724Tests
Type of change
This change is