Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: streamline system.temp_files table scan #16659

Merged
merged 7 commits into from
Oct 28, 2024
Merged

refactor: streamline system.temp_files table scan #16659

merged 7 commits into from
Oct 28, 2024

Conversation

dantengsky
Copy link
Member

@dantengsky dantengsky commented Oct 22, 2024 •
edited
Loading

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Changed the table scan of system.temp_files to return the result set as a stream instead of a single block, providing better responsiveness and memory usage.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Refactoring

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@dantengsky
refactor: streamline system.temp_files result set
c178d78 
Changed the implementation to return the result set as a stream instead
of a single block, providing better responsiveness and memory usage.
@dantengsky
chore: remove old implementation
688b175
@github-actions github-actions bot added the pr-refactor this PR changes the code base without new features or bugfix label Oct 22, 2024
@dantengsky dantengsky changed the title refactor: streamline system.temp_files result set refactor: streamline system.temp_files table scan Oct 22, 2024
@dantengsky
fix: incorrect finish processing
6fbdc20 
should not stop processing if only step_limit is reached
@dantengsky dantengsky force-pushed the chore-refine-system-temp-files branch from 96d8f01 to 6fbdc20 Compare October 23, 2024 02:04
@dantengsky dantengsky marked this pull request as ready for review October 23, 2024 15:01
@dantengsky
Copy link
Member Author

@sundy-li PTAL

@BohuTANG BohuTANG merged commit 6f7ede7 into databendlabs:main Oct 28, 2024
72 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sundy-li sundy-li sundy-li approved these changes

Assignees
No one assigned
Labels
pr-refactor this PR changes the code base without new features or bugfix
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dantengsky @sundy-li @BohuTANG