-
Notifications
You must be signed in to change notification settings - Fork 315
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Implement RegionScanner for SeqScan #4060
Conversation
3e15ab6
to
6a04fa5
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4060 +/- ##
==========================================
- Coverage 85.46% 85.17% -0.30%
==========================================
Files 994 994
Lines 174348 174947 +599
==========================================
Hits 149005 149005
- Misses 25343 25942 +599 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
tests/cases/standalone/common/tql-explain-analyze/analyze.result
Outdated
Show resolved
Hide resolved
This seems to be a risky thing. Can we add some indirect layer like a queue or something else to take back the control of parallelism? |
There is already a semaphore to control the parallelism of reading files. |
Signed-off-by: Ruihang Xia <[email protected]>
Signed-off-by: Ruihang Xia <[email protected]>
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
This PR implements RegionScanner for SeqScan.
ScanPart
so it can represent the time range of the part and keeps ranges from different files separately.StreamContext
andScanPartList
to thescan_region
mod so we can reuse it inSeqScan
.SeqDistributor
to organize parts for parallel scanIf there are multiple files to read in a time range, it will spawn a task to read each file. The parallelism is limited by a semaphore in the scanner.
Checklist