Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Record and control the memory usage for index join's build range phase #28650

Closed
winoros opened this issue Oct 8, 2021 · 1 comment · Fixed by #29068
Closed

Record and control the memory usage for index join's build range phase #28650

winoros opened this issue Oct 8, 2021 · 1 comment · Fixed by #29068
Labels
report/customer Customers have encountered this bug. sig/execution SIG execution sig/planner SIG: Planner type/enhancement The issue or PR belongs to an enhancement.

Comments

@winoros
Copy link
Member

winoros commented Oct 8, 2021

Enhancement

When the SQL is something like .... from t1, t2 where t1.a=t2.a and t2.b in (...) and the IN CLAUSE is rather big, we may consume a lot of memory to build the index ranges if we choose the index join and t1 is the driven table.

We need to record these memory usages and design a way to control its usage to avoid the possible OOM risk.

@winoros winoros added type/enhancement The issue or PR belongs to an enhancement. sig/planner SIG: Planner sig/execution SIG execution labels Oct 8, 2021
@XuHuaiyu
Copy link
Contributor

When the number of the arguments in the IN CLAUSE is rather big, OOM may happen cause the memory consumption of encoding these arguments is not tracked.
We can reproduce this problem using the following file:
test.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
report/customer Customers have encountered this bug. sig/execution SIG execution sig/planner SIG: Planner type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants