Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding TakeOrderedAndProject and BroadcastNestedLoopJoin, removing Project from speedup generation #548

Merged
merged 1 commit into from
Sep 11, 2023

Conversation

mattahrens
Copy link
Collaborator

Closes #480

Changes in this PR

  • removed ProjectExec from speedup ratio calculation to leave as environment default due to issues with logic found in validation
  • added in BroadcastNestedLoopJoin and TakeOrderedAndProject execs with logic
  • removed SubqueryBroadcastExec and BroadcastExchangeExec from operator list to default to 1.0 given expected speedup to be minimal

Validated on customer event logs and Dataproc SF3K event logs and improved accuracy overall. Dataproc went from ~23% error to ~18% error.

…oject from speedup generation

Signed-off-by: Matt Ahrens <[email protected]>
@mattahrens mattahrens self-assigned this Sep 8, 2023
@mattahrens mattahrens added the core_tools Scope the core module (scala) label Sep 8, 2023
@mattahrens mattahrens merged commit e943d89 into NVIDIA:dev Sep 11, 2023
9 checks passed
@mattahrens mattahrens deleted the add-more-execs-speedup-480 branch September 11, 2023 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core_tools Scope the core module (scala)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] Expand speedup factor calculation for qualification tool estimations to more operators
2 participants