Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SelectExpression.VisitChildren optimizations #17594

Merged
merged 1 commit into from
Sep 3, 2019

Conversation

roji
Copy link
Member

@roji roji commented Sep 3, 2019

Profiling the scenario in #17455 showed that SelectExpression.VisitChildren was a major compilation perf hotspot (own time, not including child calls). This PR changes the logic to only allocate new data structures and copy if a Visit actually returned a different result.

This reduced the time for #17455 from 22 seconds to 15 on my machine. Memory allocated notably goes down from 61.86MB to 12.17MB.

Also added a compilation micro-benchmark which does three joins, here are the results. Note that #17455 includes 6 results and so the difference is considerably more significant.

Before

BenchmarkDotNet=v0.11.3, OS=ubuntu 19.04
Intel Xeon W-2133 CPU 3.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=3.0.100-preview8-013656
  [Host] : .NET Core 3.0.0-rc1-19430-09 (CoreCLR 4.700.19.43003, CoreFX 4.700.19.42010), 64bit RyuJIT

Toolchain=InProcessToolchain  

        Method |     Mean |    Error |   StdDev | Gen 0/1k Op | Gen 1/1k Op | Gen 2/1k Op | Allocated Memory/Op |
-------------- |---------:|---------:|---------:|------------:|------------:|------------:|--------------------:|
 MultipleJoins | 215.8 ms | 4.067 ms | 3.605 ms |  15000.0000 |   1000.0000 |           - |            61.86 MB |

After

BenchmarkDotNet=v0.11.3, OS=ubuntu 19.04
Intel Xeon W-2133 CPU 3.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=3.0.100-preview8-013656
  [Host] : .NET Core 3.0.0-rc1-19430-09 (CoreCLR 4.700.19.43003, CoreFX 4.700.19.42010), 64bit RyuJIT

Toolchain=InProcessToolchain  

        Method |     Mean |    Error |   StdDev | Gen 0/1k Op | Gen 1/1k Op | Gen 2/1k Op | Allocated Memory/Op |
-------------- |---------:|---------:|---------:|------------:|------------:|------------:|--------------------:|
 MultipleJoins | 173.2 ms | 3.538 ms | 3.309 ms |   2666.6667 |    666.6667 |           - |            12.17 MB |

Do not allocate and copy the different components if visitation hasn't
changed anything.

Affects #17455
@smitpatel
Copy link
Contributor

LGTM as long as test pass. Requested review from Andriy just in case.


changed |= projection != item;
if (newProjections != _projection)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (newProjections != _projection)
if (changed)

Copy link
Member

@AndriySvyryd AndriySvyryd Sep 3, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also in the case below

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Careful, changed tracks if any component of the SelectExpression was changed, including previous unrelated components. This change seems like it would make us falsely go into the condition.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it should be fine for these two cases, since changed is not mutated before

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see... I'd rather leave it like this - who knows if we'll shift the code fragments around and this would become invalid... There's no real gain in replacing a single reference comparison with a bool check...

@roji roji merged commit 55013bb into release/3.1 Sep 3, 2019
@ghost ghost deleted the SelectExpressionOptimization branch September 3, 2019 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants