Upgrade to 181 #98
Merged
Upgrade to 181 #98
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The comment become obsolete in commit 6155b1d.
Must be reverted and replaced with the proper solution after JDBC driver gets the support for specifying so.
The method is removed in Guava 22.0
This allows having multiple executions.
This fixes exec:java from forking the build lifecycle.
This is rarely used and never worked very well.
Factor out the pattern of creating and tearing down the RuleTester in each of the iterative rule unit test classes. Requested by kokosing.
Before this commit, `Operator.finish()` could be called when operator is blocked, but only if it got blocked during last `addInput()`. This commit removes this condition and makes delivery of `finish()` unconditionally clear.
Mock (unregistered) connectors should not be used with RuleTester as some optimizer rules/matcher rules may depend on possibility to access connector metadata. This becomes a problem with addition of cost/statistics computation.
Currently the getSizeInBytes/getRegionSizeInBytes/getRetainedSizeInBytes methods all return integers. To implement these methods Block implementations usually sum up the size of their internal states and cast that sum to integer using different cast functions, which are different in different Block implementations, and occasionally incorrect (using Ints.saturatedCast()).
Move test utility classes for DbResourceGroupConfigurationManager to H2TestUtil
Change this to CONFIGURATION_MANAGER_NAME since that is what the variable represents.
Change the method names for creating new test sessions more descriptive and concise.
Change getSimpleQueryRunner to be a simple wrapper on createQueryRunner in H2TestUtil
Since the properties map only contains one property inline the map.
Since we disable auto-detecting bean properties in the ObjectMapper Jackson was failing to serialize the Exception field of HeartbeatFailureDetector.Stats instances.
For non-transactional connectors the createTable call can fail with ALREADY_EXISTS exception if multiple threads race to see that the table doesn't exist and try to create the table at the same time.
Override dropColumn
This adds an ExecutorService to the IndexLookup class, submitting all tasks to be run in parallel and waiting for the results to come back for each column constraint.
Add a generic ProjectOffPushDownRule to factor out the boilerplate of rules migrated from PruneUnreferencedOutputs. ProjectOffPushDownRule looks for a Project parent over a child of some type N, such that the parent doesn't use all the output columns of the child. Given that situation, it invokes the abstract protected pushDownProjectOff to possibly rewrite the child to produce fewer outputs.
skipOnlyWhen name was very confusing, not it should be easier to understand what it means.
Thanks to that it is easier to understand the plan pattern.
Since there are some expressions which contains Nodes (see FunctionCall, which contains Window which is a Node), to properly validate them context class has to be Node.
Added support for: - SimpleCaseExpression - FunctionCall
All the subquery related nodes have to be rewritten before AddExchanges and AddLocalExchanges. Additionally, remove NoSubqueryRelatedNodeLeftChecker which was a dead code, and now it become even more obsolete.
Migrate PruneUnreferencedOutputs handling of OutputNode to the iterative optimizer. This rule adds an explicit project-off underneath an OutputNode which doesn't use all of it's child's outputs, so that other pruning rules will match against the pattern of a project over the original child.
* OperatorSpillContext declared thread-safety and this commit makes the class thread-safe * LocalSpillContext is used in asynchronous spilling, so it is also made thread-safe
Using a plan node searcher without a lookup is completely fine, as it requires to have accessible fully fledge plan node tree. On the other side, using a plan node searcher with lookup enables optimizer rules to overuse Lookup. Rule by design should not traverse the tree. Hence, such plan node searcher usage should be replaced with usage of traits and better pattern matching.
Gzip compressors created by apache hadoop use system native memory and hold them forever in a codec pool. This can easily lead to memory leak and make memory tracking hard. Replace the compression library with the JDK version. We still use native memory but no longer use codec pool and release memory as soon as we finish compression.
Otherwise this can result in memory leaks.
…lock When computing the retained size of a SliceArrayBlock, an IdentityHashMap will be created with expected size set to the number of slices. For a SliceArrayBlock contains 1 million slices, the IdentityHashMap will contain an Object array with 4,194,304 entires which will cause humongous allocation. Most of the slices will share the same base byte array, and setting such huge expected size is a waste and cause GC issues. This commit changes to use the default expected size. This aligns with other usage of IdentityHashMap including SliceArrayBlock.deepCopyAndCompact.
…function signature
|
@xiaoyao1991, earlier merge was to master instead of twitter-master |
|
👍 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.