[SPARK-23375][SQL] Eliminate unneeded Sort in Optimizer #20560

mgaido91 · 2018-02-09T14:08:11Z

What changes were proposed in this pull request?

Added a new rule to remove Sort operation when its child is already sorted.
For instance, this simple code:

spark.sparkContext.parallelize(Seq(("a", "b"))).toDF("a", "b").registerTempTable("table1")
val df = sql(s"""SELECT b
                | FROM (
                |     SELECT a, b
                |     FROM table1
                |     ORDER BY a
                | ) t
                | ORDER BY a""".stripMargin)
df.explain(true)

before the PR produces this plan:

== Parsed Logical Plan ==
'Sort ['a ASC NULLS FIRST], true
+- 'Project ['b]
   +- 'SubqueryAlias t
      +- 'Sort ['a ASC NULLS FIRST], true
         +- 'Project ['a, 'b]
            +- 'UnresolvedRelation `table1`

== Analyzed Logical Plan ==
b: string
Project [b#7]
+- Sort [a#6 ASC NULLS FIRST], true
   +- Project [b#7, a#6]
      +- SubqueryAlias t
         +- Sort [a#6 ASC NULLS FIRST], true
            +- Project [a#6, b#7]
               +- SubqueryAlias table1
                  +- Project [_1#3 AS a#6, _2#4 AS b#7]
                     +- SerializeFromObject [staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, assertnotnull(assertnotnull(input[0, scala.Tuple2, true]))._1, true, false) AS _1#3, staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, assertnotnull(assertnotnull(input[0, scala.Tuple2, true]))._2, true, false) AS _2#4]
                        +- ExternalRDD [obj#2]

== Optimized Logical Plan ==
Project [b#7]
+- Sort [a#6 ASC NULLS FIRST], true
   +- Project [b#7, a#6]
      +- Sort [a#6 ASC NULLS FIRST], true
         +- Project [_1#3 AS a#6, _2#4 AS b#7]
            +- SerializeFromObject [staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, assertnotnull(input[0, scala.Tuple2, true])._1, true, false) AS _1#3, staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, assertnotnull(input[0, scala.Tuple2, true])._2, true, false) AS _2#4]
               +- ExternalRDD [obj#2]

== Physical Plan ==
*(3) Project [b#7]
+- *(3) Sort [a#6 ASC NULLS FIRST], true, 0
   +- Exchange rangepartitioning(a#6 ASC NULLS FIRST, 200)
      +- *(2) Project [b#7, a#6]
         +- *(2) Sort [a#6 ASC NULLS FIRST], true, 0
            +- Exchange rangepartitioning(a#6 ASC NULLS FIRST, 200)
               +- *(1) Project [_1#3 AS a#6, _2#4 AS b#7]
                  +- *(1) SerializeFromObject [staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, assertnotnull(input[0, scala.Tuple2, true])._1, true, false) AS _1#3, staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, assertnotnull(input[0, scala.Tuple2, true])._2, true, false) AS _2#4]
                     +- Scan ExternalRDDScan[obj#2]

while after the PR produces:

== Parsed Logical Plan ==
'Sort ['a ASC NULLS FIRST], true
+- 'Project ['b]
   +- 'SubqueryAlias t
      +- 'Sort ['a ASC NULLS FIRST], true
         +- 'Project ['a, 'b]
            +- 'UnresolvedRelation `table1`

== Analyzed Logical Plan ==
b: string
Project [b#7]
+- Sort [a#6 ASC NULLS FIRST], true
   +- Project [b#7, a#6]
      +- SubqueryAlias t
         +- Sort [a#6 ASC NULLS FIRST], true
            +- Project [a#6, b#7]
               +- SubqueryAlias table1
                  +- Project [_1#3 AS a#6, _2#4 AS b#7]
                     +- SerializeFromObject [staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, assertnotnull(assertnotnull(input[0, scala.Tuple2, true]))._1, true, false) AS _1#3, staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, assertnotnull(assertnotnull(input[0, scala.Tuple2, true]))._2, true, false) AS _2#4]
                        +- ExternalRDD [obj#2]

== Optimized Logical Plan ==
Project [b#7]
+- Sort [a#6 ASC NULLS FIRST], true
   +- Project [_1#3 AS a#6, _2#4 AS b#7]
      +- SerializeFromObject [staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, assertnotnull(input[0, scala.Tuple2, true])._1, true, false) AS _1#3, staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, assertnotnull(input[0, scala.Tuple2, true])._2, true, false) AS _2#4]
         +- ExternalRDD [obj#2]

== Physical Plan ==
*(2) Project [b#7]
+- *(2) Sort [a#6 ASC NULLS FIRST], true, 0
   +- Exchange rangepartitioning(a#6 ASC NULLS FIRST, 5)
      +- *(1) Project [_1#3 AS a#6, _2#4 AS b#7]
         +- *(1) SerializeFromObject [staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, assertnotnull(input[0, scala.Tuple2, true])._1, true, false) AS _1#3, staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, assertnotnull(input[0, scala.Tuple2, true])._2, true, false) AS _2#4]
            +- Scan ExternalRDDScan[obj#2]

this means that an unnecessary sort operation is not performed after the PR.

How was this patch tested?

added UT

SparkQA · 2018-02-09T17:22:57Z

Test build #87261 has finished for PR 20560 at commit 550ff99.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
abstract class KeepOrderUnaryNode extends UnaryNode
case class Subquery(child: LogicalPlan) extends KeepOrderUnaryNode
case class Project(projectList: Seq[NamedExpression], child: LogicalPlan)
case class GlobalLimit(limitExpr: Expression, child: LogicalPlan) extends KeepOrderUnaryNode

gatorsmile · 2018-02-09T18:14:44Z

Thanks! This should be added as a separate rule. It is actually to resolve the comment in #11480 (comment)

I did not review it carefully, but it requires more test cases, including unit tests and end-to-end tests

mgaido91 · 2018-02-10T12:10:00Z

@gatorsmile thanks for your comment. I moved it to a separate rule and added more tests.

As per the added value of this rule, I see 3 main points:

Let's imagine that a user exposes a cached sorted relation which can be queried by other users via JDBC. Other users cannot know that the table is already sorted and they may write query which cause an unnecessary sort.
Many tools which produce automatic SQL code are not very smart in creating it, so they can generate queries which cause unneeded sorts.
I think this is also enabling for more interesting use cases. What I am thinking about is that we may have some datasources which store sorted data and if we can express this in the logical plan, then we may avoid unneeded sorts.

What do you think?
Thanks.

SparkQA · 2018-02-10T15:13:52Z

Test build #87288 has finished for PR 20560 at commit 81e4828.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class LocalLimit(limitExpr: Expression, child: LogicalPlan) extends KeepOrderUnaryNode

gatorsmile · 2018-02-10T17:28:58Z

@mgaido91 Yeah, we definitely should include this rule. We just need more careful review and comprehensive test cases. Thanks for your work!

mgaido91 · 2018-02-10T17:31:53Z

thank you @gatorsmile for taking a look at this. Let me know if there is something I can/should improve. Thanks.

mgaido91 · 2018-03-01T16:37:09Z

@gatorsmile sorry, do you have time now to take a look at this? Or may I ping you some days later if you are busy? Thanks.

gatorsmile · 2018-03-01T17:05:33Z

Will review this in the next few days.

mgaido91 · 2018-03-28T11:08:37Z

kindly ping @gatorsmile

gatorsmile · 2018-03-30T16:55:58Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala

+  /**
+   * If the current plan contains sorted data, it contains the sorted order.
+   */
+  def sortedOrder: Seq[SortOrder] = Nil


def outputOrdering?

gatorsmile · 2018-03-30T16:58:35Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala

@@ -219,6 +219,11 @@ abstract class LogicalPlan
   * Refreshes (or invalidates) any metadata/data cached in the plan recursively.
   */
  def refresh(): Unit = children.foreach(_.refresh())
+
+  /**
+   * If the current plan contains sorted data, it contains the sorted order.


Returns the output ordering that this plan generates.

gatorsmile · 2018-03-30T17:04:59Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala

+object RemoveRedundantSorts extends Rule[LogicalPlan] {
+  def apply(plan: LogicalPlan): LogicalPlan = plan transform {
+    case Sort(orders, true, child) if child.sortedOrder.nonEmpty
+        && child.sortedOrder.zip(orders).forall { case (s1, s2) => s1.satisfies(s2) } =>


Why not using SortOrder.orderingSatisfies?

gatorsmile · 2018-03-30T17:07:14Z

cc @cloud-fan @hvanhovell @wzhfy

mgaido91 · 2018-04-03T14:38:02Z

retest this please

SparkQA · 2018-04-03T18:44:05Z

Test build #88849 has finished for PR 20560 at commit 1c33263.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-04-04T07:05:23Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala

+object RemoveRedundantSorts extends Rule[LogicalPlan] {
+  def apply(plan: LogicalPlan): LogicalPlan = plan transform {
+    case Sort(orders, true, child) if child.outputOrdering.nonEmpty
+        && SortOrder.orderingSatisfies(child.outputOrdering, orders) =>


shall we do it after planning as we already have SparkPlan.outputOrdering?

ah they are different. This is global ordering,

cloud-fan · 2018-04-04T07:06:47Z

...alyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala

  override def output: Seq[Attribute] = child.output
 }

-case class Project(projectList: Seq[NamedExpression], child: LogicalPlan) extends UnaryNode {
+case class Project(projectList: Seq[NamedExpression], child: LogicalPlan)


Like ProjectExec.outputOrdering, we can propagate ordering for aliased attributes.

sorry, I don't fully understand what you mean. In ProjectExec.outputOrdering we are getting the child.outputOrdering exactly as it is done here.

cloud-fan · 2018-04-04T07:08:18Z

...alyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala

@@ -867,6 +871,11 @@ case class RepartitionByExpression(

  override def maxRows: Option[Long] = child.maxRows
  override def shuffle: Boolean = true
+
+  override def outputOrdering: Seq[SortOrder] = partitioning match {
+    case RangePartitioning(ordering, _) => ordering


RangePartitioning doesn't guarantee ordering inside partition, we can't do this.

SparkQA · 2018-04-04T16:06:43Z

Test build #88888 has finished for PR 20560 at commit 60ea6fc.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

wzhfy · 2018-04-10T11:45:10Z

...alyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RemoveRedundantSortsSuite.scala

+class RemoveRedundantSortsSuite extends PlanTest {
+  override val conf = new SQLConf().copy(CASE_SENSITIVE -> true, ORDER_BY_ORDINAL -> false)
+  val catalog = new SessionCatalog(new InMemoryCatalog, EmptyFunctionRegistry, conf)
+  val analyzer = new Analyzer(catalog, conf)


If we don't use ordinal number, we can remove these.

wzhfy · 2018-04-10T11:45:55Z

...alyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RemoveRedundantSortsSuite.scala

+  test("remove redundant order by") {
+    val orderedPlan = testRelation.select('a, 'b).orderBy('a.asc, 'b.desc_nullsFirst)
+    val unnecessaryReordered = orderedPlan.select('a).orderBy('a.asc, 'b.desc_nullsFirst)
+    val optimized = Optimize.execute(analyzer.execute(unnecessaryReordered))


just use unnecessaryReordered.analyze?

wzhfy · 2018-04-10T11:47:32Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala

@@ -733,6 +735,17 @@ object EliminateSorts extends Rule[LogicalPlan] {
  }
 }

+/**
+ * Removes Sort operations on already sorted data


how about Removes Sort operation if the child is already sorted?

wzhfy · 2018-04-10T11:49:35Z

...alyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala

@@ -522,6 +524,8 @@ case class Range(
  override def computeStats(): Statistics = {
    Statistics(sizeInBytes = LongType.defaultSize * numElements)
  }
+
+  override def outputOrdering: Seq[SortOrder] = output.map(a => SortOrder(a, Descending))


ordering is the same when step in Range is positive or negative?

Nice catch, thanks! I missed it!

wzhfy · 2018-04-10T11:51:56Z

sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala

+    val resorted = query.sort('key.desc)
+    assert(resorted.queryExecution.optimizedPlan.collect { case s: Sort => s}.isEmpty)
+    assert(resorted.select('key).collect().map(_.getInt(0)).toSeq ==
+      (1 to 100).sorted(Ordering[Int].reverse))


(1 to 100).reverse?

wzhfy · 2018-04-10T11:52:40Z

sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala

+      (1 to 100).sorted(Ordering[Int].reverse))
+    // with a different order, the sort is needed
+    val sortedAsc = query.sort('key)
+    assert(sortedAsc.queryExecution.optimizedPlan.collect { case s: Sort => s}.nonEmpty)


.nonEmpty -> .size == 1

SparkQA · 2018-04-10T15:03:25Z

Test build #89118 has finished for PR 20560 at commit 1c7cae6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-04-10T19:10:50Z

Test build #89135 has finished for PR 20560 at commit e376c19.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

henryr

Just a couple suggestions, feel free to ignore.

henryr · 2018-04-04T01:08:11Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala

@@ -274,3 +279,7 @@ abstract class BinaryNode extends LogicalPlan {

  override final def children: Seq[LogicalPlan] = Seq(left, right)
 }
+
+abstract class KeepOrderUnaryNode extends UnaryNode {


OrderPreservingUnaryNode? Or perhaps do you think this would be better modeled as a mixin trait?

thanks for the suggestion. I'd love to hear also @cloud-fan's and @wzhfy's opinion on this in order to choose all together the best name for it. What do you think?

OrderPreservingUnaryNode sounds better.

It only makes sense for unary node, so I don't think mixin trait is a good idea.

henryr · 2018-04-10T23:03:40Z

sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala

 import org.apache.spark.sql.catalyst.plans.physical._
 import org.apache.spark.sql.execution.columnar.InMemoryRelation
-import org.apache.spark.sql.execution.exchange.{EnsureRequirements, ReusedExchangeExec, ReuseExchange, ShuffleExchangeExec}
+import org.apache.spark.sql.execution.exchange.{EnsureRequirements, ReusedExchangeExec, ReuseExchange,
+  ShuffleExchangeExec}


revert this?

it's a unnecessary change. We don't have length limit for imports

henryr · 2018-04-11T00:00:33Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala

+    case Sort(orders, true, child) if child.outputOrdering.nonEmpty
+        && SortOrder.orderingSatisfies(child.outputOrdering, orders) =>
+      child
+  }


You might not want to do it in this PR, but you could easily remove another simple kind of redundant sort, e.g.:

rel.orderBy('a.desc).orderBy('a.asc)

(and I think that orderBy is not stable, so any two consecutive orderBy operators are redundant).

Yes, you're right. Probably we can do this in other PR. May you open a JIRA for this? Thanks!

This is a good follow-up

Filed SPARK-23973 for this

cloud-fan · 2018-04-11T11:09:57Z

sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala

@@ -169,4 +169,6 @@ case class InMemoryRelation(

  override protected def otherCopyArgs: Seq[AnyRef] =
    Seq(_cachedColumnBuffers, sizeInBytesStats, statsOfPlanToCache)
+
+  override def outputOrdering: Seq[SortOrder] = child.outputOrdering


in SparkPlan

/** Specifies how data is ordered in each partition. */ def outputOrdering: Seq[SortOrder] = Nil

So we can't do this

We should carry the logical ordering from the cached logical plan when building the InMemoryRelation

SparkQA · 2018-04-11T13:43:38Z

Test build #89195 has finished for PR 20560 at commit a1846ab.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-04-11T16:51:05Z

Test build #89197 has finished for PR 20560 at commit 4e441f8.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-04-12T10:52:26Z

Test build #89249 has finished for PR 20560 at commit 6e95e37.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

mgaido91 · 2018-04-12T10:59:04Z

retest this please

SparkQA · 2018-04-12T14:48:16Z

Test build #89257 has finished for PR 20560 at commit 6e95e37.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-04-13T02:36:55Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala

+ */
+object RemoveRedundantSorts extends Rule[LogicalPlan] {
+  def apply(plan: LogicalPlan): LogicalPlan = plan transform {
+    case Sort(orders, true, child) if child.outputOrdering.nonEmpty


child.outputOrdering.nonEmpty looks like unnecessary

SparkQA · 2018-04-13T13:27:47Z

Test build #89330 has finished for PR 20560 at commit 6c5f04c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-04-13T17:01:10Z

thanks, merging to master!

## What changes were proposed in this pull request? In SPARK-23375 we introduced the ability of removing `Sort` operation during query optimization if the data is already sorted. In this follow-up we remove also a `Sort` which is followed by another `Sort`: in this case the first sort is not needed and can be safely removed. The PR starts from henryr's comment: apache#20560 (comment). So credit should be given to him. ## How was this patch tested? added UT Author: Marco Gaido <[email protected]> Closes apache#21072 from mgaido91/SPARK-23973.

rxin · 2018-04-25T21:28:06Z

Just saw this - this seems like a somewhat awkward way to do it by just matching on filter / project. Is the main thing lacking a way to do back propagation for properties? (We can only do forward propagation at the moment on properties so we can't eliminate subtree's sort based on the parent's sort).

cloud-fan · 2018-04-26T02:09:29Z

@rxin It seems you are talking about the followup PR: #21072

I think this is the way we do back propagation in catalyst: match a specific node, traverse down the subtree with the properties.

For forward propagation, we also need to carefully handle some nodes that would stop the propagation. In RemoveRedundantSorts.canEliminateSort, we are doing the same thing: only list the nodes that can retain the properties. e.g. Limit should stop propagating the sorting property. I think Project, Filter, Hint is good enough as an initial list, we can expand it later.

gatorsmile · 2018-10-13T19:02:11Z

sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala

@@ -64,7 +64,8 @@ case class InMemoryRelation(
    tableName: Option[String])(
    @transient var _cachedColumnBuffers: RDD[CachedBatch] = null,
    val sizeInBytesStats: LongAccumulator = child.sqlContext.sparkContext.longAccumulator,
-    statsOfPlanToCache: Statistics)
+    statsOfPlanToCache: Statistics,
+    override val outputOrdering: Seq[SortOrder])


This should be added to otherCopyArgs ; otherwise, we will lose it when doing the tree transformation. #22715 fixed it.

…ssing ## What changes were proposed in this pull request? #20560/[SPARK-23375](https://issues.apache.org/jira/browse/SPARK-23375) introduced an optimizer rule to eliminate redundant Sort. For a test case named "Sort metrics" in `SQLMetricsSuite`, because range is already sorted, sort is removed by the `RemoveRedundantSorts`, which makes this test case meaningless. This PR modifies the query for testing Sort metrics and checks Sort exists in the plan. ## How was this patch tested? Modify the existing test case. Closes #23258 from seancxmao/sort-metrics. Authored-by: seancxmao <[email protected]> Signed-off-by: Sean Owen <[email protected]>

…ssing ## What changes were proposed in this pull request? apache#20560/[SPARK-23375](https://issues.apache.org/jira/browse/SPARK-23375) introduced an optimizer rule to eliminate redundant Sort. For a test case named "Sort metrics" in `SQLMetricsSuite`, because range is already sorted, sort is removed by the `RemoveRedundantSorts`, which makes this test case meaningless. This PR modifies the query for testing Sort metrics and checks Sort exists in the plan. ## How was this patch tested? Modify the existing test case. Closes apache#23258 from seancxmao/sort-metrics. Authored-by: seancxmao <[email protected]> Signed-off-by: Sean Owen <[email protected]>

[SPARK-23375][SQL] Eliminate unneeded Sort in Optimizer

550ff99

Use separate rule and add more tests

81e4828

gatorsmile reviewed Mar 30, 2018

View reviewed changes

rename to outputOrdering

1c33263

cloud-fan reviewed Apr 4, 2018

View reviewed changes

remove outputOrdering for RangePartitioning

60ea6fc

wzhfy reviewed Apr 10, 2018

View reviewed changes

fix Range ordering + add more Range UT + address comments

1c7cae6

fix ut failure

e376c19

henryr reviewed Apr 11, 2018

View reviewed changes

cloud-fan reviewed Apr 11, 2018

View reviewed changes

mgaido91 added 2 commits April 11, 2018 15:04

rename to OrderPreservingUnaryNode

930ef7b

fix InMemoryRelation

a1846ab

mgaido91 added 2 commits April 11, 2018 16:09

Merge branch 'master' of github.com:apache/spark into SPARK-23375

0cd4899

fix build failure

4e441f8

fix UT

6e95e37

cloud-fan reviewed Apr 13, 2018

View reviewed changes

address comment

6c5f04c

asfgit closed this in 25892f3 Apr 13, 2018

mgaido91 mentioned this pull request Apr 14, 2018

[SPARK-23973][SQL] Remove consecutive Sorts #21072

Closed

gatorsmile reviewed Oct 13, 2018

View reviewed changes

This was referenced Dec 8, 2018

[SPARK-23375][SQL][FOLLOWUP][TEST] Test Sort metrics while Sort is missing #23258

Closed

[SPARK-26277][SQL][TEST] WholeStageCodegen metrics should be tested with whole-stage codegen enabled #23224

Closed

c21 mentioned this pull request Nov 15, 2021

[SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter #34568

Closed

[SPARK-23375][SQL] Eliminate unneeded Sort in Optimizer #20560

[SPARK-23375][SQL] Eliminate unneeded Sort in Optimizer #20560

Conversation

mgaido91 commented Feb 9, 2018

What changes were proposed in this pull request?

How was this patch tested?

SparkQA commented Feb 9, 2018

gatorsmile commented Feb 9, 2018

mgaido91 commented Feb 10, 2018

SparkQA commented Feb 10, 2018

gatorsmile commented Feb 10, 2018

mgaido91 commented Feb 10, 2018

mgaido91 commented Mar 1, 2018

gatorsmile commented Mar 1, 2018

mgaido91 commented Mar 28, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gatorsmile commented Mar 30, 2018

mgaido91 commented Apr 3, 2018

SparkQA commented Apr 3, 2018

cloud-fan Apr 4, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SparkQA commented Apr 4, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SparkQA commented Apr 10, 2018

SparkQA commented Apr 10, 2018

henryr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SparkQA commented Apr 11, 2018

SparkQA commented Apr 11, 2018

SparkQA commented Apr 12, 2018

mgaido91 commented Apr 12, 2018

SparkQA commented Apr 12, 2018

Choose a reason for hiding this comment

SparkQA commented Apr 13, 2018

cloud-fan commented Apr 13, 2018

rxin commented Apr 25, 2018

cloud-fan commented Apr 26, 2018

Choose a reason for hiding this comment

cloud-fan Apr 4, 2018 •

edited

Loading