[SPARK-35250][SQL][DOCS] Fix duplicated STOP_AT_DELIMITER to SKIP_VAL…

…UE at CSV's unescapedQuoteHandling option documentation ### What changes were proposed in this pull request? This is rather a followup of #30518 that should be ported back to `branch-3.1` too. `STOP_AT_DELIMITER` was mistakenly used twice. The duplicated `STOP_AT_DELIMITER` should be `SKIP_VALUE` in the documentation. ### Why are the changes needed? To correctly document. ### Does this PR introduce _any_ user-facing change? Yes, it fixes the user-facing documentation. ### How was this patch tested? I checked them via running linters. Closes #32423 from HyukjinKwon/SPARK-35250. Authored-by: HyukjinKwon <[email protected]> Signed-off-by: HyukjinKwon <[email protected]> (cherry picked from commit 8aaa9e8) Signed-off-by: HyukjinKwon <[email protected]>
apache · May 3, 2021 · 89f5ec7 · 89f5ec7
1 parent 25002e0
commit 89f5ec7
Show file tree

Hide file tree

Showing 4 changed files with 4 additions and 4 deletions.
diff --git a/python/pyspark/sql/readwriter.py b/python/pyspark/sql/readwriter.py
@@ -701,7 +701,7 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non
             * ``STOP_AT_DELIMITER``: If unescaped quotes are found in the input, consider the value
               as an unquoted value. This will make the parser accumulate all characters until the
               delimiter or a line ending is found in the input.
-            * ``STOP_AT_DELIMITER``: If unescaped quotes are found in the input, the content parsed
+            * ``SKIP_VALUE``: If unescaped quotes are found in the input, the content parsed
               for the given value will be skipped and the value set in nullValue will be produced
               instead.
             * ``RAISE_ERROR``: If unescaped quotes are found in the input, a TextParsingException

diff --git a/python/pyspark/sql/streaming.py b/python/pyspark/sql/streaming.py
@@ -915,7 +915,7 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non
             * ``STOP_AT_DELIMITER``: If unescaped quotes are found in the input, consider the value
               as an unquoted value. This will make the parser accumulate all characters until the
               delimiter or a line ending is found in the input.
-            * ``STOP_AT_DELIMITER``: If unescaped quotes are found in the input, the content parsed
+            * ``SKIP_VALUE``: If unescaped quotes are found in the input, the content parsed
               for the given value will be skipped and the value set in nullValue will be produced
               instead.
             * ``RAISE_ERROR``: If unescaped quotes are found in the input, a TextParsingException

diff --git a/sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala b/sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala
@@ -743,7 +743,7 @@ class DataFrameReader private[sql](sparkSession: SparkSession) extends Logging {
    *     <li>`STOP_AT_DELIMITER`: If unescaped quotes are found in the input, consider the value
    *     as an unquoted value. This will make the parser accumulate all characters until the
    *     delimiter or a line ending is found in the input.</li>
-   *     <li>`STOP_AT_DELIMITER`: If unescaped quotes are found in the input, the content parsed
+   *     <li>`SKIP_VALUE`: If unescaped quotes are found in the input, the content parsed
    *     for the given value will be skipped and the value set in nullValue will be produced
    *     instead.</li>
    *     <li>`RAISE_ERROR`: If unescaped quotes are found in the input, a TextParsingException

diff --git a/sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala b/sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala
@@ -412,7 +412,7 @@ final class DataStreamReader private[sql](sparkSession: SparkSession) extends Lo
    *     <li>`STOP_AT_DELIMITER`: If unescaped quotes are found in the input, consider the value
    *     as an unquoted value. This will make the parser accumulate all characters until the
    *     delimiter or a line ending is found in the input.</li>
-   *     <li>`STOP_AT_DELIMITER`: If unescaped quotes are found in the input, the content parsed
+   *     <li>`SKIP_VALUE`: If unescaped quotes are found in the input, the content parsed
    *     for the given value will be skipped and the value set in nullValue will be produced
    *     instead.</li>
    *     <li>`RAISE_ERROR`: If unescaped quotes are found in the input, a TextParsingException