Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-3359][BUILD][DOCS] Print examples and disable group and tparam tags in javadoc #15939

Closed
wants to merge 2 commits into from

Conversation

HyukjinKwon
Copy link
Member

@HyukjinKwon HyukjinKwon commented Nov 19, 2016

What changes were proposed in this pull request?

This PR proposes/fixes two things.

  • Remove many errors to generate javadoc with Java8 from unrecognisable tags, @tparam and @group.

    [error] .../spark/mllib/target/java/org/apache/spark/ml/classification/Classifier.java:18: error: unknown tag: group
    [error]   /** @group setParam */
    [error]       ^
    [error] .../spark/mllib/target/java/org/apache/spark/ml/classification/Classifier.java:8: error: unknown tag: tparam
    [error]  * @tparam FeaturesType  Type of input features.  E.g., <code>Vector</code>
    [error]    ^
    ...
    

    It does not fully resolve the problem but remove many errors. It seems both @group and @tparam are unrecognisable in javadoc. It seems we can't print them pretty in javadoc in a way of @example here because they appear differently (both examples can be found in http://spark.apache.org/docs/2.0.2/api/scala/index.html#org.apache.spark.ml.classification.Classifier).

  • Print @example in javadoc.
    Currently, there are few @example tag in several places.

    ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * @example This operation might be used to evaluate a graph
    ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * @example We might use this operation to change the vertex values
    ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * @example This function might be used to initialize edge
    ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * @example This function might be used to initialize edge
    ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * @example This function might be used to initialize edge
    ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * @example We can use this function to compute the in-degree of each
    ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * @example This function is used to update the vertices with new values based on external data.
    ./graphx/src/main/scala/org/apache/spark/graphx/GraphLoader.scala:   * @example Loads a file in the following format:
    ./graphx/src/main/scala/org/apache/spark/graphx/GraphOps.scala:   * @example This function is used to update the vertices with new
    ./graphx/src/main/scala/org/apache/spark/graphx/GraphOps.scala:   * @example This function can be used to filter the graph based on some property, without
    ./graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala: * @example We can use the Pregel abstraction to implement PageRank:
    ./graphx/src/main/scala/org/apache/spark/graphx/VertexRDD.scala: * @example Construct a `VertexRDD` from a plain RDD:
    ./repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkCommandLine.scala: * @example new SparkCommandLine(Nil).settings
    ./repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkIMain.scala:   * @example addImports("org.apache.spark.SparkContext")
    ./sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala: * @example {{{
    

Before

2016-11-20 2 43 23

After
2016-11-20 1 27 17

How was this patch tested?

Maunally tested by jekyll build with Java 7 and 8

java version "1.7.0_80"
Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

Note: this does not make sbt unidoc suceed with Java 8 yet but it reduces the number of errors with Java 8.

@HyukjinKwon HyukjinKwon changed the title [SPARK-3359][BUILD][DOCS] Print examples in javadoc and disable group and tparam tags in javadoc [SPARK-3359][BUILD][DOCS] Print examples and disable group and tparam tags in javadoc Nov 19, 2016
@@ -742,7 +742,10 @@ object Unidoc {
"-windowtitle", "Spark " + version.value.replaceAll("-SNAPSHOT", "") + " JavaDoc",
"-public",
"-noqualifier", "java.lang",
"-tag", """note:a:Note\:"""
"-tag", """example:a:Example\:""",
"-tag", """note:a:Note\:""",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This precedence respects scaladoc.

2016-11-20 3 07 35

After this PR, javadoc looks like

2016-11-20 3 10 39

Copy link
Member

@srowen srowen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I hadn't seen this mechanism until you used it in the last PR. This is great, and probably the way we will finally get javadoc 8 to work. That's fantastic. Does this actually let sbt unidoc complete successfully on Java 8?

@SparkQA
Copy link

SparkQA commented Nov 19, 2016

Test build #68895 has finished for PR 15939 at commit 03f88d4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 19, 2016

Test build #68896 has finished for PR 15939 at commit 8514ec4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member Author

Oh, no. It seems still there are a lot of errors. I just meant to show what I have tried to test with .. :).
Thanks for approving!

@srowen
Copy link
Member

srowen commented Nov 20, 2016

Merged to master/2.1

@HyukjinKwon
Copy link
Member Author

Thank you Sean!

asfgit pushed a commit that referenced this pull request Nov 20, 2016
… tags in javadoc

## What changes were proposed in this pull request?

This PR proposes/fixes two things.

- Remove many errors to generate javadoc with Java8 from unrecognisable tags, `tparam` and `group`.

  ```
  [error] .../spark/mllib/target/java/org/apache/spark/ml/classification/Classifier.java:18: error: unknown tag: group
  [error]   /** group setParam */
  [error]       ^
  [error] .../spark/mllib/target/java/org/apache/spark/ml/classification/Classifier.java:8: error: unknown tag: tparam
  [error]  * tparam FeaturesType  Type of input features.  E.g., <code>Vector</code>
  [error]    ^
  ...
  ```

  It does not fully resolve the problem but remove many errors. It seems both `group` and `tparam` are unrecognisable in javadoc. It seems we can't print them pretty in javadoc in a way of `example` here because they appear differently (both examples can be found in http://spark.apache.org/docs/2.0.2/api/scala/index.html#org.apache.spark.ml.classification.Classifier).

- Print `example` in javadoc.
  Currently, there are few `example` tag in several places.

  ```
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example This operation might be used to evaluate a graph
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example We might use this operation to change the vertex values
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example This function might be used to initialize edge
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example This function might be used to initialize edge
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example This function might be used to initialize edge
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example We can use this function to compute the in-degree of each
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example This function is used to update the vertices with new values based on external data.
  ./graphx/src/main/scala/org/apache/spark/graphx/GraphLoader.scala:   * example Loads a file in the following format:
  ./graphx/src/main/scala/org/apache/spark/graphx/GraphOps.scala:   * example This function is used to update the vertices with new
  ./graphx/src/main/scala/org/apache/spark/graphx/GraphOps.scala:   * example This function can be used to filter the graph based on some property, without
  ./graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala: * example We can use the Pregel abstraction to implement PageRank:
  ./graphx/src/main/scala/org/apache/spark/graphx/VertexRDD.scala: * example Construct a `VertexRDD` from a plain RDD:
  ./repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkCommandLine.scala: * example new SparkCommandLine(Nil).settings
  ./repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkIMain.scala:   * example addImports("org.apache.spark.SparkContext")
  ./sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala: * example {{{
  ```

**Before**

  <img width="505" alt="2016-11-20 2 43 23" src="https://cloud.githubusercontent.com/assets/6477701/20457285/26f07e1c-aecb-11e6-9ae9-d9dee66845f4.png">

**After**
  <img width="499" alt="2016-11-20 1 27 17" src="https://cloud.githubusercontent.com/assets/6477701/20457240/409124e4-aeca-11e6-9a91-0ba514148b52.png">

## How was this patch tested?

Maunally tested by `jekyll build` with Java 7 and 8

```
java version "1.7.0_80"
Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
```

```
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
```

Note: this does not make sbt unidoc suceed with Java 8 yet but it reduces the number of errors with Java 8.

Author: hyukjinkwon <[email protected]>

Closes #15939 from HyukjinKwon/SPARK-3359-javadoc.

(cherry picked from commit c528812)
Signed-off-by: Sean Owen <[email protected]>
@asfgit asfgit closed this in c528812 Nov 20, 2016
uzadude pushed a commit to uzadude/spark that referenced this pull request Jan 27, 2017
… tags in javadoc

## What changes were proposed in this pull request?

This PR proposes/fixes two things.

- Remove many errors to generate javadoc with Java8 from unrecognisable tags, `tparam` and `group`.

  ```
  [error] .../spark/mllib/target/java/org/apache/spark/ml/classification/Classifier.java:18: error: unknown tag: group
  [error]   /** group setParam */
  [error]       ^
  [error] .../spark/mllib/target/java/org/apache/spark/ml/classification/Classifier.java:8: error: unknown tag: tparam
  [error]  * tparam FeaturesType  Type of input features.  E.g., <code>Vector</code>
  [error]    ^
  ...
  ```

  It does not fully resolve the problem but remove many errors. It seems both `group` and `tparam` are unrecognisable in javadoc. It seems we can't print them pretty in javadoc in a way of `example` here because they appear differently (both examples can be found in http://spark.apache.org/docs/2.0.2/api/scala/index.html#org.apache.spark.ml.classification.Classifier).

- Print `example` in javadoc.
  Currently, there are few `example` tag in several places.

  ```
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example This operation might be used to evaluate a graph
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example We might use this operation to change the vertex values
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example This function might be used to initialize edge
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example This function might be used to initialize edge
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example This function might be used to initialize edge
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example We can use this function to compute the in-degree of each
  ./graphx/src/main/scala/org/apache/spark/graphx/Graph.scala:   * example This function is used to update the vertices with new values based on external data.
  ./graphx/src/main/scala/org/apache/spark/graphx/GraphLoader.scala:   * example Loads a file in the following format:
  ./graphx/src/main/scala/org/apache/spark/graphx/GraphOps.scala:   * example This function is used to update the vertices with new
  ./graphx/src/main/scala/org/apache/spark/graphx/GraphOps.scala:   * example This function can be used to filter the graph based on some property, without
  ./graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala: * example We can use the Pregel abstraction to implement PageRank:
  ./graphx/src/main/scala/org/apache/spark/graphx/VertexRDD.scala: * example Construct a `VertexRDD` from a plain RDD:
  ./repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkCommandLine.scala: * example new SparkCommandLine(Nil).settings
  ./repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkIMain.scala:   * example addImports("org.apache.spark.SparkContext")
  ./sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralGenerator.scala: * example {{{
  ```

**Before**

  <img width="505" alt="2016-11-20 2 43 23" src="https://cloud.githubusercontent.com/assets/6477701/20457285/26f07e1c-aecb-11e6-9ae9-d9dee66845f4.png">

**After**
  <img width="499" alt="2016-11-20 1 27 17" src="https://cloud.githubusercontent.com/assets/6477701/20457240/409124e4-aeca-11e6-9a91-0ba514148b52.png">

## How was this patch tested?

Maunally tested by `jekyll build` with Java 7 and 8

```
java version "1.7.0_80"
Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
```

```
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
```

Note: this does not make sbt unidoc suceed with Java 8 yet but it reduces the number of errors with Java 8.

Author: hyukjinkwon <[email protected]>

Closes apache#15939 from HyukjinKwon/SPARK-3359-javadoc.
@HyukjinKwon HyukjinKwon deleted the SPARK-3359-javadoc branch January 2, 2018 03:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants