Skip to content

Commit

Permalink
[SPARK-35143][SQL][SHELL] Add default log level config for spark-sql
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?
Add default log config for spark-sql

### Why are the changes needed?
The default log level for spark-sql is `WARN`. How to change the log level is confusing, we need a default config.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Change config `log4j.logger.org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver=INFO` in log4j.properties, then spark-sql's default log level changed.

Closes #32248 from hddong/spark-35413.

Lead-authored-by: hongdongdong <[email protected]>
Co-authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: hyukjinkwon <[email protected]>
  • Loading branch information
hddong and HyukjinKwon committed Apr 23, 2021
1 parent cab205e commit 7582dc8
Showing 1 changed file with 5 additions and 3 deletions.
8 changes: 5 additions & 3 deletions conf/log4j.properties.template
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,12 @@ log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n

# Set the default spark-shell log level to WARN. When running the spark-shell, the
# log level for this class is used to overwrite the root logger's log level, so that
# the user can have different defaults for the shell and regular Spark apps.
# Set the default spark-shell/spark-sql log level to WARN. When running the
# spark-shell/spark-sql, the log level for these classes is used to overwrite
# the root logger's log level, so that the user can have different defaults
# for the shell and regular Spark apps.
log4j.logger.org.apache.spark.repl.Main=WARN
log4j.logger.org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver=WARN

# Settings to quiet third party logs that are too verbose
log4j.logger.org.sparkproject.jetty=WARN
Expand Down

0 comments on commit 7582dc8

Please sign in to comment.