v23.08.1

Packages

• Maven Release: https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark-tools_2.12/23.08.1/
• PyPI Package: https://pypi.org/project/spark-rapids-user-tools/23.08.1/

Changes

User Tools

• [DOC] Fix help command in documentation (#540)
• Implement a cross-CSP storage driver (#485)
• Build tools package as single artifact for restricted environments (#516)

Core

• Remove memoryOverhead recommendations for Standalone Spark (#557)
• [FEA] Add support to TIMESTAMP functions (#549)
• Fix handling of current_database and ArrayBuffer (#556)
• Add translate as supported expression in qualification tools (#546)
• Adding TakeOrderedAndProject and BroadcastNestedLoopJoin, removing Project from speedup generation (#548)
• Qualification should treat promote_precision as supported (#545)
• Improve tool error message for files with text extensions (#544)
• Improve parsing of aggregate expressions (#535)
• Bump default build to use Spark-333 (#537)
• Improve AutoTuner plugin recommendation for Fat mode (#543)
• Updating speedup generation for more execs from NDS + validation script (#530)
• [FEA] Reset speedup factors for qualification tool in EMR 6.12 environments (#529)
• Add min, median and max columns to AccumProfileResults (#522)
• [FEA] Reset speedup factors for qualification tool in Databricks 12.2 environments (#524)
• Filter parser should check ignored-functions (#520)
• Update speedup factors for qualification tool in Dataproc 2.1 environments (#509)

Miscellaneous

• Changing max_value to total based on profiler core changes (#555)
• Add platform encoding to plugins defined in pom (#526)

v23.08.0

Packages

• Maven Release: https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark-tools_2.12/23.08.0/
• PyPI Package: https://pypi.org/project/spark-rapids-user-tools/23.08.0/

Changes

User Tools

• Support offline execution of user tools in restricted environments (#497)
• Handle deprecation errors in python packaging (#513)
• Adds profiling support for EMR in user tools. (#500)

Core

• Fix unit-tests for Spark-340 and Add spark-versions to gh-workflow (#503)

Miscellaneous

• fix gh-workflow for Python unit-tests (#505)
• Refactoring the speedup factor generation to support WholeStageCodegen parsing and environment defaults (#493)
• Try fix push issue in release action [skip ci] (#495)
• Revert "Push to protected branch using third-party action (#492)" (#494)
• Push to protected branch using third-party action (#492)
• Add secrets in the release.yml (#491)
• Add sign-off and token in release workflow (#490)

v23.06.4

Packages

• Maven Release: https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark-tools_2.12/23.06.4/
• PyPI Package: https://pypi.org/project/spark-rapids-user-tools/23.06.4/

Changes

User Tools

• Creating custom speedup factors README with generation script (#488)
• Bump dev-version to 23.06.4 (#468)

Core

• [FEA] Enhance qualification tool to handle custom speedup factor file as input (#475)
• Bump dev-version to 23.06.4 (#468)

Miscellaneous

• Add Github workflow to automate changelog and release pages (#486)
• Fix negative GPU speedup in the core qualification module (#487)
• Verify the integrity the downloaded dependencies (#470)

v23.06.3

New minor release 23.06.3:

• mvn jar file: https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark-tools_2.12/23.06.3/
• pypi package: https://pypi.org/project/spark-rapids-user-tools/

Changes:

1. Bump dev version to 23.06.3 (#442) (githubweb)
2. Qualification command should run without cluster-info (#444) (githubweb)
3. Fix mvn warning message for formatting string (#446) ([githubweb)
4. Add Recommendations in AutoTuner for Databricks platform (#440) (githubweb)
5. Update the minimum spark version requirements to run tools (#452) (githubweb)
6. Generate speedup factors for A10 instance types in EMR (#450) (githubweb)
7. Qualification tool: Add ExistenceJoin to supported jointype (#432) (githubweb)
8. Bump certifi from 2022.12.7 to 2023.7.22 (#460) (githubweb)
9. Add Progress Bar in Profiling tools (#456) (githubweb)
10. [FEA] Include bootstrap recommended configs in qualification output (#451) (githubweb)
11. Status Reporting for Event Logs and Applications in Qualification Tool (#464) (githubweb)
12. Support wildcards in eventlogs arguments (#465) (githubweb)
13. Refactor arguments to handle options after event logs (#466) (githubweb)
14. Update version by jenkins-spark-rapids-tools-auto-release-30 (githubweb)

v23.06.2

New minor release 23.06.2:

• mvn jar file: https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark-tools_2.12/23.06.2/
• pypi package: https://pypi.org/project/spark-rapids-user-tools/

Changes:

1. Qualification tool to report with granularity on unsupported operators (#401) (githubweb)
2. Bug fix for supported expressions (#408) (githubweb)
3. [DOC] Fix documentation and consistency in arguments for the EMR user-tools (#413) (githubweb)
4. Remove serverless execution mode from the EMR-wrapper (#421) (githubweb)
5. Remove cluster-mode execution wrapper (#424) (githubweb)
6. [TASK] Implement classes and methods to support user tools profiling command for Databricks Azure (#423) (githubweb)
7. Add platform argument in profiling tool for custom tuning based on the platform (#414) (githubweb)
8. [DOC] Add documentation for running user tools profiling command for Databricks Azure (#430) (githubweb)
9. update diagnostic unit-tests to support var directory (#429) (githubweb)
10. Fix broken tables and code blocks in profiling-tool docs (#435) (githubweb)
11. Support onprem platform for user-tools profiling command (#431) (githubweb)
12. Add ascli as a new simplified tools command (#426) (details
13. update more details for databricks azure user qualification tool doc (#438) (githubweb)
14. Bump build version to 23.06.2 (#439) (details

v23.02.0 rapids-4-spark-tools jar release

Release of Qualification and Profiling tools jar.

You can find the download URL here: https://repo.maven.apache.org/maven2/com/nvidia/rapids-4-spark-tools_2.12/23.02.0/

v22.10.2 release

• Support to handle tools jar arguments in the user tools wrapper

v22.10.1 release

• Init the project