Skip to content

Spark 2.4.6

Latest
Latest
Compare
Choose a tag to compare
@paulovn paulovn released this 20 Nov 18:41
· 12 commits to master since this release
v2.2.2
7fc638c
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

v. 2.2.2

  • Spark is 2.4.6
  • Base OS Updated (R is now 3.6.2)
  • Updated some Python packages. PyLucene is now 8.1.1
  • Added package for Brotli codec (to use in Parquet files)
  • Updated Spark config
    • Use the optimized version of Hadoop FileOutputCommiter
    • Packages for Spark 2.4.6
    • Compatibiliy Setting for PyArrow >= 0.15.0
  • Refactored management scripts
    • Start jupyter service automatically at system startup
    • Wrap Jupyter actual launch to wait until the notebook directory (shared folder)
      is ready
Assets 2
Loading