Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-33882][ML] Add a vectorized BLAS implementation #30810

Closed
wants to merge 20 commits into from

Commits on Dec 16, 2020

  1. [ML] Add a vectorized BLAS implementation

    Whenever a native BLAS implementation isn't available on the system, Spark
    automatically falls back onto a Java implementation. With the recent release
    of the Vector API in the OpenJDK [1], we can use hardware acceleration for such
    operations.
    
    This patch introduces a VectorizedBLAS class which implements such
    hardware-accelerated BLAS operations. This feature is hidden behind the
    "vectorized" profile that you can enable by passing "-Pvectorized" to sbt or
    maven.
    
    The Vector API has been introduced in JDK 16. Following discussion on the
    mailing list, this API is introduced transparently and needs to be enabled
    explicitely.
    
    [1] https://openjdk.java.net/jeps/338
    luhenry committed Dec 16, 2020
    Configuration menu
    Copy the full SHA
    48e4af1 View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2020

  1. Improve performance

    All implmentations beat f2j on microbenchmarks on x86 (w/ AVX-2).
    
    See https://github.com/luhenry/vectorizedblas/releases/tag/v0.1.5 for more details
    luhenry committed Dec 18, 2020
    Configuration menu
    Copy the full SHA
    c949d44 View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2020

  1. Use dev.ludovic.vectorizedblas package directly

    It simplifies the build process, and will allow for faster iterations on my end
    luhenry committed Dec 20, 2020
    Configuration menu
    Copy the full SHA
    bc9cfb0 View commit details
    Browse the repository at this point in the history
  2. Make sure we don't depend on spark-core in mllib-local

    We still use it for the benchmarks but it should go away at some point
    luhenry committed Dec 20, 2020
    Configuration menu
    Copy the full SHA
    494254b View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2b875e6 View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2020

  1. Configuration menu
    Copy the full SHA
    628af4f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7d6ac61 View commit details
    Browse the repository at this point in the history

Commits on Dec 22, 2020

  1. Fix indentation

    luhenry committed Dec 22, 2020
    Configuration menu
    Copy the full SHA
    53bf41b View commit details
    Browse the repository at this point in the history
  2. Extend benchmark coverage

    luhenry committed Dec 22, 2020
    Configuration menu
    Copy the full SHA
    0018109 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    d4b737c View commit details
    Browse the repository at this point in the history
  4. Add java 17 build to CI

    luhenry committed Dec 22, 2020
    Configuration menu
    Copy the full SHA
    3ac7e0e View commit details
    Browse the repository at this point in the history
  5. Bump dev.ludovic:vectorizedblas-blas to 0.1.9

    Brings acceleration for sscal, dgemm[N,N], dgemm[N,T], dgemv[N], sgemv[N] and sgemv[T]
    luhenry committed Dec 22, 2020
    Configuration menu
    Copy the full SHA
    0c52da5 View commit details
    Browse the repository at this point in the history
  6. Bring back VectorizedBLAS in the repo

    This reverts commit bc9cfb0.
    luhenry committed Dec 22, 2020
    Configuration menu
    Copy the full SHA
    e23a0fd View commit details
    Browse the repository at this point in the history
  7. Update according to review

    luhenry committed Dec 22, 2020
    Configuration menu
    Copy the full SHA
    18e62f9 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    1aea88a View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    43a82b1 View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2021

  1. Configuration menu
    Copy the full SHA
    14a0e60 View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2021

  1. Configuration menu
    Copy the full SHA
    521cba2 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2021

  1. Configuration menu
    Copy the full SHA
    d93b274 View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2021

  1. Configuration menu
    Copy the full SHA
    10595df View commit details
    Browse the repository at this point in the history