new feature: Explore if OpenDAL can support KvikIO (aka Nvidia GPUDirect Storage)

[Xuanwo]

Feature Description

GPUDirect Storage

Nvidia GPUDirect Storage is a technology introduced by Nvidia that enables GPUs to read files directly.

With GPUDirect Storage:

• Higher Bandwidth: GDS offers 2x-8x higher bandwidth by enabling direct data transfers between storage and GPU, bypassing CPU involvement.
• Lower Latency: Explicit and direct transfers reduce latency by 3.8x and ensure stable latency even as GPU concurrency increases.
• Reduced CPU Load: DMA engines near storage reduce CPU load, maintaining higher bandwidth with minimal CPU utilization, and GPU utilization remains near zero during data transfers.
• Superior IO Bandwidth: GPUs achieve higher IO bandwidth (215 GB/s) compared to CPUs (50 GB/s).
• Scalable Storage Access: GDS allows fast access to large data sets, whether stored locally or remotely, nearly saturating GPU bandwidth by combining data transfers from various sources (CPU memory, local storage, remote storage).
• Efficient Data Caching: Large distributed data sets can be cached in local storage, and working tables can be cached in CPU memory for collaborative use with the CPU, enhancing overall system performance.

cuFile

Nvidia exposes GDS support via cuFile which included in CUDA toolkit.

KvikIO

KvikIO is a High Performance File IO lib.

KvikIO is a Python and C++ library for high performance file IO. It provides C++ and Python bindings to cuFile, which enables GPUDirect Storage (GDS). KvikIO also works efficiently when GDS isn't available and can read/write both host and device data seamlessly. The C++ library is header-only making it easy to include in existing projects.

Problem and Solution

Perhaps OpenDAL could implement a service based on kvikio, allowing our users to interact with GDS using the familiar OpenDAL API.

kvikio currently offers only C++ and Python APIs. Those interested should first create a Rust binding for kvikio. To explore the performance of GDS, one should also have the CUDA toolkit and a compatible graphics card.

Additional Context

No response

Are you willing to contribute to the development of this feature?

• Yes, I am willing to contribute to the development of this feature.

[yuchanns]

Interesting! What a shame I don't have a Nvidia GPU.

[Xuanwo]

Interesting! What a shame I don't have a Nvidia GPU.

AMD supports similar technology called DirectGMA, but I'm not sure if it can read files.

[morristai]

This is definitely an ambitious idea, and I would like to explore it further. However, after some research, I found that even if we create a Rust binding for KvikIO, it might not be sufficient.

The reason is that to utilize GDS, we need to use the CUDA API to manage GPU memory. Unfortunately, there is not yet a mature Rust library for CUDA. In KvikIO’s examples, to create a GPU memory buffer for KvikIO to read/write, they use APIs from cuda_runtime_api.h in C++ and CuPy in Python. These APIs are fully supported by Nvidia, and creating a Rust binding for them would be challenging.

If we use KvikIO without CUDA, as demonstrated in the example in the KvikIO repo (cpp/examples/basic_no_cuda.cpp), it won't use GPU memory at all. See the discussion in the issue for more details.

We could still try some third-party libraries like cudarc; maybe it will be sufficient for our use case. Perhaps others might have better ideas?