Skip to content
This repository has been archived by the owner on Jan 9, 2020. It is now read-only.

Specify hostpath volume and mount the volume in Spark driver and executor pods #614

Open
sumansomasundar opened this issue Feb 13, 2018 · 1 comment

Comments

@sumansomasundar
Copy link

In some cases (for example, log aggregation) it is desirable to have a hostpath volume mounted in the driver and executor pods. With #38 this will be possible, but I am not sure when it will be available.

So, something similar to #397 will be helpful for this case as well.

What I am proposing is an option to specify the hostpath volume and have it mounted in the driver and executor pods.

@sumansomasundar
Copy link
Author

I have made changes that support the following configuration for spark-submit command:
--conf spark.kubernetes.driver.hostpath.<hostpath_dir>=<container_mount_path>
If this approach sounds good, I can create a pull request for this change.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant