Save beakerx.json to paths specified in JUPTYER_CONFIG_DIR

[vishnu2kmohan]

The %%spark magic is currently hardwired to store its configuration at Paths.get(System.getProperty("user.home") + File.separator + ".jupyter" + File.separator + "beakerx.json"))));

It should instead honor paths (if) specified in the JUPYTER_CONFIG_DIR environment variable

e.g., I have JUPYTER_CONFIG_DIR set to /mnt/mesos/sandbox/.jupyter which yields the following acceptable config locations with jupyter --paths:

$ jupyter --paths
config:
    /mnt/mesos/sandbox/.jupyter
    /opt/conda/etc/jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /mnt/mesos/sandbox/.local/share/jupyter
    /opt/conda/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /mnt/mesos/sandbox/.local/share/jupyter/runtime

Instead, the Spark Magic fails to initialize when I run the notebook as the non-privileged user nobody because it attempts to store beakerx.json under /nonexistent/.jupyter, which as the name implies, is non-existent :)

$ grep nobody /etc/passwd
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin

[scottdraves]

Thanks for the report, will get this into the next release.

[scottdraves]

the envar would be a good quick fix.
would it be more accurate to use Python API:
jupyter_core.paths.jupyter_config_dir()
?

[scottdraves]

beakerx/kernel/sparkex/src/main/java/com/twosigma/beakerx/widget/SparkUiDefaultsImpl.java

Line 43 in 8893d41

this.path = path;

should this class be in python, ie reuse

beakerx/beakerx/beakerx/environment.py

Line 48 in 8893d41

config_path = path.join(paths.jupyter_config_dir(), 'beakerx.json')

could call into it with py4j?