0.1.6.0-opensource-beta1

This is a beta of functionality with open source spark. It currently only creates Spark / Scala Kernels

0.1.4.5 DSE Spark Kernel for Scala and Python

Changes:

1. Add Databricks CSV Support
2. Allow statements to span lines

To install the kernels

1. Ensure DSE is in your path
2. type setup.sh [<spark master IP>] - The spark master IP is optional as the default is 127.0.0.1

Configure spark parameters using spark's spark-defaults.conf.

DSE Spark Kernel for Scala and Python

To install the kernels

1. Ensure DSE is in your path
2. type setup.sh [<spark master IP>] - The spark master IP is optional as the default is 127.0.0.1

Configure spark parameters using spark's spark-defaults.conf.

DSE Spark Kernel for Scala and Python

0.1.4.3-cassandra

Update README.md

DSE Support

Some cool new features:

1. DSE cluster support. This will pull in all of the necessary DSE jars.
2. %%showschema magic

Setting up DSE Cluster support example. Full documentation to follow. See prior release documentation for in instructions on setting up the kernel.json file.

kernel.json file. This includes an example of overriding spark.executor.memory

{
    "display_name": "DSE Spark (Cluster) 1.4.1 (Scala 2.10.4)",
    "language": "scala",
    "argv": [
        "/users/stevelowenthal/repos/spark-kernel/kernel/target/pack/bin/sparkkernel-dse",
        "--profile",
        "{connection_file}",
        "--spark-configuration",
       "spark.cassandra.connection.host=127.0.0.1",
        "--spark-configuration",
       "spark.executor.memory=2g",
       "-master",
       "spark://127.0.0.1:7077"
     ],
     "codemirror_mode": "scala"
}

Jupyter notebook spark-kernel with spark 1.4 and Cassandra support

First release of the iPython notebook spark-kernel with Cassandra support

To get jupyter notebook

Obviously you need python. Install these python packages

pip install jupyter

To set it up:

unpack the zip file just a bit below

create the directory

~/.ipython/kernels/spark

create the file

~/.ipython/kernels/spark/kernel.json

and paste in the following contents: Note you need to update a path to sparkkernel

{
    "display_name": "Spark-Cassandra (Scala 2.10.4)",
    "language": "scala",
    "argv": [
        "/<path>/<to>/spark-kernel/bin/sparkkernel",
        "--profile",
        "{connection_file}",
     ],
     "codemirror_mode": "scala"
}

If you nee to override the connection host, add these lines to the argv map above

   "--spark-configuration",
   "spark.cassandra.connection.host=127.0.0.1"

To run it

jupyter notebook

In the browser - create a new spark notebook

... and spark away

If you don't get output, try adding a .toString on the end. There seems to be a bug rendering some types.
0.1.4-cassandra
Fixed output formatting issue

Running CQL Statements from within the notebook

Simply prefix a cell containing a CQL statement with %%Cql

%%Cql select * from system.local

Jupyter notebook spark-kernel with Cassandra support

First release of the iPython notebook spark-kernel with Cassandra support

To get jupyter notebook

Obviously you need python. Install these python packages

pip install jupyter

To set it up:

unpack the zip file just a bit below

create the directory

~/.ipython/kernels/spark

create the file

~/.ipython/kernels/spark/kernel.json

and paste in the following contents: Note you need to update a path to sparkkernel

{
    "display_name": "Spark 1.2.1 (Scala 2.10.4)",
    "language": "scala",
    "argv": [
        "/<path>/<to>/spark-kernel/bin/sparkkernel",
        "--profile",
        "{connection_file}",
     ],
     "codemirror_mode": "scala"
}

If you nee to override the connection host, add these lines to the argv map above

   "--spark-configuration",
   "spark.cassandra.connection.host=127.0.0.1"

To run it

jupyter notebook

In the browser - create a new spark notebook

... and spark away

If you don't get output, try adding a .toString on the end. There seems to be a bug rendering some types.
0.1.4-cassandra
Fixed output formatting issue

Running CQL Statements from within the notebook

Simply prefix a cell containing a CQL statement with %%Cql

%%Cql select * from system.local

Jupyter notebook spark-kernel with Cassandra support

First release of the iPython notebook spark-kernel with Cassandra support

To get jupyter notebook

Obviously you need python. Install these python packages

pip install jupyter

To set it up:

unpack the zip file just a bit below

create the directory

~/.ipython/kernels/spark

create the file

~/.ipython/kernels/spark/kernel.json

and paste in the following contents: Note you need to update a path to sparkkernel

{
    "display_name": "Spark 1.2.1 (Scala 2.10.4)",
    "language": "scala",
    "argv": [
        "/<path>/<to>/spark-kernel/bin/sparkkernel",
        "--profile",
        "{connection_file}",
     ],
     "codemirror_mode": "scala"
}

If you nee to override the connection host, add these lines to the argv map above

   "--spark-configuration",
   "spark.cassandra.connection.host=127.0.0.1"

To run it

jupyter notebook

In the browser - create a new spark notebook

... and spark away

If you don't get output, try adding a .toString on the end. There seems to be a bug rendering some types.

Running CQL Statements from within the notebook

Simply prefix a cell containing a CQL statement with %%Cql

%%Cql select * from system.local

First release of the iPython notebook spark-kernel with Cassandra support

First release of the iPython notebook spark-kernel with Cassandra support

To get ipython notebook

Obviously you need python. Install these python packages

pip install ipython
pip install notebook

To set it up:

unpack the zip file

create the directory

~/.ipython/kernels/spark

create the file

~/.ipython/kernels/spark/kernel.json

and paste in the following contents: Note you need to update a path to sparkkernel

{
    "display_name": "Spark 1.2.1 (Scala 2.10.4)",
    "language": "scala",
    "argv": [
        "/<path>/<to>/spark-kernel/bin/sparkkernel",
        "--profile",
        "{connection_file}",
     ],
     "codemirror_mode": "scala"
}

If you nee to override the connection host, add these lines to the argv map above

   "--spark-configuration",
   "spark.cassandra.connection.host=127.0.0.1"

To run it

ipython notebook

In the browser - create a new spark notebook

... and spark away

If you don't get output, try adding a .toString on the end. There seems to be a bug rendering some types.