Skip to content

This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.

License

Notifications You must be signed in to change notification settings

NashTech-Labs/real-time-stream-processing-engine

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

#real-time-stream-processing-engine

This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.

alt tag

#Pre-Requisites for this project ####Elasticsearch Setup i) Download the Elasticsearch 5.0.0-alpha5 or latest version and unzip it.

ii) Run the following command.

    $ bin/elasticsearch

####Kafka Setup i) Download the Kafka-0.10.0.1 or latest version and unzip it.

ii) Run the following commands for starting Zookeeper & Kafka:

    $ bin/zookeeper-server-start.sh config/zookeeper.properties 
    $ bin/kafka-server-start.sh config/server.properties

###Getting Started:

Clone and run in local mode:

    $ [email protected]:knoldus/sentiment-analysis-engine.git
    $ cd sentiment-analysis-engine
    $ bin/activator run

##Limitations

  1. Elasticsearch 5.0.0-alpha5 is not production ready.

  2. Kafka-0.10.0.1 is experimental in current spark version(spark-2.0.0).

About

This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published