Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-1332] Improve Spark Streaming's Network Receiver and InputDStream API [WIP] #300

Closed
wants to merge 12 commits into from
Closed

[SPARK-1332] Improve Spark Streaming's Network Receiver and InputDStream API [WIP] #300

wants to merge 12 commits into from

Commits on Apr 1, 2014

  1. Refactored the NetworkReceiver API for future stability.

    @tdas
    tdas committed Apr 1, 2014
    Configuration menu
    Copy the full SHA
    a36cc48 View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2014

  1. Refactored the code that runs the NetworkReceiver into further classe… 

    …s and traits to make them more testable.
    @tdas
    tdas committed Apr 2, 2014
    Configuration menu
    Copy the full SHA
    3223e95 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2014

  1. Merge remote-tracking branch 'apache-github/master' into network-rece… 

    …iver-api

Conflicts:
	external/mqtt/src/main/scala/org/apache/spark/streaming/mqtt/MQTTInputDStream.scala
	streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala
	streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
	streaming/src/main/scala/org/apache/spark/streaming/scheduler/NetworkInputTracker.scala
	streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
    @tdas
    tdas committed Apr 15, 2014
    Configuration menu
    Copy the full SHA
    9e37a0b View commit details
    Browse the repository at this point in the history

  2. Fixed graceful shutdown by removing interrupts on receiving thread.

    @tdas
    tdas committed Apr 15, 2014
    Configuration menu
    Copy the full SHA
    2c94579 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2014

  1. Made functions that create input streams return InputDStream and Netw… 

    …orkInputDStream, for both Scala and Java.
    @tdas
    tdas committed Apr 16, 2014
    Configuration menu
    Copy the full SHA
    43f5290 View commit details
    Browse the repository at this point in the history

  2. Further refactored receiver to allow restarting of a receiver.

    @tdas
    tdas committed Apr 16, 2014
    Configuration menu
    Copy the full SHA
    028bde6 View commit details
    Browse the repository at this point in the history

  3. Scala style fixes.

    @tdas
    tdas committed Apr 16, 2014
    Configuration menu
    Copy the full SHA
    8533094 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2014

  1. Fixed bugs.

    @tdas
    tdas committed Apr 17, 2014
    Configuration menu
    Copy the full SHA
    91bfa72 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2014

  1. Address some PR comments and fixed other issues.

    @tdas
    tdas committed Apr 18, 2014
    Configuration menu
    Copy the full SHA
    a75c7a6 View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2014

  1. Added more events to the StreamingListener to report errors and stopp… 

    …ed receivers.
    @tdas
    tdas committed Apr 19, 2014
    Configuration menu
    Copy the full SHA
    838dd39 View commit details
    Browse the repository at this point in the history

  2. Renamed NetworkInputDStream to ReceiverInputDStream, and ActorReceive… 

    …r related stuff.
    @tdas
    tdas committed Apr 19, 2014
    Configuration menu
    Copy the full SHA
    3a4777c View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2014

  1. Merge remote-tracking branch 'apache-github/master' into network-rece… 

    …iver-api
    @tdas
    tdas committed Apr 21, 2014
    Configuration menu
    Copy the full SHA
    ea27b38 View commit details
    Browse the repository at this point in the history