Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a stress testing methods for streams #26

Merged
merged 2 commits into from
Jan 10, 2016
Merged

Add a stress testing methods for streams #26

merged 2 commits into from
Jan 10, 2016

Conversation

marmbrus
Copy link
Owner

@marmbrus marmbrus commented Jan 8, 2016

No description provided.

tdas
tdas reviewed Jan 8, 2016
View reviewed changes
sql/core/src/test/scala/org/apache/spark/sql/streaming/MemorySourceStressSuite.scala
import scala.collection.mutable.ArrayBuffer
import scala.util.Random

class MemorySourceStressSuite extends StreamTest with SharedSQLContext {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldnt it be StreamExecutionStressSuite? So that its associated this suite with StreamExecution which is what this really intends to test

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea was that this could be used for any source (though that is implicitly also testing the stream execution).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well Kafka and Kinesis wont be in this testsuite. Probably only file would be in this.
Anyways this is just nits. Nvm.

@AmplabJenkins
Copy link

Merged build finished. Test FAILed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/spark-streaming-df-test/31/
Test FAILed.

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/spark-streaming-df-test/32/
Test PASSed.

marmbrus added a commit that referenced this pull request Jan 10, 2016
@marmbrus
Merge pull request #26 from marmbrus/stressTest
de4113f 
Add a stress testing methods for streams
@marmbrus marmbrus merged commit de4113f into streaming-df Jan 10, 2016
@marmbrus marmbrus deleted the stressTest branch March 8, 2016 00:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@marmbrus @AmplabJenkins @tdas