[SPARK-2886] Use more specific actor system name than "spark" #1810

andrewor14 · 2014-08-06T20:54:01Z

As of #1777 we log the name of the actor system when it binds to a port. The current name "spark" is super general and does not convey any meaning. For instance, the following line is taken from my driver log after setting spark.driver.port to 5001.

14/08/13 19:33:29 INFO Remoting: Remoting started; listening on addresses:
[akka.tcp://spark@andrews-mbp:5001]
14/08/13 19:33:29 INFO Remoting: Remoting now listens on addresses: 
[akka.tcp://spark@andrews-mbp:5001]
14/08/06 13:40:05 INFO Utils: Successfully started service 'spark' on port 5001.

This commit renames this to "sparkDriver" and "sparkExecutor". The goal of this unambitious PR is simply to make the logged information more explicit without introducing any change in functionality.

SparkQA · 2014-08-06T20:59:33Z

QA tests have started for PR 1810. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18048/consoleFull

SparkQA · 2014-08-06T21:11:00Z

QA results for PR 1810:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18048/consoleFull

andrewor14 · 2014-08-06T22:25:00Z

test this please

SparkQA · 2014-08-06T22:29:27Z

QA tests have started for PR 1810. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18065/consoleFull

andrewor14 · 2014-08-06T23:42:38Z

Previous test seems frozen. Jenkins, retest this please

SparkQA · 2014-08-06T23:44:39Z

QA tests have started for PR 1810. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18079/consoleFull

Hopefully this allows the tests to run succesfully...

andrewor14 · 2014-08-08T02:31:02Z

test this please

SparkQA · 2014-08-08T02:34:37Z

QA tests have started for PR 1810. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18175/consoleFull

pwendell · 2014-08-09T07:50:59Z

Jenkins, test this please.

SparkQA · 2014-08-09T07:54:46Z

QA tests have started for PR 1810. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18251/consoleFull

andrewor14 · 2014-08-09T19:04:58Z

Build always times out for some reason

andrewor14 · 2014-08-09T21:50:25Z

Ah, looks like it's not timing out on the build, but even the first test is hanging. I'll look more into why this is the case...

aarondav · 2014-08-10T00:02:21Z

core/src/main/scala/org/apache/spark/SparkEnv.scala

-
-    val (actorSystem, boundPort) = AkkaUtils.createActorSystem("spark", hostname, port, conf = conf,
-      securityManager = securityManager)
+    val actorSystemName = if (isDriver) "driverActor" else "executorActor"


Can we call this "driver actor system" / "executor actor system"? These are not individual actors, but whole systems.

sounds good

Hm looks like elsewhere we call this sparkMaster and sparkWorker, so maybe I'll keep this consistent by calling these sparkDriver and sparkExecutor. Also the akka URL looks weird if we include "*ActorSystem" in it, e.g. akka.tcp://driverActorSystem@localhost:7077

Apparently we also hard-code the actor names everywhere, so the scope of this change is a little larger than expected.

andrewor14 · 2014-08-14T02:28:01Z

streaming/src/main/scala/org/apache/spark/streaming/receiver/ReceiverSupervisorImpl.scala

 import org.apache.spark.streaming.scheduler.RegisterReceiver
-import com.google.common.base.Throwables


just some import cleanups here

SparkQA · 2014-08-14T02:29:57Z

QA tests have started for PR 1810. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18513/consoleFull

andrewor14 · 2014-08-14T02:32:20Z

The newest changes renames the actor system to sparkDriver and sparkExecutor. The scope of the changes has grown a little, as it turns out we currently hard-code the actor system name in a few places. I've changed all the places where we do this to use a common variable.

SparkQA · 2014-08-14T03:22:17Z

QA results for PR 1810:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18513/consoleFull

mateiz · 2014-08-26T02:45:52Z

This looks good to me, Andrew -- feel free to merge it.

andrewor14 · 2014-08-26T06:36:41Z

Thanks @mateiz, I have merged this into master and 1.1.

As of #1777 we log the name of the actor system when it binds to a port. The current name "spark" is super general and does not convey any meaning. For instance, the following line is taken from my driver log after setting `spark.driver.port` to 5001. ``` 14/08/13 19:33:29 INFO Remoting: Remoting started; listening on addresses: [akka.tcp://sparkandrews-mbp:5001] 14/08/13 19:33:29 INFO Remoting: Remoting now listens on addresses: [akka.tcp://sparkandrews-mbp:5001] 14/08/06 13:40:05 INFO Utils: Successfully started service 'spark' on port 5001. ``` This commit renames this to "sparkDriver" and "sparkExecutor". The goal of this unambitious PR is simply to make the logged information more explicit without introducing any change in functionality. Author: Andrew Or <[email protected]> Closes #1810 from andrewor14/service-name and squashes the following commits: 8c459ed [Andrew Or] Use a common variable for driver/executor actor system names 3a92843 [Andrew Or] Change actor name to sparkDriver and sparkExecutor 921363e [Andrew Or] Merge branch 'master' of github.com:apache/spark into service-name c8c6a62 [Andrew Or] Do not include hyphens in actor name 1c1b42e [Andrew Or] Avoid spaces in akka system name f644b55 [Andrew Or] Use more specific service name (cherry picked from commit b21ae5b) Signed-off-by: Andrew Or <[email protected]>

As of apache#1777 we log the name of the actor system when it binds to a port. The current name "spark" is super general and does not convey any meaning. For instance, the following line is taken from my driver log after setting `spark.driver.port` to 5001. ``` 14/08/13 19:33:29 INFO Remoting: Remoting started; listening on addresses: [akka.tcp://sparkandrews-mbp:5001] 14/08/13 19:33:29 INFO Remoting: Remoting now listens on addresses: [akka.tcp://sparkandrews-mbp:5001] 14/08/06 13:40:05 INFO Utils: Successfully started service 'spark' on port 5001. ``` This commit renames this to "sparkDriver" and "sparkExecutor". The goal of this unambitious PR is simply to make the logged information more explicit without introducing any change in functionality. Author: Andrew Or <[email protected]> Closes apache#1810 from andrewor14/service-name and squashes the following commits: 8c459ed [Andrew Or] Use a common variable for driver/executor actor system names 3a92843 [Andrew Or] Change actor name to sparkDriver and sparkExecutor 921363e [Andrew Or] Merge branch 'master' of github.com:apache/spark into service-name c8c6a62 [Andrew Or] Do not include hyphens in actor name 1c1b42e [Andrew Or] Avoid spaces in akka system name f644b55 [Andrew Or] Use more specific service name

…ble default) (apache#1810) Co-authored-by: Russell Spitzer <[email protected]>

Use more specific service name

f644b55

Avoid spaces in akka system name

1c1b42e

Do not include hyphens in actor name

c8c6a62

Hopefully this allows the tests to run succesfully...

aarondav reviewed Aug 10, 2014
View reviewed changes

andrewor14 added 3 commits August 13, 2014 18:55

Merge branch 'master' of github.com:apache/spark into service-name

921363e

Change actor name to sparkDriver and sparkExecutor

3a92843

Apparently we also hard-code the actor names everywhere, so the scope of this change is a little larger than expected.

Use a common variable for driver/executor actor system names

8c459ed

andrewor14 reviewed Aug 14, 2014
View reviewed changes

asfgit closed this in b21ae5b Aug 26, 2014

andrewor14 deleted the service-name branch August 27, 2014 18:14

viirya pushed a commit to viirya/spark-1 that referenced this pull request Oct 19, 2023

rdar://111955881 (Differentiate Create Table Unordered from Create Ta…

2a5afba

…ble default) (apache#1810) Co-authored-by: Russell Spitzer <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARK-2886] Use more specific actor system name than "spark" #1810

[SPARK-2886] Use more specific actor system name than "spark" #1810

andrewor14 commented Aug 6, 2014

SparkQA commented Aug 6, 2014

SparkQA commented Aug 6, 2014

andrewor14 commented Aug 6, 2014

SparkQA commented Aug 6, 2014

andrewor14 commented Aug 6, 2014

SparkQA commented Aug 6, 2014

andrewor14 commented Aug 8, 2014

SparkQA commented Aug 8, 2014

pwendell commented Aug 9, 2014

SparkQA commented Aug 9, 2014

andrewor14 commented Aug 9, 2014

andrewor14 commented Aug 9, 2014

aarondav Aug 10, 2014

markhamstra Aug 10, 2014

andrewor14 Aug 10, 2014

andrewor14 Aug 14, 2014

andrewor14 Aug 14, 2014

SparkQA commented Aug 14, 2014

andrewor14 commented Aug 14, 2014

SparkQA commented Aug 14, 2014

mateiz commented Aug 26, 2014

andrewor14 commented Aug 26, 2014

		import org.apache.spark.streaming.scheduler.RegisterReceiver
		import com.google.common.base.Throwables

[SPARK-2886] Use more specific actor system name than "spark" #1810

[SPARK-2886] Use more specific actor system name than "spark" #1810

Conversation

andrewor14 commented Aug 6, 2014

SparkQA commented Aug 6, 2014

SparkQA commented Aug 6, 2014

andrewor14 commented Aug 6, 2014

SparkQA commented Aug 6, 2014

andrewor14 commented Aug 6, 2014

SparkQA commented Aug 6, 2014

andrewor14 commented Aug 8, 2014

SparkQA commented Aug 8, 2014

pwendell commented Aug 9, 2014

SparkQA commented Aug 9, 2014

andrewor14 commented Aug 9, 2014

andrewor14 commented Aug 9, 2014

aarondav Aug 10, 2014

Choose a reason for hiding this comment

markhamstra Aug 10, 2014

Choose a reason for hiding this comment

andrewor14 Aug 10, 2014

Choose a reason for hiding this comment

andrewor14 Aug 14, 2014

Choose a reason for hiding this comment

andrewor14 Aug 14, 2014

Choose a reason for hiding this comment

SparkQA commented Aug 14, 2014

andrewor14 commented Aug 14, 2014

SparkQA commented Aug 14, 2014

mateiz commented Aug 26, 2014

andrewor14 commented Aug 26, 2014