Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize logging pending task summary #44658

Closed
Bukhtawar opened this issue Jul 21, 2019 · 2 comments
Closed

Optimize logging pending task summary #44658

Bukhtawar opened this issue Jul 21, 2019 · 2 comments
Labels
:Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. help wanted adoptme >non-issue

Comments

@Bukhtawar
Copy link
Contributor

The below allocation hotspot was observed in a 6.2 cluster where there was a flood of shard-failed events fixed in 6.4 as a part of #31313. Although this might not recur on higher versions for shard-failed specifically but still can if there are too many pending tasks but since the below code seems to be used only for logging task summary in debug mode or warning if it turned out to be a slow running task, it seems wasteful for most of the cases.

I think it would make more sense to possibly defer this.

default String describeTasks(List<T> tasks) {
return String.join(", ", tasks.stream().map(t -> (CharSequence)t.toString()).filter(t -> t.length() > 0)::iterator);
}

  [ 1] org.elasticsearch.cluster.ClusterStateTaskExecutor.lambda$describeTasks$0
  [ 2] org.elasticsearch.cluster.ClusterStateTaskExecutor$$Lambda$1757.792151473.apply
  [ 3] java.util.stream.ReduceOps$2ReducingSink.accept
  [ 4] java.util.stream.ReferencePipeline$3$1.accept
  [ 5] java.util.ArrayList$ArrayListSpliterator.forEachRemaining
  [ 6] java.util.stream.AbstractPipeline.copyInto
  [ 7] java.util.stream.AbstractPipeline.wrapAndCopyInto
  [ 8] java.util.stream.ReduceOps$ReduceOp.evaluateSequential
  [ 9] java.util.stream.AbstractPipeline.evaluate
  [10] java.util.stream.ReferencePipeline.reduce
  [11] org.elasticsearch.cluster.ClusterStateTaskExecutor.describeTasks
  [12] org.elasticsearch.cluster.service.MasterService$Batcher$UpdateTask.describeTasks
  [13] org.elasticsearch.cluster.service.TaskBatcher.lambda$runIfNotProcessed$6
  [14] org.elasticsearch.cluster.service.TaskBatcher$$Lambda$1752.1418423337.apply
  [15] java.util.stream.ReferencePipeline$3$1.accept
  [16] java.util.HashMap$EntrySpliterator.forEachRemaining
  [17] java.util.stream.AbstractPipeline.copyInto
  [18] java.util.stream.AbstractPipeline.wrapAndCopyInto
  [19] java.util.stream.ReduceOps$ReduceOp.evaluateSequential
  [20] java.util.stream.AbstractPipeline.evaluate
  [21] java.util.stream.ReferencePipeline.reduce
  [22] org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed
  [23] org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run
  [24] org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run
@tlrx tlrx added the :Distributed/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) label Jul 22, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@DaveCTurner DaveCTurner added :Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. and removed :Distributed/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) labels Jul 24, 2019
@DaveCTurner
Copy link
Contributor

We haven't seen any interest in this since it was opened, so I am closing it as an indication that it's not something we expect to work on in future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. help wanted adoptme >non-issue
Projects
None yet
Development

No branches or pull requests

4 participants