Skip to content

Commit

Permalink
Fix race condition on error returned after context cancellation (#12743)
Browse files Browse the repository at this point in the history
After #11981 docker client watching event uses the context provided by
the metricset wrapper. This produces a race condition on error
reporting: the errors channel will receive a context cancelled error when
the context is done, so both paths can be chosen by the select. If the
errors one is chosen, an error will be reported and a reconnect will be
attempted, that will fail.

Alternativelly we could have created another context and cancel it after
the reporter is done, in a similar fashion to what was done before #11981,
but then we would be breaking the chain of derived contexts.
  • Loading branch information
jsoriano authored Jul 10, 2019
1 parent 0ae766b commit d36878a
Showing 1 changed file with 10 additions and 2 deletions.
12 changes: 10 additions & 2 deletions metricbeat/module/docker/event/event.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,13 +94,21 @@ func (m *MetricSet) Run(ctx context.Context, reporter mb.ReporterV2) {
m.reportEvent(reporter, event)

case err := <-errors:
// An error can be received on context cancellation, don't reconnect
// if context is done.
select {
case <-ctx.Done():
m.logger.Debug("docker", "Event watcher stopped")
return
default:
}
// Restart watch call
m.logger.Error("Error watching for docker events: %v", err)
m.logger.Errorf("Error watching for docker events: %v", err)
time.Sleep(1 * time.Second)
break WATCH

case <-ctx.Done():
m.logger.Debug("docker", "event watcher stopped")
m.logger.Debug("docker", "Event watcher stopped")
return
}
}
Expand Down

0 comments on commit d36878a

Please sign in to comment.