Skip to content
This repository has been archived by the owner on May 23, 2024. It is now read-only.

Do not apply adaptive sampler to child spans #457

Merged

Conversation

yurishkuro
Copy link
Member

Which problem is this PR solving?

Per-operation adaptive sampling should only apply to service endpoints, not to operation names of the inner/children spans. Otherwise it's a problem for adaptive sampling backend that only records operations from the root spans, meaning all child operations are getting the default probability, which may be higher than the calculated probabilities.

Short description of the changes

  • Add a test that demonstrates the problem.
  • Change per-operation sampler to only allow sampling "local root spans", which is equivalent to the actual root span of the trace because spans with remote parent are finalized right away.

Signed-off-by: Yuri Shkuro <[email protected]>
@codecov
Copy link

codecov bot commented Oct 22, 2019

Codecov Report

Merging #457 into master will increase coverage by 0.34%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #457      +/-   ##
==========================================
+ Coverage   87.39%   87.74%   +0.34%     
==========================================
  Files          59       59              
  Lines        3476     3484       +8     
==========================================
+ Hits         3038     3057      +19     
+ Misses        325      315      -10     
+ Partials      113      112       -1
Impacted Files Coverage Δ
tracer.go 95.85% <100%> (+0.03%) ⬆️
sampler.go 91.57% <100%> (+2.33%) ⬆️
span_context.go 93.46% <100%> (+2.73%) ⬆️
span.go 97.44% <0%> (+1.53%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 790ac7d...64b1b27. Read the comment docs.

sampler.go Outdated
return SamplingDecision{Sample: sampled, Retryable: true, Tags: tags}
}

// OnSetOperationName implements OnSetOperationName of SamplerV2.
func (s *AdaptiveSampler) OnSetOperationName(span *Span, operationName string) SamplingDecision {
samplerV1 := s.getSamplerForOperation(operationName)
sampled, tags := samplerV1.IsSampled(span.context.TraceID(), operationName)
sampled := false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't the logic be:
if root:
isSampled()
else
inherent from context?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not the business of this sampler to "inherit from context", that would be done by the tracer when appropriate. If this sampler is called, it needs to make a decision based on the data it has.

sampler.go Outdated
@@ -365,14 +365,22 @@ func (s *AdaptiveSampler) IsSampled(id TraceID, operation string) (bool, []Tag)
func (s *AdaptiveSampler) OnCreateSpan(span *Span) SamplingDecision {
operationName := span.OperationName()
samplerV1 := s.getSamplerForOperation(operationName)
sampled, tags := samplerV1.IsSampled(span.context.TraceID(), operationName)
sampled := false

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small nit: seems like some copy pasta here could be put into single method

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@@ -290,6 +293,7 @@ func (t *Tracer) startSpanWithOptions(
ctx.samplingState = parent.samplingState
if parent.remote {
ctx.samplingState.setFinal()
ctx.samplingState.localRootSpan = ctx.spanID

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would this do? I thought for remote, it's just final and there-fore no samplers should be called?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for consistency of localRootSpan property the assignment makes sense, but doesn't affect anything today.

DRY
Signed-off-by: Yuri Shkuro <[email protected]>
Copy link

@andreynkravtsov andreynkravtsov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

sampler.go Outdated
@@ -361,18 +361,23 @@ func (s *AdaptiveSampler) IsSampled(id TraceID, operation string) (bool, []Tag)
return false, nil
}

func (s *AdaptiveSampler) trySampling(span *Span, operationName string) (sampled bool, tags []Tag) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I don't see these named return values frequently in Jaeger. Why use them here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is technically idiomatic Go, makes the code shorter, and the signature self-documenting.

But I can change to vars.

Signed-off-by: Yuri Shkuro <[email protected]>
@yurishkuro yurishkuro merged commit 8fad375 into jaegertracing:master Oct 28, 2019
@yurishkuro yurishkuro deleted the no-adaptive-sampling-on-child-spans branch October 28, 2019 17:24
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants