Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky query frontend split interval unit test #3110

Merged
merged 1 commit into from
Sep 2, 2020

Conversation

yeya24
Copy link
Contributor

@yeya24 yeya24 commented Sep 1, 2020

Signed-off-by: Ben Ye [email protected]

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

Fix #3084

In the previous test case, I used counter to mock HTTP handler and check how many times the handler is being called.

func counter() (*int, http.Handler) {
	count := 0
	var lock sync.Mutex
	return &count, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		lock.Lock()
		defer lock.Unlock()
		count++
	})
}

But this mock handler doesn't return a meaningful result to query frontend so it might cause other requests canceled at the same time, which causes the flaky. https://github.com/cortexproject/cortex/blob/master/pkg/querier/queryrange/limits.go#L104

So if we return promql response which is valid for query frontend, then the test case should work.

Verification

Used the function below to test it works as expected.

func TestFlaky(t *testing.T) {
	testRequest := &ThanosRequest{
		Path:  "/api/v1/query_range",
		Start: 0,
		End:   2 * hour,
		Step:  10 * seconds,
	}

	codec := NewThanosCodec(true)
	tpw, err := NewTripperWare(&fakeLimits{}, nil, codec, nil,
		time.Hour, 0, nil, log.NewNopLogger())
	testutil.Ok(t, err)

	rt, err := newFakeRoundTripper()
	testutil.Ok(t, err)
	defer rt.Close()

	for i := 0; i < 10000; i++ {
		c, handler := promqlResults(false)
		rt.setHandler(handler)

		ctx := user.InjectOrgID(context.Background(), "1")
		httpReq, err := codec.EncodeRequest(ctx, testRequest)
		testutil.Ok(t, err)

		_, err = tpw(rt).RoundTrip(httpReq)
		testutil.Ok(t, err)

		testutil.Equals(t, 2, *c)
	}
}

Copy link
Member

@kakkoyun kakkoyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. Good work, thanks for jumping on it.

@bwplotka bwplotka merged commit 4bf2447 into thanos-io:master Sep 2, 2020
@bwplotka
Copy link
Member

bwplotka commented Sep 2, 2020

Thanks!

@yeya24 yeya24 deleted the fix-flaky-ut branch September 2, 2020 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flaky QueryFrontend Unit Test
3 participants