Possibility to not fail the test during retries

[samybenatt]

cute/roundtripper.go

Lines 103 to 108 in 9389b60

	return cuteErrors.NewAssertError(
	"Assert response code",
	fmt.Sprintf("Response code expect %v, but was %v", it.Expect.Code, resp.StatusCode),
	resp.StatusCode,
	it.Expect.Code)
	}

Would it be possible to have a way to configure this part to return optional error.

For example, here:

cute/test.go

Lines 56 to 58 in 9389b60

	Count int
	Delay time.Duration
	}

type RequestRepeatPolitic struct {
        Count int
        OptionalError bool
        Delay time.Duration
}

Add an OptionalError bool that, if defined to true, will return cuteErrors.NewOptionalError instead of cuteErrors.NewAssertError
If the count has reached its last index, then switch back cuteErrors.NewAssertError and ultimately fail the test

[siller174]

Hello.
It's possible. I think, I will do it together with #64

[siller174]

Or you can do it as you described.

[samybenatt]

Or you can do it as you described.

Sure, I just wanted to validate the approach with you first, let me create a pull request!

[samybenatt]

@siller174 #71
Here is the PR

[siller174]

Thank you! I will check PR

[siller174]

@samybenatt Sorry for the delay. Could you please review my implementation for your request?
https://github.com/ozontech/cute/pull/72/files

[samybenatt]

@samybenatt Sorry for the delay. Could you please review my implementation for your request? https://github.com/ozontech/cute/pull/72/files

Hi @siller174, thanks for taking time to implement this! I've sent very minor comments on the PR but overall lgtm!

[siller174]

Thank you again! Released https://github.com/ozontech/cute/releases/tag/v0.1.20

[samybenatt]

@siller174 Based on my latest comment I think I wrongly understood your implementation

I think there is still one thing missing:

• If Optional: true and all retries are failing: then mark the whole test as failing, else mark the test as success

Basically, in pseudo-code:
if currentTryCount < RequestPoliticRetryCount and retry.status = success:
test.status = success
else:
test.status = failure

[samybenatt]

@siller174
#72 (comment)
To complete the above comment in the PR

Here is a concrete example:

import (
	"context"
	"github.com/ozontech/cute/asserts/headers"
	"net/http"
	"testing"

	"github.com/ozontech/cute"
)

func Test_Table_Array(t *testing.T) {
	tests := []*cute.Test{
		{
			Name: "test_1",
			Request: &cute.Request{
				Repeat: &cute.RequestRepeatPolitic{Count: 4, Optional: true},
				Builders: []cute.RequestBuilder{
					cute.WithURI("https://httpstat.us/Random/203,200,201"),
					cute.WithMethod(http.MethodGet),
				},
			},
			Expect: &cute.Expect{
				Code:          200,
				AssertHeaders: []cute.AssertHeaders{headers.Present("Content-Type")},
			},
		},
		{
			Name:       "test_2",
			Middleware: nil,
			Request: &cute.Request{
				Repeat: &cute.RequestRepeatPolitic{Count: 2, Optional: true},
				Builders: []cute.RequestBuilder{
					cute.WithURI("https://httpstat.us/Random/201,202"),
					cute.WithMethod(http.MethodGet),
				},
			},
			Expect: &cute.Expect{
				Code: 200,
			},
		},
	}

	cute.NewTestBuilder().
		Title("Example table test").
		Tag("table_test").
		Description("Execute array tests").
		CreateTableTest().
		PutTests(tests...).
		ExecuteTest(context.Background(), t)
}

The above example tests will be:

• test_1 -> random HTTP code between 200/201/203
• test_2 -> random HTTP code between 201/202 (never 200)

Here is the result in the allure report and in the test run:

And in the allure report:

As you can see, for the test_1, my assert header is never executed while one of the retry was successful
For the test_2, the whole test is marked as success while all the retries fail up to the last retry