Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add EventuallyWithT assertion #1264

Merged

Conversation

tobikris
Copy link
Contributor

@tobikris tobikris commented Sep 13, 2022

Summary

This adds a version of Eventually that allows to do assertions in the condition.

Changes

  • Added EventuallyWithT assertion.
  • Added CollectT helper struct.

Motivation

The current Eventually only returns one summary error in case of a failure. This is not very helpful if several conditions need to be matched in the condition function. This version collects all failures of a tick and replays the last tick's failures after the timeout.
Thus the last iteration's errors can be shown to the user.

Example Usage

externalValue := false
go func() {
	time.Sleep(8*time.Second)
	externalValue = true
}()
assert.EventuallyWithTf(t, func(c *assert.CollectT) {
	// add assertions as needed; any assertion failure will fail the current tick
	assert.True(c, externalValue, "expected 'externalValue' to be true")
}, 1*time.Second, 10*time.Second, "external state has not changed to 'true'; still: %v", externalState)

Related issues

Closes #902

arikkfir and others added 4 commits October 20, 2022 08:25
This change updates the "EventuallyWithT" assertion variants (regular, formatted,
requirement) to consider a condition as "met" if no assertion errors were raised
in a tick.

This allows to write easier conditions which simply contain assertions, without
needing to return a bool. The equivalent of a condition returning true in the
previous implementation would be a a condition with a single "assert.True(..)" call.
Change "EventuallyWithT" condition acceptance to no-errors raised
@mrIncompetent
Copy link

@boyan-soubachov if you have a minute 🙏

@l-abels
Copy link

l-abels commented Apr 14, 2023

I implemented something similar about a year ago, this looks like a good solution. What needs to happen to get this merged? I'm running up against this use case again and would love to have this available.

Copy link
Collaborator

@boyan-soubachov boyan-soubachov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

@boyan-soubachov boyan-soubachov merged commit 4b2f4d2 into stretchr:master May 9, 2023
@tobikris tobikris deleted the eventually-with-assertions branch May 23, 2023 18:28
@dolmen
Copy link
Collaborator

dolmen commented Jul 22, 2023

Methods Reset and Copy of CollectT should have been made private.

CollectT should not have been exposed directly (this is an implementation detail) but only an subset of the testing.TB interface in the signature of the condition argument of EventuallyWithT.

But all this is now part of the public API. :(

@dolmen
Copy link
Collaborator

dolmen commented Jul 22, 2023

I can't understand how adding such a feature has gone into a patch release of testify. Does this gives us (maintainers) the permission to also fix it in a patch release?

Usage in the wild on GitHub: https://github.com/search?q=content%3Aassert.EventuallyWithT+language%3Ago+NOT+user%3Astretchr+NOT+path%3Aassert%2F+NOT+path%3Arequire%2F&type=code

Cc: @brackendawson

@brackendawson
Copy link
Collaborator

Probably the minor version should have been bumped when this was released. Fixing it should be a patch unless we chnage the API. I think we should do the right things, perhaps this discussion deserves an issue to tackle release higyne? I'm also somewhat interested in some automated testing before releases using a lot of real modules that import testify, there have been some regressions in recent releases.

@dolmen dolmen added the assert.Eventually About assert.Eventually/EventuallyWithT label Jul 31, 2023
algitbot pushed a commit to alpinelinux/build-server-status that referenced this pull request May 6, 2024
This MR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [github.com/stretchr/testify](https://github.com/stretchr/testify) | require | minor | `v1.8.0` -> `v1.9.0` |

---

### Release Notes

<details>
<summary>stretchr/testify (github.com/stretchr/testify)</summary>

### [`v1.9.0`](https://github.com/stretchr/testify/releases/tag/v1.9.0)

[Compare Source](stretchr/testify@v1.8.4...v1.9.0)

#### What's Changed

-   Fix Go modules version by [@&#8203;SuperQ](https://github.com/SuperQ) in stretchr/testify#1394
-   Document that require is not safe to call in created goroutines by [@&#8203;programmer04](https://github.com/programmer04) in stretchr/testify#1392
-   Remove myself from MAINTAINERS.md by [@&#8203;mvdkleijn](https://github.com/mvdkleijn) in stretchr/testify#1367
-   Correct spelling/grammar by [@&#8203;echarrod](https://github.com/echarrod) in stretchr/testify#1389
-   docs: Update URLs in README by [@&#8203;davidjb](https://github.com/davidjb) in stretchr/testify#1349
-   Update mockery link to Github Pages in README by [@&#8203;LandonTClipp](https://github.com/LandonTClipp) in stretchr/testify#1346
-   docs: Fix typos in tests and comments by [@&#8203;alexandear](https://github.com/alexandear) in stretchr/testify#1410
-   CI: tests from go1.17 by [@&#8203;SuperQ](https://github.com/SuperQ) in stretchr/testify#1409
-   Fix adding ? when no values passed by [@&#8203;lesichkovm](https://github.com/lesichkovm) in stretchr/testify#1320
-   codegen: use standard header for generated files by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1406
-   mock: AssertExpectations log reason only on failure by [@&#8203;hikyaru-suzuki](https://github.com/hikyaru-suzuki) in stretchr/testify#1360
-   assert: fix flaky TestNeverTrue by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1417
-   README: fix typos "set up" vs "setup" by [@&#8203;ossan-dev](https://github.com/ossan-dev) in stretchr/testify#1428
-   mock: move regexp compilation outside of `Called` by [@&#8203;aud10slave](https://github.com/aud10slave) in stretchr/testify#631
-   assert: refactor internal func getLen() by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1445
-   mock: deprecate type AnythingOfTypeArgument ([#&#8203;1434](stretchr/testify#1434)) by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1441
-   Remove no longer needed assert.canConvert by [@&#8203;alexandear](https://github.com/alexandear) in stretchr/testify#1470
-   assert: ObjectsAreEqual: use time.Equal for time.Time types by [@&#8203;tscales](https://github.com/tscales) in stretchr/testify#1464
-   Bump actions/checkout from 3 to 4 by [@&#8203;dependabot](https://github.com/dependabot) in stretchr/testify#1466
-   Bump actions/setup-go from 3.2.0 to 4.1.0 by [@&#8203;dependabot](https://github.com/dependabot) in stretchr/testify#1451
-   fix: make EventuallyWithT concurrency safe by [@&#8203;czeslavo](https://github.com/czeslavo) in stretchr/testify#1395
-   assert: fix httpCode and HTTPBody occur panic when http.Handler read Body by [@&#8203;hidu](https://github.com/hidu) in stretchr/testify#1484
-   assert.EqualExportedValues: fix handling of arrays by [@&#8203;zrbecker](https://github.com/zrbecker) in stretchr/testify#1473
-   .github: use latest Go versions by [@&#8203;kevinburkesegment](https://github.com/kevinburkesegment) in stretchr/testify#1489
-   assert: Deprecate EqualExportedValues by [@&#8203;HaraldNordgren](https://github.com/HaraldNordgren) in stretchr/testify#1488
-   suite: refactor test assertions by [@&#8203;alexandear](https://github.com/alexandear) in stretchr/testify#1474
-   suite: fix SetupSubTest and TearDownSubTest execution order by [@&#8203;linusbarth](https://github.com/linusbarth) in stretchr/testify#1471
-   docs: Fix deprecation comments for http package by [@&#8203;alexandear](https://github.com/alexandear) in stretchr/testify#1335
-   Add map support doc comments to Subset and NotSubset by [@&#8203;jedevc](https://github.com/jedevc) in stretchr/testify#1306
-   TestErrorIs/TestNotErrorIs: check error message contents by [@&#8203;craig65535](https://github.com/craig65535) in stretchr/testify#1435
-   suite: fix subtest names (fix [#&#8203;1501](stretchr/testify#1501)) by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1504
-   assert: improve unsafe.Pointer tests by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1505
-   assert: simplify isNil implementation by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1506
-   assert.InEpsilonSlice: fix expected/actual order and other improvements by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1483
-   Fix dependency cycle with objx [#&#8203;1292](stretchr/testify#1292) by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1453
-   mock: refactor TestIsArgsEqual by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1444
-   mock: optimize argument matching checks by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1416
-   assert: fix TestEventuallyTimeout by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1412
-   CI: add go 1.21 in GitHub Actions by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1450
-   suite: fix recoverAndFailOnPanic to report test failure at the right location by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1502
-   Update maintainers by [@&#8203;brackendawson](https://github.com/brackendawson) in stretchr/testify#1533
-   assert: Fix EqualValues to handle overflow/underflow by [@&#8203;arjunmahishi](https://github.com/arjunmahishi) in stretchr/testify#1531
-   assert: better formatting for Len() error by [@&#8203;kevinburkesegment](https://github.com/kevinburkesegment) in stretchr/testify#1485
-   Ensure AssertExpectations does not fail in skipped tests by [@&#8203;ianrose14](https://github.com/ianrose14) in stretchr/testify#1331
-   suite: fix deadlock in suite.Require()/Assert() by [@&#8203;arjunmahishi](https://github.com/arjunmahishi) in stretchr/testify#1535
-   Revert "assert: ObjectsAreEqual: use time.Equal for time.Time type" by [@&#8203;brackendawson](https://github.com/brackendawson) in stretchr/testify#1537
-   \[chore] Add issue templates by [@&#8203;arjunmahishi](https://github.com/arjunmahishi) in stretchr/testify#1538
-   Update the build status badge by [@&#8203;brackendawson](https://github.com/brackendawson) in stretchr/testify#1540
-   Update Github workflows setup-go to V5 by [@&#8203;hendrywiranto](https://github.com/hendrywiranto) in stretchr/testify#1545
-   Support Pointer to Struct in EqualExportedValues by [@&#8203;Lucaber](https://github.com/Lucaber) in stretchr/testify#1517
-   README: drop link to gorc by [@&#8203;guettli](https://github.com/guettli) in stretchr/testify#1248
-   http_assertions: honour the msgAndArgs provided with each assertion by [@&#8203;arjunmahishi](https://github.com/arjunmahishi) in stretchr/testify#1548
-   fix typos in comments and tests by [@&#8203;ccoVeille](https://github.com/ccoVeille) in stretchr/testify#1247
-   Include the auto-release notes in releases by [@&#8203;brackendawson](https://github.com/brackendawson) in stretchr/testify#1550
-   Add `NotImplements` and variants by [@&#8203;hslatman](https://github.com/hslatman) in stretchr/testify#1385
-   Add support to compare uintptr by [@&#8203;bogdandrutu](https://github.com/bogdandrutu) in stretchr/testify#1339
-   build(deps): bump github.com/stretchr/objx from 0.5.1 to 0.5.2 by [@&#8203;dependabot](https://github.com/dependabot) in stretchr/testify#1552

#### New Contributors

-   [@&#8203;SuperQ](https://github.com/SuperQ) made their first contribution in stretchr/testify#1394
-   [@&#8203;programmer04](https://github.com/programmer04) made their first contribution in stretchr/testify#1392
-   [@&#8203;echarrod](https://github.com/echarrod) made their first contribution in stretchr/testify#1389
-   [@&#8203;davidjb](https://github.com/davidjb) made their first contribution in stretchr/testify#1349
-   [@&#8203;LandonTClipp](https://github.com/LandonTClipp) made their first contribution in stretchr/testify#1346
-   [@&#8203;alexandear](https://github.com/alexandear) made their first contribution in stretchr/testify#1410
-   [@&#8203;lesichkovm](https://github.com/lesichkovm) made their first contribution in stretchr/testify#1320
-   [@&#8203;dolmen](https://github.com/dolmen) made their first contribution in stretchr/testify#1406
-   [@&#8203;hikyaru-suzuki](https://github.com/hikyaru-suzuki) made their first contribution in stretchr/testify#1360
-   [@&#8203;ossan-dev](https://github.com/ossan-dev) made their first contribution in stretchr/testify#1428
-   [@&#8203;aud10slave](https://github.com/aud10slave) made their first contribution in stretchr/testify#631
-   [@&#8203;tscales](https://github.com/tscales) made their first contribution in stretchr/testify#1464
-   [@&#8203;czeslavo](https://github.com/czeslavo) made their first contribution in stretchr/testify#1395
-   [@&#8203;hidu](https://github.com/hidu) made their first contribution in stretchr/testify#1484
-   [@&#8203;zrbecker](https://github.com/zrbecker) made their first contribution in stretchr/testify#1473
-   [@&#8203;kevinburkesegment](https://github.com/kevinburkesegment) made their first contribution in stretchr/testify#1489
-   [@&#8203;linusbarth](https://github.com/linusbarth) made their first contribution in stretchr/testify#1471
-   [@&#8203;jedevc](https://github.com/jedevc) made their first contribution in stretchr/testify#1306
-   [@&#8203;craig65535](https://github.com/craig65535) made their first contribution in stretchr/testify#1435
-   [@&#8203;arjunmahishi](https://github.com/arjunmahishi) made their first contribution in stretchr/testify#1531
-   [@&#8203;ianrose14](https://github.com/ianrose14) made their first contribution in stretchr/testify#1331
-   [@&#8203;hendrywiranto](https://github.com/hendrywiranto) made their first contribution in stretchr/testify#1545
-   [@&#8203;Lucaber](https://github.com/Lucaber) made their first contribution in stretchr/testify#1517
-   [@&#8203;guettli](https://github.com/guettli) made their first contribution in stretchr/testify#1248
-   [@&#8203;ccoVeille](https://github.com/ccoVeille) made their first contribution in stretchr/testify#1247
-   [@&#8203;hslatman](https://github.com/hslatman) made their first contribution in stretchr/testify#1385
-   [@&#8203;bogdandrutu](https://github.com/bogdandrutu) made their first contribution in stretchr/testify#1339

**Full Changelog**: stretchr/testify@v1.8.4...v1.9.0

### [`v1.8.4`](https://github.com/stretchr/testify/releases/tag/v1.8.4)

[Compare Source](stretchr/testify@v1.8.3...v1.8.4)

#### What's Changed

-   Create GitHub release when new release tag is pushed by [@&#8203;aldas](https://github.com/aldas) in stretchr/testify#1354

#### New Contributors

-   [@&#8203;aldas](https://github.com/aldas) made their first contribution in stretchr/testify#1354

**Full Changelog**: stretchr/testify@v1.8.3...v1.8.4

### [`v1.8.3`](https://github.com/stretchr/testify/releases/tag/v1.8.3)

[Compare Source](stretchr/testify@v1.8.2...v1.8.3)

#### What's Changed

-   Compare public elements of struct by [@&#8203;mchlp](https://github.com/mchlp) in stretchr/testify#1309
-   assert: fix error message formatting for NotContains by [@&#8203;wwade](https://github.com/wwade) in stretchr/testify#1362
-   allow testing for functional options by [@&#8203;nbaztec](https://github.com/nbaztec) in stretchr/testify#1023
-   add EventuallyWithT assertion by [@&#8203;tobikris](https://github.com/tobikris) in stretchr/testify#1264
-   EqualExportedValues: Handle nested pointer, slice and map fields by [@&#8203;HaraldNordgren](https://github.com/HaraldNordgren) in stretchr/testify#1379

#### New Contributors

-   [@&#8203;mchlp](https://github.com/mchlp) made their first contribution in stretchr/testify#1309
-   [@&#8203;wwade](https://github.com/wwade) made their first contribution in stretchr/testify#1362
-   [@&#8203;nbaztec](https://github.com/nbaztec) made their first contribution in stretchr/testify#1023
-   [@&#8203;tobikris](https://github.com/tobikris) made their first contribution in stretchr/testify#1264

**Full Changelog**: stretchr/testify@v1.8.2...v1.8.3

### [`v1.8.2`](https://github.com/stretchr/testify/releases/tag/v1.8.2)

[Compare Source](stretchr/testify@v1.8.1...v1.8.2)

#### What's Changed

-   Add opportunity to trigger setup/teardown for subtest by [@&#8203;qerdcv](https://github.com/qerdcv) in stretchr/testify#1246
-   fix: fix bug for check unsafe.Pointer isNil by [@&#8203;sunpe](https://github.com/sunpe) in stretchr/testify#1319
-   Fix Call.Unset() panic (issue [#&#8203;1236](stretchr/testify#1236)) by [@&#8203;lisitsky](https://github.com/lisitsky) in stretchr/testify#1250
-   Fix `CallerInfo()` source file paths by [@&#8203;bozaro](https://github.com/bozaro) in stretchr/testify#1288
-   assert: Fix Subset/NotSubset when map is missing keys from the subset by [@&#8203;danielwhite](https://github.com/danielwhite) in stretchr/testify#1261

#### New Contributors

-   [@&#8203;qerdcv](https://github.com/qerdcv) made their first contribution in stretchr/testify#1246
-   [@&#8203;sunpe](https://github.com/sunpe) made their first contribution in stretchr/testify#1319
-   [@&#8203;lisitsky](https://github.com/lisitsky) made their first contribution in stretchr/testify#1250
-   [@&#8203;bozaro](https://github.com/bozaro) made their first contribution in stretchr/testify#1288
-   [@&#8203;danielwhite](https://github.com/danielwhite) made their first contribution in stretchr/testify#1261

**Full Changelog**: stretchr/testify@v1.8.1...v1.8.2

### [`v1.8.1`](https://github.com/stretchr/testify/releases/tag/v1.8.1)

[Compare Source](stretchr/testify@v1.8.0...v1.8.1)

#### What's Changed

-   Bump github.com/stretchr/objx from 0.4.0 to 0.5.0 by [@&#8203;dependabot](https://github.com/dependabot) in stretchr/testify#1283

**Full Changelog**: stretchr/testify@v1.8.0...v1.8.1

</details>

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

&nbsp;
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yODYuMSIsInVwZGF0ZWRJblZlciI6IjM3LjI4Ni4xIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIiwibGFiZWxzIjpbXX0=-->

See merge request alpine/infra/build-server-status!12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
assert.Eventually About assert.Eventually/EventuallyWithT
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Better alternative to Eventually()
7 participants