Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

*: Remove unnecessary configuration reload from ContentPathReloader and improve its tests #6496

Merged
merged 10 commits into from
Jul 5, 2023
Merged

*: Remove unnecessary configuration reload from ContentPathReloader and improve its tests #6496

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
273f714
Fix and improve PathContentReloader tests
douglascamata Jul 3, 2023
5b89be6
Run tests in parallel
douglascamata Jul 3, 2023
19c5d18
Add changelog entry
douglascamata Jul 3, 2023
7ee3542
Merge branch 'main' of github.com:thanos-io/thanos into improve-conte…
douglascamata Jul 3, 2023
097229e
Fix table tests
douglascamata Jul 3, 2023
9a5705d
Fix typo
douglascamata Jul 4, 2023
043bdcb
Merge branch 'main' of github.com:thanos-io/thanos into improve-conte…
douglascamata Jul 4, 2023
560578d
Merge branch 'improve-contentpathreloader-tests' of github.com:dougla…
douglascamata Jul 4, 2023
b7028c6
Rerun CI
douglascamata Jul 4, 2023
47d711b
Rerun CI
douglascamata Jul 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ We use *breaking :warning:* to mark changes that are not backward compatible (re
- [#6467](https://github.com/thanos-io/thanos/pull/6467) Mixin (Receive): add alert for tenant reaching head series limit.

### Fixed
- [#6496](https://github.com/thanos-io/thanos/pull/6496): *: Remove unnecessary configuration reload from `ContentPathReloader` and improve its tests.
- [#6456](https://github.com/thanos-io/thanos/pull/6456) Store: fix crash when computing set matches from regex pattern
- [#6427](https://github.com/thanos-io/thanos/pull/6427) Receive: increasing log level for failed uploads to error
- [#6172](https://github.com/thanos-io/thanos/pull/6172) query-frontend: return JSON formatted errors for invalid PromQL expression in the split by interval middleware.
Expand Down
1 change: 1 addition & 0 deletions pkg/extkingpin/path_content_reloader.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ func PathContentReloader(ctx context.Context, fileContent fileContent, logger lo
reloadFunc()
level.Debug(logger).Log("msg", "configuration reloaded after debouncing")
})
reloadTimer.Stop()
}
defer watcher.Close()
for {
Expand Down
10 changes: 9 additions & 1 deletion pkg/extkingpin/path_content_reloader_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,13 +57,15 @@ func TestPathContentReloader(t *testing.T) {
testutil.Ok(t, pathContent.Rewrite([]byte("test modified again")))
},
},
wantReloads: 1,
wantReloads: 2,
},
{
name: "Chmod doesn't trigger reload",
args: args{
runSteps: func(t *testing.T, testFile string, pathContent *staticPathContent) {
testutil.Ok(t, os.Chmod(testFile, 0777))
testutil.Ok(t, os.Chmod(testFile, 0666))
testutil.Ok(t, os.Chmod(testFile, 0777))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this trigger 2 reloads now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one triggers zero reload... see line 71 below.

The one above it triggers 2 reloads, which is truly what we expect, because the file is rewritten twice. Previously it could pass due to the code being much faster than the async fs watcher notification for the second rewrite of the file.

},
},
wantReloads: 0,
Expand All @@ -79,7 +81,9 @@ func TestPathContentReloader(t *testing.T) {
},
}
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
testFile := path.Join(t.TempDir(), "test")
testutil.Ok(t, os.WriteFile(testFile, []byte("test"), 0666))
pathContent, err := NewStaticPathContent(testFile)
Expand All @@ -98,6 +102,10 @@ func TestPathContentReloader(t *testing.T) {
testutil.Ok(t, err)

tt.args.runSteps(t, testFile, pathContent)
if tt.wantReloads == 0 {
// Give things a little time to async. The fs watcher events are heavily async and could be delayed.
douglascamata marked this conversation as resolved.
Show resolved Hide resolved
time.Sleep(1 * time.Second)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Crossposting from c6623f8#r120681785. Please check testutil.Equals(t, tt.wantReloads, reloadCount) in a loop instead or something. This time.Sleep will probably lead to flakiness.

Copy link
Contributor Author

@douglascamata douglascamata Jul 5, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GiedriusS I have some other follow ups to this part of the code and will include this suggested change for the next PR. 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this loop will have a timeout though, as I don't want a test failure to end up with an infinite loop or rely on test.timeout when running tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 you can use runutil.Repeat with a context to achieve that

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GiedriusS I actually have to keep the sleep there for one reason: a real scenario might have a delayed event from the fs and the test might pass if the assertion runs quickly before the event arrives... but would fail, as expected, if it had waited slightly more.

I'm adding the repeat there though for all the other scenarios.

}
wg.Wait()
testutil.Equals(t, tt.wantReloads, reloadCount)
})
Expand Down