Skip to content

Commit

Permalink
downsample: ensure consistent order
Browse files Browse the repository at this point in the history
Ensure that we have a consistent order of blocks that we are going to
downsample. `range` over maps doesn't enforce any particular order on
purpose.

This is needed for https://github.com/thanos-io/thanos/pull/3031/files.
ATM in that PR before downsampling we delete all directories which do
not match blocks ULIDs in the remote object storage. Ideally, we should
only keep around the files of a block which we are about to downsample.

It is impossible to do that properly ATM if during another iteration
we'd start from a different block. Thus, let's have a consistent order.

Signed-off-by: Giedrius Statkevičius <[email protected]>
  • Loading branch information
GiedriusS committed Feb 26, 2021
1 parent 0d86697 commit 6597d4e
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion cmd/thanos/downsample.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"context"
"os"
"path/filepath"
"sort"
"time"

"github.com/go-kit/kit/log"
Expand Down Expand Up @@ -194,7 +195,17 @@ func downsampleBucket(
}
}

for _, m := range metas {
metasULIDS := make([]ulid.ULID, 0, len(metas))
for k := range metas {
metasULIDS = append(metasULIDS, k)
}
sort.Slice(metasULIDS, func(i, j int) bool {
return metasULIDS[i].Compare(metasULIDS[j]) < 0
})

for _, mk := range metasULIDS {
m := metas[mk]

switch m.Thanos.Downsample.Resolution {
case downsample.ResLevel0:
missing := false
Expand Down

0 comments on commit 6597d4e

Please sign in to comment.