Compactor: Make debug.max-compaction-level Public

[Jakob3xD]

Is your proposal related to a problem?

One of my minio servers has a compactor running with a retention set to 30 Days for all resolution.
Currently the compactor creates blocks with a duration of 14 days. This can leads to an storage issue as only every two weeks blocks get removed by moving out of the retention time frame. More over the downsampler creates high storage peaks while creating the 5m and 1h blocks out of the 14d raw block.

Describe the solution you'd like

My wish is to make the hidden flag debug.max-compaction-level a public one with a small documentation entry with why this can be set and the downsides if there are some.

Additional context

Description of the following graph:
It displays the last 8 weeks.
The high peaks are created by the downsampler creating the blocks of 5m and 1h retention out of the raw 14d blocks and deleting the old 2d blocks. In two weeks the storage increase by ~3TB and drops by the end for around 2.9TB. Limiting the duration to 2d will increase the block drop ration and resulting in a more flatted graph and a better storage usage.

[Jakob3xD]

There is already a hidden flag for my request. Setting debug.max-compaction-level to 3 will end at 2d blocks.

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.