Improve composing index-header by pulling only required pieces of index.

[bwplotka]

Orthogonal to sparse file proposal we may need to explore composing index-cache/index-header from parts of the index.

We do not compose the block only when:

• No index-cache.json is in the local filesystem
• No index-cache.json is in object storage.

Currently, we create index-cache by mmaping whole index file which might cause excessive resource usage, especially if we syncronize many blocks at the same time without locally cached objects.

The proposal is to do pull pieces of the index. To do so we need TOC which is at the end of the file as well as other pieces. This might mean changes in object store API: https://github.com/thanos-io/thanos/pull/1567/files#diff-c4e16588aa4d5a179f37f9a55308eb7eR245

If the process will be quick enough to do per request, we might move out of startup towards request time for this, improving startup time.

[bwplotka]

This is well suitable to be designed and done together with: #942

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.