enhance: load deltalogs on demand when doing compactions

[tedxu]

See #37234

[mergify]

@tedxu E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

[codecov]

Codecov Report

Attention: Patch coverage is 81.31868% with 17 lines in your changes missing coverage. Please review.

Project coverage is 80.89%. Comparing base (90948e9) to head (93bac9f).
Report is 26 commits behind head on master.

Files with missing lines	Patch %	Lines
...ternal/datanode/compaction/clustering_compactor.go	35.71%	6 Missing and 3 partials ⚠️
internal/datanode/compaction/mix_compactor.go	83.87%	3 Missing and 2 partials ⚠️
internal/datanode/compaction/merge_sort.go	88.88%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #37310      +/-   ##
==========================================
- Coverage   83.11%   80.89%   -2.23%     
==========================================
  Files        1029     1321     +292     
  Lines      157231   183062   +25831     
==========================================
+ Hits       130689   148095   +17406     
- Misses      21379    29780    +8401     
- Partials     5163     5187      +24     

Components	Coverage Δ
Client	∅ <ø> (∅)
Core	67.16% <ø> (∅)
Go	83.13% <78.84%> (-0.01%)	⬇️

Files with missing lines	Coverage Δ
internal/datanode/compaction/compactor_common.go	84.40% <100.00%> (-0.25%)	⬇️
internal/datanode/compaction/merge_sort.go	79.13% <88.88%> (+5.60%)	⬆️
internal/datanode/compaction/mix_compactor.go	77.29% <83.87%> (+0.71%)	⬆️
...ternal/datanode/compaction/clustering_compactor.go	67.14% <35.71%> (-0.39%)	⬇️

... and 337 files with indirect coverage changes

[tedxu]

/run-cpu-e2e

[mergify]

@tedxu E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

[tedxu]

/run-cpu-e2e

[XuanYang-cn]

/lgtm

[congqixia]

/approve

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: congqixia, tedxu, XuanYang-cn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• internal/datanode/OWNERS [congqixia]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment