enhance: avoid memory copy and serde in mix compaction

[tedxu]

See: #37234

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: tedxu
To complete the pull request process, please assign wxyucs after the PR has been reviewed.
You can assign the PR to them by writing /assign @wxyucs in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• internal/datanode/OWNERS
• internal/storage/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tedxu]

In this PR, the serialization / deserialization operations and memory copies are avoided in mix compaction. From which, the mix compaction performance is improved significantly. See benchmark results below:

Before:

BenchmarkMixCompactor-8   	       1	1509607958 ns/op	1605694208 B/op	13207679 allocs/op

After

BenchmarkMixCompactor-8             3	   472317972 ns/op	924115029 B/op	  7417982 allocs/op

[mergify]

@tedxu E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

[codecov]

Codecov Report

Attention: Patch coverage is 69.81132% with 64 lines in your changes missing coverage. Please review.

Project coverage is 82.56%. Comparing base (f813fb4) to head (b5b66db).
Report is 9 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/datanode/compaction/segment_writer.go	56.60%	18 Missing and 5 partials ⚠️
internal/storage/serde.go	60.00%	20 Missing and 2 partials ⚠️
internal/datanode/compaction/mix_compactor.go	73.58%	11 Missing and 3 partials ⚠️
internal/storage/serde_events.go	90.19%	3 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #37479       +/-   ##
===========================================
+ Coverage   67.31%   82.56%   +15.24%     
===========================================
  Files         290     1043      +753     
  Lines       25377   161159   +135782     
===========================================
+ Hits        17082   133056   +115974     
- Misses       8295    22841    +14546     
- Partials        0     5262     +5262     

Components	Coverage Δ
Client	52.03% <ø> (∅)
Core	∅ <ø> (∅)
Go	83.20% <69.81%> (∅)

Files with missing lines	Coverage Δ
internal/storage/serde_events.go	75.46% <90.19%> (ø)
internal/datanode/compaction/mix_compactor.go	76.42% <73.58%> (ø)
internal/storage/serde.go	85.49% <60.00%> (ø)
internal/datanode/compaction/segment_writer.go	74.04% <56.60%> (ø)

... and 1329 files with indirect coverage changes

[XuanYang-cn]

/lgtm

[sre-ci-robot]

New changes are detected. LGTM label has been removed.

[mergify]

@tedxu cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

[mergify]

@tedxu E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

[tedxu]

rerun cpp-unit-test

[tedxu]

/run-cpu-e2e

[mergify]

@tedxu E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.