lightning: optimize lightning memory and perf (#1020)

[ti-chi-bot]

This is an automated cherry-pick of #1020

What problem does this PR solve?

Optimize lightning performance with better memory allocation.

What is changed and how it works?

• Reuse the bytes buffer in session encode
• Avoid doing sort.Slice if local-writer's writeBatch is already sorted
• Pre-allocate a larger writeBatch to avoid slice reallocate too many times.

Benchmark result:
Each source file in the following two dataset are about 256MiB in size, and the target table contains 4 indexes(1 int primary key and 3 secondary keys).

branch	data size	cost
master(921e0da)	100GiB	16m48s
opt-local-writer	100GiB	13m3s
this pr	100GiB	11m37s
opt-local-writer	500GiB	1h2m
this pr	500GiB	56m40s

This PR is based on #753.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Code changes

Side effects

Related changes

• Need to cherry-pick to the release branch

Release Note

• Optimize local writer memory allocation.

[Little-Wallace]

Please fix conflict.
rest LGTM

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• Little-Wallace
• glorv

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[glorv]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 8ef9ed9