lightning: reduce memory use (#847)

[ti-srebot]

cherry-pick #847 to release-4.0
You can switch your code base to this Pull Request by using git-extras:

# In br repo:
git pr https://github.com/pingcap/br/pull/866

After apply modifications, you can push your change to this PR via:

git push [email protected]:ti-srebot/br.git pr/866:release-4.0-01adaa0a1ece

---

What problem does this PR solve?

Lightning's Local Backend may OOM because some cache are too aggressive.

What is changed and how it works?

(Split off from #815).

1. Make the engine's mem table size configurable (keep default at 512 MiB)
2. Make the local writer's mem table size configurable (reduce default from 512 MiB to 128 MiB)
3. Reduced capacity of LocalWriter.kvsChan from 1024 to 16. Originally this will make Lightning retain 1024 × (region-conc) × (encoded-kvs) in the channel buffer, which can hold over 3 GiB.

Check List

Tests

• Unit test

Code changes

Side effects

Related changes

• Need to cherry-pick to the release branch

Release Note

• Added configurations tikv-importer.engine-mem-cache-size and tikv-importer.local-writer-mem-cache-size to tune between memory usage and performance.

[overvenus]

/lgtm

[lance6716]

/lgtm

[kennytm]

/merge

[ti-srebot]

/run-all-tests

[ti-srebot]

@ti-srebot merge failed.

[kennytm]

/merge

[2021-03-17T03:50:10.641Z] + echo 'Failed to start gcs-server'

[2021-03-17T03:50:10.641Z] Failed to start gcs-server

[kennytm]

/run-integration-test

[kennytm]

/run-integration-test

[2021-03-17T04:14:57.013Z] TEST: [br_full_ddl] fail due to stats are not equal