Skip to content
This repository has been archived by the owner on Jul 24, 2024. It is now read-only.

*: skip creating Domain for backup without stats (#876) #887

Merged
merged 2 commits into from
Mar 18, 2021

Conversation

ti-srebot
Copy link
Contributor

@ti-srebot ti-srebot commented Mar 17, 2021

cherry-pick #876 to release-5.0
You can switch your code base to this Pull Request by using git-extras:

# In br repo:
git pr https://github.com/pingcap/br/pull/887

After apply modifications, you can push your change to this PR via:

git push [email protected]:ti-srebot/br.git pr/887:release-5.0-de9666974ff3

What problem does this PR solve?

This PR reduces memory usage during backup by skipping initializing Domain which loads all table info into memory.

Close #818

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
Manual tests
  1. Load 40k 40 fields YCSB tables
for i in $(seq 1 40000); do { go-ycsb load mysql -p workload=core -p mysql.host=172.16.5.33 -p mysql.port=47904 -p mysql.user=root -p recordcount=20 -p fieldcount=40 -p fieldlength=16 -p threadcount=4 -p table=usertable$i; } done;
  1. Backup 40k tables
br --pd http://PD_ADDR backup full -s "local:///path/to/backupdir"

The master branch BR takes about 3.09GB of memory.

This PR BR takes about 2.21G of memory.

Code changes

  • Has exported function/method change
  • Has exported variable/fields change

Related changes

  • Need to cherry-pick to the release branch

Release Note

  • Reduce memory usage during backup.

@kennytm
Copy link
Collaborator

kennytm commented Mar 17, 2021

/lgtm

@ti-srebot ti-srebot added the status/LGT1 LGTM1 label Mar 17, 2021
@ti-srebot ti-srebot added status/LGT2 LGTM2 and removed status/LGT1 LGTM1 labels Mar 18, 2021
@overvenus overvenus merged commit 8d83ca7 into pingcap:release-5.0 Mar 18, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants