Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

util: create the tmpdir if the directory is removed by mistake. (#18970) #19272

Merged
merged 3 commits into from
Aug 19, 2020

Conversation

ti-srebot
Copy link
Contributor

cherry-pick #18970 to release-4.0


What problem does this PR solve?

Issue Number:
close #18960
close #18055

Problem Summary:

  1. If the temp directory is removed, all SQLs that trigger spill action will get unexpected error open: ...... no such file or directory.
  2. If remove some files failed, log the error and continue to remove other temp files.
  3. If there is a temp file removed failed when calling Close, other temp files can't remove successfully.

What is changed and how it works?

Proposal: xxx

What's Changed:

  1. Before creating temp files and writing them, check the temp directory is valid. If the directory invalid, create the directory before spilling.
  2. Log and ignore the error when calling Close().

How it Works:

Related changes

  • Need to cherry-pick to the release branch 4.0

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    Case 1:
  1. Start a tidb instance.
  2. Remove the temp directory.
  3. Run a SQL that spills some data to disk.
  4. SQL runs successfully, and the temp directory is created again.

Case 2:

  1. Start a tidb instance.
  2. Run a SQL that spills some data to disk.
  3. Remove some temp files when running.
  4. SQL runs failed, and the other temp files are removed successfully.

Side effects

Release note

  • Fix unexpected error when spilling disk.

@ti-srebot
Copy link
Contributor Author

/run-all-tests

@wshwsh12
Copy link
Contributor

/run-all-tests

@wshwsh12
Copy link
Contributor

/run-unit-test

Copy link
Contributor

@XuHuaiyu XuHuaiyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 19, 2020
Copy link
Contributor

@SunRunAway SunRunAway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 19, 2020
@jebter
Copy link

jebter commented Aug 19, 2020

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 19, 2020
@ti-srebot
Copy link
Contributor Author

/run-all-tests

@jebter
Copy link

jebter commented Aug 19, 2020

/run-integration-ddl-test

@ti-srebot
Copy link
Contributor Author

@ti-srebot merge failed.

@jebter jebter merged commit a113adf into pingcap:release-4.0 Aug 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/config component/executor sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug. type/4.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants