Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem: memiavl don't recover corrupted wal tail #1073

Merged
merged 12 commits into from
Jun 21, 2023

Conversation

yihuang
Copy link
Collaborator

@yihuang yihuang commented Jun 20, 2023

Solution:

  • truncate the corrupted segment tail

👮🏻👮🏻👮🏻 !!!! REFERENCE THE PROBLEM YOUR ARE SOLVING IN THE PR TITLE AND DESCRIBE YOUR SOLUTION HERE !!!! DO NOT FORGET !!!! 👮🏻👮🏻👮🏻

PR Checklist:

  • Have you read the CONTRIBUTING.md?
  • Does your PR follow the C4 patch requirements?
  • Have you rebased your work on top of the latest master?
  • Have you checked your code compiles? (make)
  • Have you included tests for any non-trivial functionality?
  • Have you checked your code passes the unit tests? (make test)
  • Have you checked your code formatting is correct? (go fmt)
  • Have you checked your basic code style is fine? (golangci-lint run)
  • If you added any dependencies, have you checked they do not contain any known vulnerabilities? (go list -json -m all | nancy sleuth)
  • If your changes affect the client infrastructure, have you run the integration test?
  • If your changes affect public APIs, does your PR follow the C4 evolution of public contracts?
  • If your code changes public APIs, have you incremented the crate version numbers and documented your changes in the CHANGELOG.md?
  • If you are contributing for the first time, please read the agreement in CONTRIBUTING.md now and add a comment to this pull request stating that your PR is in accordance with the Developer's Certificate of Origin.

Thank you for your code, it's appreciated! :)

Solution:
- fix in wal and update dependencies
@codecov
Copy link

codecov bot commented Jun 20, 2023

Codecov Report

Merging #1073 (9cfa107) into main (06a0b5d) will decrease coverage by 0.40%.
The diff coverage is 55.71%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1073      +/-   ##
==========================================
- Coverage   46.53%   46.13%   -0.40%     
==========================================
  Files          81       82       +1     
  Lines        7135     7203      +68     
==========================================
+ Hits         3320     3323       +3     
- Misses       3462     3522      +60     
- Partials      353      358       +5     
Impacted Files Coverage Δ
memiavl/wal.go 54.41% <54.41%> (ø)
memiavl/db.go 57.04% <100.00%> (ø)

... and 2 files with indirect coverage changes

CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
Signed-off-by: yihuang <[email protected]>
@yihuang yihuang marked this pull request as ready for review June 21, 2023 07:19
@yihuang yihuang requested a review from a team as a code owner June 21, 2023 07:19
@yihuang yihuang requested review from JayT106 and calvinaco and removed request for a team June 21, 2023 07:19
@yihuang yihuang requested a review from mmsqe June 21, 2023 07:22
@yihuang yihuang enabled auto-merge June 21, 2023 07:29
if uint64(len(data)-n) < size {
return 0, wal.ErrCorrupt
}
return n + int(size), nil

Check failure

Code scanning / gosec

Potential integer overflow by integer type conversion

Potential integer overflow by integer type conversion
if n <= 0 {
return 0, wal.ErrCorrupt
}
if uint64(len(data)-n) < size {

Check failure

Code scanning / gosec

Potential integer overflow by integer type conversion

Potential integer overflow by integer type conversion
memiavl/wal.go Fixed Show fixed Hide fixed
"io/ioutil"
"os"
"path/filepath"
"unsafe"

Check failure

Code scanning / gosec

Blocklisted import unsafe

Blocklisted import unsafe
return 0, wal.ErrCorrupt
}
line := data[:idx]
dres := gjson.Get(*(*string)(unsafe.Pointer(&line)), "data")

Check warning

Code scanning / gosec

Use of unsafe calls should be audited

Use of unsafe calls should be audited
"io/ioutil"
"os"
"path/filepath"
"unsafe"

Check notice

Code scanning / CodeQL

Sensitive package import

Certain system packages contain functions which may be a possible source of non-determinism
@yihuang yihuang added this pull request to the merge queue Jun 21, 2023
Merged via the queue into crypto-org-chain:main with commit f83c74b Jun 21, 2023
@yihuang yihuang deleted the wal-recover-tail branch June 21, 2023 22:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants