HDDS-10770. [Hsync] Allow overwrite hsynced file #6603
Conversation
ChenSammi
changed the title
| sb.append(" "); | ||
| sb.append("length ").append(length); | ||
| sb.append(" "); | ||
| sb.append("keyOffset ").append(keyOffset); |
There was a problem hiding this comment.
should it print offset as well?
| try { | ||
| outputStream1.write(newData); | ||
| } catch (IOException e) { | ||
| assertTrue(e.getCause() instanceof OMException); | ||
| assertTrue(((OMException)e.getCause()).getResult() == OMException.ResultCodes.KEY_NOT_FOUND); | ||
| assertTrue(e.getMessage().contains("already deleted/overwritten")); | ||
| } |
There was a problem hiding this comment.
If exception is expected here, use assertThrows()
| try { | |
| outputStream1.write(newData); | |
| } catch (IOException e) { | |
| assertTrue(e.getCause() instanceof OMException); | |
| assertTrue(((OMException)e.getCause()).getResult() == OMException.ResultCodes.KEY_NOT_FOUND); | |
| assertTrue(e.getMessage().contains("already deleted/overwritten")); | |
| } | |
| IOException e = assertThrows(IOException.class, | |
| () -> outputStream1.write(newData)); | |
| assertTrue(e.getCause() instanceof OMException); | |
| assertTrue(((OMException)e.getCause()).getResult() == OMException.ResultCodes.KEY_NOT_FOUND); | |
| assertTrue(e.getMessage().contains("already deleted/overwritten")); |
| } | ||
| // commit overwritten hsync key, should fail | ||
| try { | ||
| outputStream1.close(); |
|
@jojochuang , thanks for the review. The comments are addressed in a new patch. |
| @@ -369,7 +390,7 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager, TermIn | |||
|
|
|||
| omClientResponse = new OMKeyCommitResponse(omResponse.build(), | |||
| omKeyInfo, dbOzoneKey, dbOpenKey, omBucketInfo.copyObject(), | |||
| oldKeyVersionsToDeleteMap, isHSync, newOpenKeyInfo); | |||
| oldKeyVersionsToDeleteMap, isHSync, newOpenKeyInfo, dbOpenKeyToDeleteKey, openKeyToDelete); | |||
There was a problem hiding this comment.
Ignore passing dbOpenKeyToDeleteKey, can be retrieved using openKeyToDelete.
There was a problem hiding this comment.
@ashishkumar50 , thanks for the review. I would prefer keep passing the dbOpenKeyToDeleteKey.
| if (openKeyInfo.getMetadata().containsKey(OzoneConsts.DELETED_HSYNC_KEY)) { | ||
| throw new OMException("Open Key " + openKeyName + " is already deleted", | ||
| if (openKeyInfo.getMetadata().containsKey(OzoneConsts.DELETED_HSYNC_KEY) || | ||
| openKeyInfo.getMetadata().containsKey(OzoneConsts.OVERWRITTEN_HSYNC_KEY)) { |
There was a problem hiding this comment.
OVERWRITTEN_HSYNC_KEY is added during commit/hsync. If hsync is not called and allocate block request is called from client, in the corner case it will allocate block for both clients. Doesn't impact but an extra block will be allocated.
There was a problem hiding this comment.
This case cannot be avoided due to the invisibility of openKey.
| @@ -225,6 +225,7 @@ private RecoverLeaseResponse doWork(OzoneManager ozoneManager, | |||
| throw new OMException("Open Key " + keyName + " is already deleted", | |||
| KEY_NOT_FOUND); | |||
| } | |||
|
|
|||
There was a problem hiding this comment.
Remove unchanged file.
|
Thanks @ashishkumar50 and @jojochuang for the code review. |
(cherry picked from commit d7e5b3a)
What changes were proposed in this pull request?
Allow overwritten a hsynced file, without calling data loss.
The overall rule is
a. the later commit overwrites early commit
b. avoid mutual overwrites. Once a file is overwritten, it cannot later overwritten others again.
To support this, when a hsynced file is overwritten, it will have "overwritten" metadata as a flag. All following allocate block, further hsync or commit of this file will fail.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-10770
How was this patch tested?
new integration tests