What's the different between CreateFile and CreateKey? #5308
-
I found that there are two methods to create an object(file) in Ozone:
I did not find the difference between the two in both document and the design doc, which made me very confused.
So why we have two seprated interface to create keys in OM? What's the different of them? Why file create needs bucket level write lock? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
I think the API's evolved to be the same , createFile got introduced here https://issues.apache.org/jira/browse/HDDS-1300 pre OM HA , The difference was that createKey earlier also was used to create multipart uploads and other OBS related functionality but createFile was specifically called from Ozone FS. Now, I guess multipart uploads have different request response api's so ultimately and the functionality converged b/w createFile & key. public static final boolean OZONE_OM_KEY_PATH_LOCK_ENABLED_DEFAULT = false; |
Beta Was this translation helpful? Give feedback.
-
I looked into the code and could find a difference, The CreateFileRequest has an extra field 'recursive' . The effect of this is that if recursive is true, the parent directory of the file would be created even if it doesn't exist. This is not true for createKey as it doesn't have a recursive param that the client can set, parent dirs are created for createKey depending on the value of I wrote a simple unit test when @Test
public void testCreateKeyAndCreateFile() throws IOException {
// ozone.om.enable.filesystem.paths = false
OzoneBucket bucket =
TestDataUtil.createVolumeAndBucket(client, BucketLayout.LEGACY);
String keyName = "a/b/c";
TestDataUtil.createKey(bucket, keyName, "");
List<String> keys = new ArrayList<>();
Iterator<? extends OzoneKey> keyIter = bucket.listKeys("");
while (keyIter.hasNext()){
keys.add(keyIter.next().getName());
}
// only a single key
Assert.assertEquals(1,keys.size());
bucket.deleteKey("a/b/c");
keys.clear();
try (OzoneOutputStream oos = bucket.createFile(keyName, 0,
RatisReplicationConfig.getInstance(HddsProtos.ReplicationFactor.THREE),
true, true)) {
oos.flush();
}
keyIter = bucket.listKeys("");
while (keyIter.hasNext()){
keys.add(keyIter.next().getName());
}
// 3 keys for createFile a/, a/b/ , a/b/c even though
// ozone.om.enable.filesystem.paths = false
Assert.assertEquals(3,keys.size());
} Hope this answers your question! |
Beta Was this translation helpful? Give feedback.
I looked into the code and could find a difference, The CreateFileRequest has an extra field 'recursive' . The effect of this is that if recursive is true, the parent directory of the file would be created even if it doesn't exist.
This is not true for createKey as it doesn't have a recursive param that the client can set, parent dirs are created for createKey depending on the value of
ozone.om.enable.filesystem.paths
. This can be set to either true/false in LEGACY buckets. If true, parent dir is created , else parent dir is not created for createKey. For other bucket types, this config doesn't take any effect (for FSO it's always true and OBS false)I wrote a simple unit test when
ozone…