Skip to content

Commit

Permalink
Support getFileBlockLocation in LocalCacheFileSystem
Browse files Browse the repository at this point in the history
### What changes are proposed in this pull request?

Delegate `getFileBlockLocation` to external file system in `LocalCacheFileSystem`.

### Why are the changes needed?

Otherwise, `LocalCacheFileSystem` inherits the default behavior of `org.apache.hadoop.fs.FileSystem` which returns `localhost` only. 

### Does this PR introduce any user facing changes?

No.

			pr-link: #17672
			change-id: cid-eb545dbd8ed42001d074fecfb9c8d6b118a559c1
  • Loading branch information
maobaolong authored Jul 24, 2023
1 parent cb2d994 commit f3d1af8
Showing 1 changed file with 19 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import alluxio.wire.FileInfo;

import com.google.common.base.Preconditions;
import org.apache.hadoop.fs.BlockLocation;
import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hadoop.fs.FSDataOutputStream;
import org.apache.hadoop.fs.FileStatus;
Expand Down Expand Up @@ -213,4 +214,22 @@ public boolean mkdirs(Path f, FsPermission permission) throws IOException {
public FileStatus getFileStatus(Path f) throws IOException {
return mExternalFileSystem.getFileStatus(f);
}

@Override
public BlockLocation[] getFileBlockLocations(FileStatus file, long start,
long len) throws IOException {
// Applications use the block information here to schedule/distribute the tasks.
// Return the UFS locations directly instead of the local cache location,
// so the application can schedule the tasks accordingly
return mExternalFileSystem.getFileBlockLocations(file, start, len);
}

@Override
public BlockLocation[] getFileBlockLocations(Path p, long start, long len)
throws IOException {
// Applications use the block information here to schedule/distribute the tasks.
// Return the UFS locations directly instead of the local cache location,
// so the application can schedule the tasks accordingly
return mExternalFileSystem.getFileBlockLocations(p, start, len);
}
}

0 comments on commit f3d1af8

Please sign in to comment.