Skip to content

Commit

Permalink
[SPARK-45432][CORE] Remove deprecated Hadoop-2 LocatedFileStatus co…
Browse files Browse the repository at this point in the history
…nstructor

### What changes were proposed in this pull request?

This PR aims to remove `Hadoop 2`'s `LocatedFileStatus` constructor which is deprecated at Apache Hadoop 3.0.0.

### Why are the changes needed?

Since SPARK-42452 removes Hadoop 2 at Apache Spark 3.5.0, we can use Hadoop 3 API completely.

[HDFS-6984](https://issues.apache.org/jira/browse/HDFS-6984) deprecated this constructor in favor of new constructor at Apache Hadoop 3.0.0-beta1

- https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/fs/LocatedFileStatus.html#LocatedFileStatus-long-boolean-int-long-long-long-org.apache.hadoop.fs.permission.FsPermission-java.lang.String-java.lang.String-org.apache.hadoop.fs.Path-org.apache.hadoop.fs.Path-org.apache.hadoop.fs.BlockLocation:A-

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#43239 from dongjoon-hyun/SPARK-45432.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
dongjoon-hyun authored and LuciferYang committed Oct 7, 2023
1 parent 0b33488 commit d100813
Showing 1 changed file with 2 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,8 @@ private[spark] object HadoopFSUtils extends Logging {
}
}
val lfs = new LocatedFileStatus(f.getLen, f.isDirectory, f.getReplication, f.getBlockSize,
f.getModificationTime, 0, null, null, null, null, f.getPath, locations)
f.getModificationTime, 0, null, null, null, null, f.getPath,
f.hasAcl, f.isEncrypted, f.isErasureCoded, locations)
if (f.isSymlink) {
lfs.setSymlink(f.getSymlink)
}
Expand Down

0 comments on commit d100813

Please sign in to comment.