From bb56226c10df9d07540c97f62e31ee09981464cf Mon Sep 17 00:00:00 2001 From: Dongjoon Hyun Date: Fri, 6 Oct 2023 00:22:15 -0700 Subject: [PATCH] [SPARK-45432][CORE] Remove deprecated Hadoop-2 `LocatedFileStatus` constructor ### What changes were proposed in this pull request? This PR aims to remove `Hadoop 2`'s `LocatedFileStatus` constructor which is deprecated at Apache Hadoop 3.0.0. ### Why are the changes needed? Since SPARK-42452 removes Hadoop 2 at Apache Spark 3.5.0, we can use Hadoop 3 API completely. [HDFS-6984](https://issues.apache.org/jira/browse/HDFS-6984) deprecated this constructor in favor of new constructor at Apache Hadoop 3.0.0-beta1 - https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/fs/LocatedFileStatus.html#LocatedFileStatus-long-boolean-int-long-long-long-org.apache.hadoop.fs.permission.FsPermission-java.lang.String-java.lang.String-org.apache.hadoop.fs.Path-org.apache.hadoop.fs.Path-org.apache.hadoop.fs.BlockLocation:A- ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #43239 from dongjoon-hyun/SPARK-45432. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun --- core/src/main/scala/org/apache/spark/util/HadoopFSUtils.scala | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/core/src/main/scala/org/apache/spark/util/HadoopFSUtils.scala b/core/src/main/scala/org/apache/spark/util/HadoopFSUtils.scala index bb03cac8f1f62..db1eeeef47127 100644 --- a/core/src/main/scala/org/apache/spark/util/HadoopFSUtils.scala +++ b/core/src/main/scala/org/apache/spark/util/HadoopFSUtils.scala @@ -272,7 +272,8 @@ private[spark] object HadoopFSUtils extends Logging { } } val lfs = new LocatedFileStatus(f.getLen, f.isDirectory, f.getReplication, f.getBlockSize, - f.getModificationTime, 0, null, null, null, null, f.getPath, locations) + f.getModificationTime, 0, null, null, null, null, f.getPath, + f.hasAcl, f.isEncrypted, f.isErasureCoded, locations) if (f.isSymlink) { lfs.setSymlink(f.getSymlink) }