Skip to content

Commit

Permalink
[Spark] Add utils to find logPath from dataPath in CoordinatedCommits…
Browse files Browse the repository at this point in the history
…Utils (#3779)

#### Which Delta project/connector is this regarding?

- [x] Spark
- [ ] Standalone
- [ ] Flink
- [ ] Kernel
- [ ] Other (fill in here)

## Description

Add utils to find logPath from dataPath in CoordinatedCommitsUtils

## How was this patch tested?

Build

## Does this PR introduce _any_ user-facing changes?

No
  • Loading branch information
sumeet-db authored Oct 22, 2024
1 parent 8442bc6 commit 4f96aa1
Showing 1 changed file with 8 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ public class CoordinatedCommitsUtils {

private CoordinatedCommitsUtils() {}

/** The subdirectory in which to store the delta log. */
private static final String LOG_DIR_NAME = "_delta_log";

/** The subdirectory in which to store the unbackfilled commit files. */
private static final String COMMIT_SUBDIR = "_commits";

Expand Down Expand Up @@ -127,6 +130,11 @@ public static FileStatus writeUnbackfilledCommitFile(
return commitPath.getFileSystem(hadoopConf).getFileStatus(commitPath);
}

/** Returns path to the directory which holds the delta log */
public static Path logDirPath(Path tablePath) {
return new Path(tablePath, LOG_DIR_NAME);
}

/** Returns path to the directory which holds the unbackfilled commits */
public static Path commitDirPath(Path logPath) {
return new Path(logPath, COMMIT_SUBDIR);
Expand Down

0 comments on commit 4f96aa1

Please sign in to comment.