Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add operators to ignore list and update WindowExpr parser #890

Merged
merged 5 commits into from
Apr 1, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -760,15 +760,21 @@ object SQLPlanParser extends Logging {
trim.replaceAll("""^\[+""", "").replaceAll("""\]+$""", "").
replaceAll("cast\\(", "").split("windowspecdefinition").map(_.trim)

// Get functionname from each array element except the last one as it doesn't contain
// Get function name from each array element except the last one as it doesn't contain
// any window function
for ( i <- 0 to windowExprs.size - 1 ) {
val windowFunc = windowFunctionPattern.findAllIn(windowExprs(i)).toList
val expr = windowFunc.last
val functionName = getFunctionName(windowFunctionPattern, expr)
functionName match {
case Some(func) => parsedExpressions += func
case _ => // NO OP
if (windowExprs.nonEmpty) {
// Iterate through each expression in windowExprs
for (expr <- windowExprs) {
amahussein marked this conversation as resolved.
Show resolved Hide resolved
val windowFunc = windowFunctionPattern.findAllIn(expr).toList
// Check to ensure windowFunc is not empty before accessing last element
if (windowFunc.nonEmpty) {
val expr = windowFunc.last
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This variable name shadows a variable name in the outer scope "expr"

val functionName = getFunctionName(windowFunctionPattern, expr)
functionName match {
case Some(func) => parsedExpressions += func
case _ => // NO OP
}
}
}
}
parsedExpressions.distinct.toArray
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -415,6 +415,8 @@ object ExecHelper {
private val ExecuteRefreshTable = "Execute RefreshTable"
private val ExecuteRepairTableCommand = "Execute RepairTableCommand"
private val ExecuteShowPartitionsCommand = "Execute ShowPartitionsCommand"
private val ExecuteClearCacheCommand = "Execute ClearCacheCommand"
private val ExecuteOptimizeTableCommandEdge = "Execute OptimizeTableCommandEdge"
// DeltaLakeOperations
private val ExecUpdateCommandEdge = "Execute UpdateCommandEdge"
private val ExecDeleteCommandEdge = "Execute DeleteCommandEdge"
Expand Down Expand Up @@ -444,11 +446,17 @@ object ExecHelper {
ExecuteRefreshTable,
ExecuteRepairTableCommand,
ExecuteShowPartitionsCommand,
ExecuteClearCacheCommand,
ExecuteOptimizeTableCommandEdge,
SubqueryExecParser.execName
)

def shouldIgnore(execName: String): Boolean = {
getAllIgnoreExecs.contains(execName)
// Normalize the execName by removing the trailing '$' character, if present.
// This is necessary because in Scala, the '$' character is often appended to the names of
// generated classes or objects, and we want to match the base name regardless of this suffix.
val normalizedExecName = execName.stripSuffix("$")
getAllIgnoreExecs.contains(normalizedExecName)
amahussein marked this conversation as resolved.
Show resolved Hide resolved
}
}

Expand Down