Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Evaluate converting expressions names to lowercase #409

Open
nartal1 opened this issue Jul 5, 2023 · 0 comments
Open

[BUG] Evaluate converting expressions names to lowercase #409

nartal1 opened this issue Jul 5, 2023 · 0 comments
Labels
bug Something isn't working core_tools Scope the core module (scala)

Comments

@nartal1
Copy link
Collaborator

nartal1 commented Jul 5, 2023

Describe the bug
This is based on the comment #408 (review) .
Currently we convert the expressions string to lower case before comparing. We need to revisit this if some of the conversions are redundant OR not required in the first place.

Targets
    Occurrences of 'toLowerCase' in Project with mask '*.scala'
Found Occurrences in Project with mask '*.scala'  (26 usages found)
    Unclassified  (26 usages found)
        rapids-4-spark-tools_2.12  (26 usages found)
            com.nvidia.spark.rapids.tool.planparser  (3 usages found)
                DataWritingCommandExecParser.scala  (1 usage found)
                    41 new ExecInfo(sqlID, s"${node.name.trim} ${wStub.dataFormat.toLowerCase.trim}", "",
                SQLPlanParser.scala  (2 usages found)
                    356 ignoreExpressions.contains(expr.toLowerCase)
                    704 functionMatches.map(_.group(1)).filter(_.toLowerCase() != "cast")
            com.nvidia.spark.rapids.tool.profiling  (8 usages found)
                AutoTuner.scala  (3 usages found)
                    626 .getOrElse("false").toLowerCase
                    1063 val sizesArr = size.toLowerCase.split("(?=[a-z])")
                    1075 val sizesArr = size.toLowerCase.split("(?=[a-z])")
                ProfileOutputWriter.scala  (1 usage found)
                    63 val suffix = header.replace(" ", "_").toLowerCase
                QualificationInfoUtils.scala  (4 usages found)
                    171 if (logType.toLowerCase.equals("dataset")) {
                    173 } else if (logType.toLowerCase.equals("udfds")) {
                    175 } else if (logType.toLowerCase.equals("udffunc")) {
                    177 } else if (logType.toLowerCase.equals("dsanddf")) {
            com.nvidia.spark.rapids.tool.qualification  (14 usages found)
                PluginTypeChecker.scala  (12 usages found)
                    115 x => (x._1.toLowerCase.replaceAll("\\`", ""), x._2))
                    136 val header = fileContents.head.split(",").map(_.toLowerCase)
                    189 val header = fileContents.head.split(",").map(_.toLowerCase)
                    197 val format = cols(0).toLowerCase
                    198 val direction = cols(1).toLowerCase()
                    203 }.keys.toSeq.map(_.toLowerCase)
                    236 val schemaLower = schema.toLowerCase
                    237 val formatInLower = format.toLowerCase
                    242 val nsFiltered = dtSupMap(NS).filter(t => schemaLower.contains(t.toLowerCase()))
                    261 val format = writeFormat.toLowerCase.trim
                    266 writeFormat.map(x => x.toLowerCase.trim).filterNot(
                    303 expr.toLowerCase.replace("_", "")
                QualificationArgs.scala  (2 usages found)
                    193 order.toLowerCase.startsWith("asc")
                    197 order.toLowerCase.startsWith("desc")
            org.apache.spark.sql.rapids.tool.qualification  (1 usage found)
                QualificationAppInfo.scala  (1 usage found)
                    179 s"${ds.format.toLowerCase()}[${ds.schema}]"
@nartal1 nartal1 added bug Something isn't working ? - Needs Triage labels Jul 5, 2023
@mattahrens mattahrens added the core_tools Scope the core module (scala) label Jul 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working core_tools Scope the core module (scala)
Projects
None yet
Development

No branches or pull requests

3 participants