Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve/flink keywords #195

Merged
merged 2 commits into from
Oct 30, 2023
Merged

Conversation

HaydenOrz
Copy link
Collaborator

@HaydenOrz HaydenOrz commented Oct 26, 2023

简介

重新整理 FlinkSQL 关键字

主要变更

  1. 重新整理 FlinkSQL 关键字;
  2. 移除 reservedKeywords 规则和 columnAlias 规则,因为它们没有被使用;
  3. 同步更新相应与关键字相关的规则;

关于关键字重新整理的说明

为什么要重新整理关键字?

在实际运行中发现,目前已有的关键字规则与FlinkSQL 实际运行情况有差异,当前正在使用的关键字规则都来源于 FlinkSQL 官方文档 但是实际上不准。

新的关键字集合以什么为参考?

Flink 官方使用 calcite 作为 SQL 解析器,新的关键字列表是从 Calcite 官方文档与Flink 源码中汇总而来。
具体汇总逻辑如下:

  1. calcite 文档中的关键字列表为基础集合,与 Flink 源码中的 parser.tdd 文件中声明的keywords集合取并集;
  2. 将上述并集结果与 Flink 源码中的 parser.tdd 文件中声明的 nonReservedKeywords 集合取差集;

nonReservedKeywords 规则相关说明

有一部分关键字在 parser rule 中有应用,比如使用频率很高的CATALOGVIEW,但是在上述集合中不存在,这些关键字也都作为非保留关键字被添加到关键字集合中,这些关键字可以作为普通的标识符使用(已经在 Flink 引擎中验证过了)

reservedKeywordsUsedAsFuncName 规则相关说明

Flink 内置了很多函数(Flink Built-in Functions),这些函数的函数名和Flink的关键字有相当一部分相同,所以必须声明一条规则,使这些冲突的关键字也能作为函数名。reservedKeywordsUsedAsFuncParam 也同理。

@HaydenOrz HaydenOrz added the improvement Improve existing feature label Oct 26, 2023
@HaydenOrz HaydenOrz marked this pull request as draft October 26, 2023 13:34
@HaydenOrz HaydenOrz marked this pull request as ready for review October 30, 2023 09:19
@liuxy0551
Copy link
Collaborator

+1

@HaydenOrz HaydenOrz self-assigned this Oct 30, 2023
@mumiao
Copy link
Collaborator

mumiao commented Oct 30, 2023

+1

@mumiao mumiao merged commit 21dcb3f into DTStack:main Oct 30, 2023
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improve existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants