Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs/design: add proposal for recursive common table expression (CTE) #19253

Closed
wants to merge 2 commits into from

Conversation

pingyu
Copy link
Contributor

@pingyu pingyu commented Aug 17, 2020

What problem does this PR solve?

Issue Number: #17472 #6824

Problem Summary:

What is changed and how it works?

Proposal: Recursive Common Table Expression

What's Changed:
(TBD)

How it Works:
(TBD)

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn: (TBD)

Check List

Tests
(TBD)

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects
(TBD)

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM
  • Breaking backward compatibility

Release note

  • Support recursive common table expression (CTE)

@ti-srebot ti-srebot added the contribution This PR is from a community contributor. label Aug 17, 2020
@zz-jason zz-jason changed the title planner,executor: support recursive CTE (design doc) docs/design: add proposal for recursive common table expression (CTE) Aug 26, 2020
@@ -0,0 +1,108 @@

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: it's better to remove empty lines of a file.

- Author(s): [Pingyu](https://github.com/pingyu) (Ping Yu)
- Last updated: 2020-08-18
- Discussion at: https://github.com/pingcap/tidb/issues/17472

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: it's better to keep only one empty line between content sections.

CTEs depending on other CTE(s) begins to execute after the dependencies are all met. Independent CTEs can execute parallel.

## Rationale
MariaDB[4] build a dependency matrix in preparatory stage to detect recursive CTEs, check seed parts, and determine dependency between CTEs.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's better to add more content about how recursive common table expression is implemented in other DBMS like what we did in the proposal of non-recursive common table expression.


## Background

From MySQL User Document, [13.2.15 WITH (Common Table Expressions)](https://dev.mysql.com/doc/refman/8.0/en/with.html):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to describe more about the usage of recursive CTE like what we did in the proposal of non-recursive CTE.

## Implementation

### Temporary Table
Temporary table is implemented by `chunk.RowContainer`, which stores tuples in memory, or spill to disk if exceeds memory quota.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to share the same code which implements the temporary table described in #1248?

## Proposal

### Seed part & Recursive part
Recursive CTE has two parts, seed part and recursive part, separated by `UNION ALL` or `UNION [DISTINCT]`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to introduce the syntax of recursive CTE beforing explaining what are seed, union all, and recursive parts.

@zz-jason
Copy link
Member

zz-jason commented Feb 9, 2021

I'm going to close this PR since it hasn't been updated for a long time. feel free to reopen if you want to continue with it. thank you for your contribution.

@zz-jason zz-jason closed this Feb 9, 2021
@guo-shaoge
Copy link
Collaborator

The design doc is great, and we need to continue to finish the implementation of recursive CTE.
I'd like to open a new PR and add some details based on your design doc. Is that OK? @pingyu

@pingyu
Copy link
Contributor Author

pingyu commented Mar 30, 2021

The design doc is great, and we need to continue to finish the implementation of recursive CTE.
I'd like to open a new PR and add some details based on your design doc. Is that OK? @pingyu

That's great !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution This PR is from a community contributor.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants