Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): implement file-based remote function register #829

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

goldmedal
Copy link
Contributor

@goldmedal goldmedal commented Oct 16, 2024

Description

When planning a SQL, DataFusion planner will strictly check the function usage. The used function should be registered in the SessionContext. In the Wren AI scenario, we always fully push down the SQL to the data source side. We can invoke the ByPassFunction (introduced by #802) which only requires the function name and return type to register a logical function for unparsing purposes.

This PR introduces a file-based register to define the required remote function easily.

Definition File

The definition file is a CSV file that contains 4 columns and with the header:

  • function_type: The possible value contains scalar, aggregate, and window. All of them are lowercase.
  • name: The function name.
  • return_type: The expected return type.
  • description: A simple description of this function. Because the description could contain a comma, we should quote the value.

Sample

function_type,name,return_type,description
scalar,add_two,int,"Adds two numbers together."
aggregate,median,int,"Returns the median value of a numeric column.".
window,max_if,int,"If the condition is true, returns the maximum value in the window."

Set up the config

On the ibis server side, we add a new environment variable called REMOTE_FUNCTION_LIST_PATH which is the path of the definition file. We should set up it before starting the server.

Known Issue

  • The window function is not yet supported

@github-actions github-actions bot added the dependencies Pull requests that update a dependency file label Oct 16, 2024
@goldmedal goldmedal marked this pull request as ready for review October 16, 2024 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants