Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OpenAPI: Add planning-mode to loadTable response #11156

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

OpenAPI: Add planning-mode to loadTable response #11156

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions open-api/rest-catalog-open-api.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -473,6 +473,13 @@ class PlanStatus(BaseModel):
)


class PlanningMode(BaseModel):
__root__: Literal['unsupported', 'supported', 'required'] = Field(
...,
description='Indicates to clients if a service supports server-side planning for a given table.',
)


class RegisterTableRequest(BaseModel):
name: str
metadata_location: str = Field(..., alias='metadata-location')
Expand Down Expand Up @@ -1176,6 +1183,17 @@ class LoadTableResult(BaseModel):
The table metadata JSON is returned in the `metadata` field. The corresponding file location of table metadata should be returned in the `metadata-location` field, unless the metadata is not yet committed. For example, a create transaction may return metadata that is staged but not committed.
Clients can check whether metadata has changed by comparing metadata locations after the table has been created.

The `planning-mode` returns a string enum to clients, whether a service supports server-side planning for a given table.

- When "required" the client must initiate server-side planning
by calling the `planTableScan` operation.

- When "supported" the client can opt to either initiate service-side planning
or client-side planning.

- When "unsupported" the service does not support server-side planning,
therefore the client will be expected to do its planning.


The `config` map returns table-specific configuration for the table's resources, including its HTTP client and FileIO. For example, config may contain a specific FileIO implementation class for the table depending on its underlying storage.

Expand Down Expand Up @@ -1203,6 +1221,7 @@ class LoadTableResult(BaseModel):
description='May be null if the table is staged as part of a transaction',
)
metadata: TableMetadata
planning_mode: Optional[PlanningMode] = Field(None, alias='planning-mode')
config: Optional[Dict[str, str]] = None


Expand Down
18 changes: 18 additions & 0 deletions open-api/rest-catalog-open-api.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3110,6 +3110,17 @@ components:

The table metadata JSON is returned in the `metadata` field. The corresponding file location of table metadata should be returned in the `metadata-location` field, unless the metadata is not yet committed. For example, a create transaction may return metadata that is staged but not committed.
Clients can check whether metadata has changed by comparing metadata locations after the table has been created.

The `planning-mode` returns a string enum to clients, whether a service supports server-side planning for a given table.

- When "required" the client must initiate server-side planning
by calling the `planTableScan` operation.

- When "supported" the client can opt to either initiate service-side planning
or client-side planning.

- When "unsupported" the service does not support server-side planning,
therefore the client will be expected to do its planning.


The `config` map returns table-specific configuration for the table's resources, including its HTTP client and FileIO. For example, config may contain a specific FileIO implementation class for the table depending on its underlying storage.
Expand Down Expand Up @@ -3138,6 +3149,8 @@ components:
description: May be null if the table is staged as part of a transaction
metadata:
$ref: '#/components/schemas/TableMetadata'
planning-mode:
$ref: '#/components/schemas/PlanningMode'
config:
type: object
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like from the sync meeting, we are saying the planning mode should be in config, something like:

config:
  type: object
  properties:
     planning-mode:
       $ref: '#/components/schemas/PlanningMode'
  additionalProperties:
    type: string

Is this the right understanding?

additionalProperties:
Expand Down Expand Up @@ -3240,6 +3253,11 @@ components:
type: string
enum: ["completed", "submitted", "cancelled", "failed"]

PlanningMode:
description: Indicates to clients if a service supports server-side planning for a given table.
type: string
enum: ["unsupported", "supported", "required"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default unsupported?


FetchPlanningResult:
type: object
description: Result of server-side scan planning for fetchPlanningResult
Expand Down