Skip to content

Commit

Permalink
feat: [dataproc] add support for Spark Connect sessions in Dataproc S…
Browse files Browse the repository at this point in the history
…erverless for Spark (#5709)

* feat: add support for Spark Connect sessions in Dataproc Serverless for Spark
docs: update docs for `filter` field in `ListSessionsRequest`

PiperOrigin-RevId: 678438691

Source-Link: googleapis/googleapis@5c181aa

Source-Link: googleapis/googleapis-gen@3b37678
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWRhdGFwcm9jLy5Pd2xCb3QueWFtbCIsImgiOiIzYjM3Njc4ZTNlZDRlNTJmMmExMjJlYTkxNzM4YTllN2I1ZjRjYmExIn0=

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Daniel Bankhead <[email protected]>
  • Loading branch information
3 people authored Sep 27, 2024
1 parent 8928489 commit fe95238
Show file tree
Hide file tree
Showing 8 changed files with 407 additions and 24 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,10 @@ message SessionTemplate {
oneof session_config {
// Optional. Jupyter session config.
JupyterConfig jupyter_session = 3 [(google.api.field_behavior) = OPTIONAL];

// Optional. Spark Connect session config.
SparkConnectConfig spark_connect_session = 11
[(google.api.field_behavior) = OPTIONAL];
}

// Output only. The email address of the user who created the template.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -163,12 +163,13 @@ message ListSessionsRequest {
// A filter is a logical expression constraining the values of various fields
// in each session resource. Filters are case sensitive, and may contain
// multiple clauses combined with logical operators (AND, OR).
// Supported fields are `session_id`, `session_uuid`, `state`, and
// `create_time`.
// Supported fields are `session_id`, `session_uuid`, `state`, `create_time`,
// and `labels`.
//
// Example: `state = ACTIVE and create_time < "2023-01-01T00:00:00Z"`
// is a filter for sessions in an ACTIVE state that were created before
// 2023-01-01.
// 2023-01-01. `state = ACTIVE and labels.environment=production` is a filter
// for sessions in an ACTIVE state that have a production environment label.
//
// See https://google.aip.dev/assets/misc/ebnf-filtering.txt for a detailed
// description of the filter syntax and a list of supported comparators.
Expand Down Expand Up @@ -289,6 +290,10 @@ message Session {
oneof session_config {
// Optional. Jupyter session config.
JupyterConfig jupyter_session = 4 [(google.api.field_behavior) = OPTIONAL];

// Optional. Spark Connect session config.
SparkConnectConfig spark_connect_session = 17
[(google.api.field_behavior) = OPTIONAL];
}

// Output only. Runtime information about session execution.
Expand Down Expand Up @@ -369,3 +374,6 @@ message JupyterConfig {
// Optional. Display name, shown in the Jupyter kernelspec card.
string display_name = 2 [(google.api.field_behavior) = OPTIONAL];
}

// Spark Connect configuration for an interactive session.
message SparkConnectConfig {}
107 changes: 105 additions & 2 deletions packages/google-cloud-dataproc/protos/protos.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit fe95238

Please sign in to comment.