Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Links to HistoryEvent #448

Merged
merged 6 commits into from
Aug 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 89 additions & 0 deletions openapi/openapiv2.json
Original file line number Diff line number Diff line change
Expand Up @@ -4733,6 +4733,23 @@
}
}
},
"LinkWorkflowEvent": {
"type": "object",
"properties": {
"namespace": {
"type": "string"
},
"workflowId": {
"type": "string"
},
"runId": {
"type": "string"
},
"eventRef": {
"$ref": "#/definitions/WorkflowEventEventReference"
}
}
},
"OperatorServiceUpdateNexusEndpointBody": {
"type": "object",
"properties": {
Expand All @@ -4751,6 +4768,13 @@
"properties": {
"operationId": {
"type": "string"
},
"links": {
"type": "array",
"items": {
"type": "object",
"$ref": "#/definitions/apinexusv1Link"
}
}
},
"description": "The operation will complete asynchronously.\nThe returned ID can be used to reference this operation."
Expand Down Expand Up @@ -4877,6 +4901,18 @@
},
"description": "Replaces the routing rule with the given source Build ID."
},
"WorkflowEventEventReference": {
"type": "object",
"properties": {
"eventId": {
"type": "string",
"format": "int64"
},
"eventType": {
"$ref": "#/definitions/v1EventType"
}
}
},
"WorkflowServiceCreateScheduleBody": {
"type": "object",
"properties": {
Expand Down Expand Up @@ -5469,6 +5505,14 @@
"userMetadata": {
"$ref": "#/definitions/v1UserMetadata",
"description": "Metadata on the workflow if it is started. This is carried over to the WorkflowExecutionInfo\nfor use by user interfaces to display the fixed as-of-start summary and details of the\nworkflow."
},
"links": {
"type": "array",
"items": {
"type": "object",
"$ref": "#/definitions/apicommonv1Link"
},
"description": "Links to be associated with the workflow."
}
}
},
Expand Down Expand Up @@ -5609,6 +5653,15 @@
}
}
},
"apicommonv1Link": {
"type": "object",
"properties": {
"workflowEvent": {
"$ref": "#/definitions/LinkWorkflowEvent"
}
},
"description": "Link can be associated with history events. It might contain information about an external entity\nrelated to the history event. For example, workflow A makes a Nexus call that starts workflow B:\nin this case, a history event in workflow A could contain a Link to the workflow started event in\nworkflow B, and vice-versa."
},
"apifailurev1Failure": {
"type": "object",
"properties": {
Expand Down Expand Up @@ -5677,6 +5730,18 @@
},
"title": "A general purpose failure message.\nSee: https://github.com/nexus-rpc/api/blob/main/SPEC.md#failure"
},
"apinexusv1Link": {
"type": "object",
"properties": {
"url": {
"type": "string",
"description": "See https://github.com/nexus-rpc/api/blob/main/SPEC.md#links."
},
"type": {
"type": "string"
}
}
},
"apinexusv1Request": {
"type": "object",
"properties": {
Expand Down Expand Up @@ -7627,6 +7692,14 @@
"$ref": "#/definitions/v1UserMetadata",
"description": "Metadata on the event. This is often carried over from commands and client calls. Most events\nwon't have this information, and how this information is used is dependent upon the interface\nthat reads it.\n\nCurrent well-known uses:\n * workflow_execution_started_event_attributes - summary and details from start workflow.\n * timer_started_event_attributes - summary represents an identifier for the timer for use by\n user interfaces."
},
"links": {
"type": "array",
"items": {
"type": "object",
"$ref": "#/definitions/apicommonv1Link"
},
"description": "Links associated with the event."
},
"workflowExecutionStartedEventAttributes": {
"$ref": "#/definitions/v1WorkflowExecutionStartedEventAttributes"
},
Expand Down Expand Up @@ -10358,6 +10431,14 @@
"type": "string"
},
"description": "Header that is expected to be attached to the callback request when the operation completes."
},
"links": {
"type": "array",
"items": {
"type": "object",
"$ref": "#/definitions/apinexusv1Link"
},
"description": "Links contain caller information and can be attached to the operations started by the handler."
}
},
"description": "A request to start an operation."
Expand Down Expand Up @@ -10494,6 +10575,14 @@
"userMetadata": {
"$ref": "#/definitions/v1UserMetadata",
"description": "Metadata on the workflow if it is started. This is carried over to the WorkflowExecutionInfo\nfor use by user interfaces to display the fixed as-of-start summary and details of the\nworkflow."
},
"links": {
"type": "array",
"items": {
"type": "object",
"$ref": "#/definitions/apicommonv1Link"
},
"description": "Links to be associated with the workflow."
}
}
},
Expand Down
95 changes: 95 additions & 0 deletions openapi/openapiv3.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5635,6 +5635,11 @@ components:
* workflow_execution_started_event_attributes - summary and details from start workflow.
* timer_started_event_attributes - summary represents an identifier for the timer for use by
user interfaces.
links:
type: array
items:
$ref: '#/components/schemas/Link'
description: Links associated with the event.
workflowExecutionStartedEventAttributes:
$ref: '#/components/schemas/WorkflowExecutionStartedEventAttributes'
workflowExecutionCompletedEventAttributes:
Expand Down Expand Up @@ -5783,6 +5788,27 @@ components:
xx:19:00. An interval of 28 days with phase zero would match
2022-02-17T00:00:00Z (among other times). The same interval with a phase of 3
days, 5 hours, and 23 minutes would match 2022-02-20T05:23:00Z instead.
Link:
type: object
properties:
workflowEvent:
$ref: '#/components/schemas/Link_WorkflowEvent'
description: |-
Link can be associated with history events. It might contain information about an external entity
related to the history event. For example, workflow A makes a Nexus call that starts workflow B:
in this case, a history event in workflow A could contain a Link to the workflow started event in
workflow B, and vice-versa.
Link_WorkflowEvent:
type: object
properties:
namespace:
type: string
workflowId:
type: string
runId:
type: string
eventRef:
$ref: '#/components/schemas/WorkflowEvent_EventReference'
ListArchivedWorkflowExecutionsResponse:
type: object
properties:
Expand Down Expand Up @@ -8043,6 +8069,11 @@ components:
Metadata on the workflow if it is started. This is carried over to the WorkflowExecutionInfo
for use by user interfaces to display the fixed as-of-start summary and details of the
workflow.
links:
type: array
items:
$ref: '#/components/schemas/Link'
description: Links to be associated with the workflow.
StartWorkflowExecutionResponse:
type: object
properties:
Expand Down Expand Up @@ -8670,6 +8701,70 @@ components:
If set, the worker is opting in to worker versioning. Otherwise, this is used only as a
marker for workflow reset points and the BuildIDs search attribute.
description: Identifies the version(s) of a worker that processed a task
WorkflowEvent_EventReference:
type: object
properties:
eventId:
type: string
eventType:
enum:
- EVENT_TYPE_UNSPECIFIED
- EVENT_TYPE_WORKFLOW_EXECUTION_STARTED
- EVENT_TYPE_WORKFLOW_EXECUTION_COMPLETED
- EVENT_TYPE_WORKFLOW_EXECUTION_FAILED
- EVENT_TYPE_WORKFLOW_EXECUTION_TIMED_OUT
- EVENT_TYPE_WORKFLOW_TASK_SCHEDULED
- EVENT_TYPE_WORKFLOW_TASK_STARTED
- EVENT_TYPE_WORKFLOW_TASK_COMPLETED
- EVENT_TYPE_WORKFLOW_TASK_TIMED_OUT
- EVENT_TYPE_WORKFLOW_TASK_FAILED
- EVENT_TYPE_ACTIVITY_TASK_SCHEDULED
- EVENT_TYPE_ACTIVITY_TASK_STARTED
- EVENT_TYPE_ACTIVITY_TASK_COMPLETED
- EVENT_TYPE_ACTIVITY_TASK_FAILED
- EVENT_TYPE_ACTIVITY_TASK_TIMED_OUT
- EVENT_TYPE_ACTIVITY_TASK_CANCEL_REQUESTED
- EVENT_TYPE_ACTIVITY_TASK_CANCELED
- EVENT_TYPE_TIMER_STARTED
- EVENT_TYPE_TIMER_FIRED
- EVENT_TYPE_TIMER_CANCELED
- EVENT_TYPE_WORKFLOW_EXECUTION_CANCEL_REQUESTED
- EVENT_TYPE_WORKFLOW_EXECUTION_CANCELED
- EVENT_TYPE_REQUEST_CANCEL_EXTERNAL_WORKFLOW_EXECUTION_INITIATED
- EVENT_TYPE_REQUEST_CANCEL_EXTERNAL_WORKFLOW_EXECUTION_FAILED
- EVENT_TYPE_EXTERNAL_WORKFLOW_EXECUTION_CANCEL_REQUESTED
- EVENT_TYPE_MARKER_RECORDED
- EVENT_TYPE_WORKFLOW_EXECUTION_SIGNALED
- EVENT_TYPE_WORKFLOW_EXECUTION_TERMINATED
- EVENT_TYPE_WORKFLOW_EXECUTION_CONTINUED_AS_NEW
- EVENT_TYPE_START_CHILD_WORKFLOW_EXECUTION_INITIATED
- EVENT_TYPE_START_CHILD_WORKFLOW_EXECUTION_FAILED
- EVENT_TYPE_CHILD_WORKFLOW_EXECUTION_STARTED
- EVENT_TYPE_CHILD_WORKFLOW_EXECUTION_COMPLETED
- EVENT_TYPE_CHILD_WORKFLOW_EXECUTION_FAILED
- EVENT_TYPE_CHILD_WORKFLOW_EXECUTION_CANCELED
- EVENT_TYPE_CHILD_WORKFLOW_EXECUTION_TIMED_OUT
- EVENT_TYPE_CHILD_WORKFLOW_EXECUTION_TERMINATED
- EVENT_TYPE_SIGNAL_EXTERNAL_WORKFLOW_EXECUTION_INITIATED
- EVENT_TYPE_SIGNAL_EXTERNAL_WORKFLOW_EXECUTION_FAILED
- EVENT_TYPE_EXTERNAL_WORKFLOW_EXECUTION_SIGNALED
- EVENT_TYPE_UPSERT_WORKFLOW_SEARCH_ATTRIBUTES
- EVENT_TYPE_WORKFLOW_EXECUTION_UPDATE_ADMITTED
- EVENT_TYPE_WORKFLOW_EXECUTION_UPDATE_ACCEPTED
- EVENT_TYPE_WORKFLOW_EXECUTION_UPDATE_REJECTED
- EVENT_TYPE_WORKFLOW_EXECUTION_UPDATE_COMPLETED
- EVENT_TYPE_WORKFLOW_PROPERTIES_MODIFIED_EXTERNALLY
- EVENT_TYPE_ACTIVITY_PROPERTIES_MODIFIED_EXTERNALLY
- EVENT_TYPE_WORKFLOW_PROPERTIES_MODIFIED
- EVENT_TYPE_NEXUS_OPERATION_SCHEDULED
- EVENT_TYPE_NEXUS_OPERATION_STARTED
- EVENT_TYPE_NEXUS_OPERATION_COMPLETED
- EVENT_TYPE_NEXUS_OPERATION_FAILED
- EVENT_TYPE_NEXUS_OPERATION_CANCELED
- EVENT_TYPE_NEXUS_OPERATION_TIMED_OUT
- EVENT_TYPE_NEXUS_OPERATION_CANCEL_REQUESTED
type: string
format: enum
WorkflowExecution:
type: object
properties:
Expand Down
28 changes: 28 additions & 0 deletions temporal/api/common/v1/message.proto
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ import "google/protobuf/duration.proto";
import "google/protobuf/empty.proto";

import "temporal/api/enums/v1/common.proto";
import "temporal/api/enums/v1/event_type.proto";
import "temporal/api/enums/v1/reset.proto";

message DataBlob {
Expand Down Expand Up @@ -206,3 +207,30 @@ message Callback {
Internal internal = 3;
}
}

// Link can be associated with history events. It might contain information about an external entity
// related to the history event. For example, workflow A makes a Nexus call that starts workflow B:
// in this case, a history event in workflow A could contain a Link to the workflow started event in
// workflow B, and vice-versa.
message Link {
message WorkflowEvent {
message EventReference {
int64 event_id = 1;
temporal.api.enums.v1.EventType event_type = 2;
}

string namespace = 1;
string workflow_id = 2;
string run_id = 3;

// Additional information about the workflow event.
// Eg: the caller workflow can send the history event details that made the Nexus call.
oneof reference {
EventReference event_ref = 100;
}
}

oneof variant {
WorkflowEvent workflow_event = 1;
}
}
2 changes: 2 additions & 0 deletions temporal/api/history/v1/message.proto
Original file line number Diff line number Diff line change
Expand Up @@ -922,6 +922,8 @@ message HistoryEvent {
// * timer_started_event_attributes - summary represents an identifier for the timer for use by
// user interfaces.
temporal.api.sdk.v1.UserMetadata user_metadata = 301;
// Links associated with the event.
repeated temporal.api.common.v1.Link links = 302;
// The event details. The type must match that in `event_type`.
oneof attributes {
WorkflowExecutionStartedEventAttributes workflow_execution_started_event_attributes = 6;
Expand Down
9 changes: 9 additions & 0 deletions temporal/api/nexus/v1/message.proto
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,12 @@ message UnsuccessfulOperationError {
Failure failure = 2;
}

message Link {
// See https://github.com/nexus-rpc/api/blob/main/SPEC.md#links.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This URL does not work. Where can I read about this arbitrary byte array and string type?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was discussed in an internal document, the public doc should be updated too.

string url = 1;
string type = 2;
}

// A request to start an operation.
message StartOperationRequest {
// Name of service to start the operation in.
Expand All @@ -66,6 +72,8 @@ message StartOperationRequest {
temporal.api.common.v1.Payload payload = 5;
// Header that is expected to be attached to the callback request when the operation completes.
map<string, string> callback_header = 6;
// Links contain caller information and can be attached to the operations started by the handler.
repeated Link links = 7;
}

// A request to cancel an operation.
Expand Down Expand Up @@ -106,6 +114,7 @@ message StartOperationResponse {
// The returned ID can be used to reference this operation.
message Async {
string operation_id = 1;
repeated Link links = 2;
}

oneof variant {
Expand Down
2 changes: 2 additions & 0 deletions temporal/api/workflowservice/v1/request_response.proto
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,8 @@ message StartWorkflowExecutionRequest {
// for use by user interfaces to display the fixed as-of-start summary and details of the
// workflow.
temporal.api.sdk.v1.UserMetadata user_metadata = 23;
// Links to be associated with the workflow.
repeated temporal.api.common.v1.Link links = 24;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you confirm this will end up on the started event? What about schedule workflow? What about child workflow? What about signal with start? We should try to make all ways of workflow starting have the same options.

We spent plenty of effort making sure user metadata was propagated in these same kinds of situations. Is everyone sure they don't want to use the user metadata propagation that already exists and just put this inside user metadata?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's added here in the server PR: https://github.com/temporalio/temporal/pull/6420/files#diff-c0867c0ff803c196fa7bc72a15d01812b84ae90eb8b40e71de4ee11d807cfad3R165-R167

At this moment, we're only considering the case of starting new workflow.

Hm, do we want the links to propagate to child workflow, scheduled workflows, etc? I don't think it really make sense.

Copy link
Member

@cretz cretz Aug 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO we should be consistent in the options like these for all ways of starting a workflow. There are 4 main ones:

  • Start workflow
  • Signal with start workflow
  • Schedule workflow
  • Start child workflow

It is important IMO, unless there's a good reason not to in certain cases, for these to all accept the same options.

I don't think it really make sense.

This is a general purpose link option, we shouldn't assume how it'll be used. Same type of thing with user metadata. In fact, arguably this is user metadata and should just be on that message, but that was rejected.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree we should have the option to link in all of these methods. Not blocking this PR but I'd at least want an issue to add it later.

}

message StartWorkflowExecutionResponse {
rodrigozhou marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
Loading