confused about Error Handling

[globalflea]

There is OnError in the various States with fields OnError.ErrorRef/s. I understand that errorRefs point to the Workflow.Errors. I am unsure how this Workflow.Errors is to be used.

I am thinking that when an Action returns an error, i just check against the State's OnError.ErrorRef and if found, i would just handle the resulting Transition or End. Or am i supposed to take the error code returned by the Action and match it against Workflow.Errors.Code to convert a code into an unique error-name? if so, is Workflow.Errors.Code supposed to be unique?

If Workflow.Errors.Code is supposed to be unique, then it seems rather difficult to distinguish "404" (as in the example) which may be returned by different servers or external endpoints

[tsurdilo]

In SW we opted to go with domain-specific error handling in order for it to be portable between different runtimes written in different languages and using different libraries etc etc.

There has to be a layer of custom mapping on a runtime side between the name/code of an error definition in the DSL and an actual error/exception that the runtime catches when doing things like handling timeouts/performing function invocations in whatever shape/form and stuff like that. So for example in DSL if you have:

  "errors": [
     {
      "name": "ServiceNotAvailable",
      "code": "503"
     }
  ]

Runtime impls should have a mapping that maps the name and/or code defined here to a particular exception, for example in this case could be mapped to idk, a GoogleJsonResponseException that has a statusCode of 503.
The error code is not unique, the error name param should be. Hope this helps.

[globalflea]

i may be missing something. Since Name is unique, it's easy to map from Name to error Code. But different external services may return same error code, then it becomes impossible to map from external service error code to a specific workflow error.

Not sure if it makes sense, but for example, if Service-A returns 503 and Service-B also returns 503, they will both be mapped to the same workflow error definition, since there is no way to distinguish between the two - i.e. they will both be mapped to "ServiceNotAvailable" - e.g. Service-A = Payment Service, Service-B = Inventory Service, how i would want to transition may depend on the Service generating the 503.

Maybe the flow would go something like this:
Ext-Svc-A -> (503) -> Action-A -> (ServiceNotAvailable) -> State-A ==> next state etc
Ext-Svc-B -> (503) -> Action-B -> (ServiceNotAvailable) -> State-A ==> next state etc

Maybe the question is whether any specific state will need to distinguish between two (or more) "ServiceNotAvailable" from different external services?

[tsurdilo]

https://github.com/serverlessworkflow/specification/blob/main/schema/errors.json#L49 -> only name is required param.

[mswiderski]

I don't think that the same name for an error is of issue as you define onErrors per state that deals with transition. With that you have enough control what to do with the same error definition. Meaning if state A calls Payment service then you define transition to deal with payment service 503 errors. Similar you do that on State B that calls Inventory service.

The missing part is having onErrors on action itself instead of only on state as there might be multiple actions in single state and then the problem with handling that properly will show up.

In addition I am also wonder why there is retries on action but not on state? Seems like it got bit mixed up since 0.6 (that I was looking into last time or I simply missed some important reasons for that).

Would it make sense to unify both? by that I mean:

• error definition has name and code - to me code is also required at least from runtime perspective as usually name is used by DSL to link it within the workflow definition but then error codes are provided by runtime or underlying protocol e.g. HTTP
• onErrors is available both on state and on action
• onErrors - links to error definition and retry definition plus defined transition or end - retry and errors for me are really tightly coupled because you want to retry when an error occurred so to retry you need to have an error

With that I think we should have most of the use cases covered without limits. What do you guys think?

[tsurdilo]

think maybe we need to look at this (having onErrors on actions) closer cause for parallel state then we would need that on action+branch+state level and it can become super confusing imo (confusing..or super powerful, not sure yet)

[globalflea]

still not getting the "usefulness" of the Workflow.Errors. i think when an Action encounters an error (e.g. 503) - it decides how to encode it into a "meaningful" error that will trigger the right transition. Since the Action (actually the developer) decides what sort of domain-error to return, why do i need the Workflow.Errors? Similarly, when the State receives the domain-error, it just checks against the ErrorRef/s and perform the transition, etc. So i still don't see where Workflow.Errors come in?

[mswiderski]

is the idea then to allow action itself to handle errors and any that are not handled to propagate to onErrors on the state level?

I'd say yes

if so i think we still need it. if an action defines for example nonRetryableError "ABC" and does not handle it in its on onErrors, but on the state level it is handled with a retry ref defined, that retry ref needs to be ignored imo.

that's the point I am trying to make, to have errors that define retries and to not have standalone retries only as it seems to bring not needed complexity. As to retry you need an error any way. Then if you don't handle the error that means it is not retryable error. Regardless of what level it is defined on (action/state etc)

Or is the idea that the state-level error handling/retries do not care about action retryable/nonretryable errors and can specify their own state-level retries (then i think this is a good point). Still tho it is kinda counter intuitive imo that an action says "do not retry me" but the state says "retry whole state" and the action is invoked again :)

this won't be an issue if errors and retries are defined as one thing as once you handle error you have option to specify retry logic - if not defined you simple state that it is not retryable error.

[mswiderski]

think maybe we need to look at this (having onErrors on actions) closer cause for parallel state then we would need that on action+branch+state level and it can become super confusing imo (confusing..or super powerful, not sure yet)

I think having error handling only on action and state should be enough. Parallel state's branches would handle errors on individual actions only which should be enough in most of the cases imo

[mswiderski]

still not getting the "usefulness" of the Workflow.Errors. i think when an Action encounters an error (e.g. 503) - it decides how to encode it into a "meaningful" error that will trigger the right transition. Since the Action (actually the developer) decides what sort of domain-error to return, why do i need the Workflow.Errors? Similarly, when the State receives the domain-error, it just checks against the ErrorRef/s and perform the transition, etc. So i still don't see where Workflow.Errors come in?

I think it was mainly designed for reusability so you don't have to define the same errors multiple times and just reference it.

[tsurdilo]

@mswiderski

this won't be an issue if errors and retries are defined as one thing as once you handle error you have option to specify retry logic

don't know if i'd like to tie retry reference to an error definition in the dsl. they should be able to be defined individually and reused individually.
also think we should allow for action retries w/o explicit error handling (on action level). for example, an action that defines retries and has nonRetryableErrors=X should mean that it's retried based on its policy for "any" errors except X. i think this approach would significantly reduce the amount of json/yaml in the spec.

[mswiderski]

don't know if i'd like to tie retry reference to an error definition in the dsl. they should be able to be defined individually and reused individually.

that's not what I mean - I meant that on errors should reference both error definition and retry definition so from action/state error handling point of view we have both in single place.

also think we should allow for action retries w/o explicit error handling (on action level). for example, an action that defines retries and has nonRetryableErrors=X should mean that it's retried based on its policy for "any" errors except X. i think this approach would significantly reduce the amount of json/yaml in the spec.

this is very good point, haven't thought about this. Though at the same time it makes a point that errors and retries are related to each other. So maybe having this logic as part of onErrors rather than separate attributes would make sense too?
I mean there is no need to have the retryableErrors attribute at all as it would pretty much be the same as defining onErrors and then having option to define nonRetryableErrors as part of onErrors would make it clearly scoped. That said, on errors you list what errors to handle (including retry ref) and ignore others. On top of that you can also set what errors should never be handled and by that retried to simplify the definition of it to not have to multiply the errors in state/action.

[mswiderski]

one additional thought that there is no way (at least as far as I understand it) to define retry logic on state as onErrors does not allow to configure it. Or did I miss something?

If that's the case it is another argument to unify onErrors for both state and actions which would handle both errors and retry requirements.

[tsurdilo]

Maybe we could decide if state-retries is something we really want to do.
This was the case in v0.6: https://serverlessworkflow.io/schemas/0.6/workflow.json
but per request we moved retry defs on action level, as otherwise we could not control individual action retries.

I don't see a good use case for retrying the whole state and if we wanted to combine these two maybe its better to move error handling to be on action level as well?

[mswiderski]

personally I think it makes sense to have retries and errors on both levels (state and action) and that's why I think combining errors with retries into single configuration makes the most sense.

[ricardozanini]

Here's our new take on the error handling in the new DSL if you want to explore: https://github.com/serverlessworkflow/specification/blob/main/dsl.md#fault-tolerance