Implementation specific context

[matthias-pichler]

Idea

Runtime expressions already specify a list of Arguments that are available. These are all defined by:

1. the workflow definition ($task, $workflow)
2. the execution input ($input, context)
3. secrets ($secrets)

If I were to take my workflow and move it from one implementation to another I would expect that all these are exactly the same since they are implementation agnostic.

I propose to add a new argument called $runtime (alternative names could be $implementation or $engine) that can be filled with implementation runtime specific context such as:

• tenantId, accountId, organization: useful for SaaS platforms using serverless workflow
• version: the version of the runtime (e.g. 0.4.8)
• runtimeName: the name of the implementation (e.g. Synapse, Sonanta)
• dryRun, featureFlags: for implementation specific features such as

Why is this needed

I have a case where I want workflow authors to be able to access their tenantId in every workflow as well as the fact if this is a test run.

using a specified expression argument like $runtime allows to quickly identify which workflows rely on implementation specific information and which are fully platform agnostic.

Alternatives

Non standard argument

Nothing is stopping implementors to expose a non standard argument like $runtime or $impl to their authors. This argument however would be harder to spot if it differs between implementations.

Secrets

All these things could be exposed using secrets. However:

• some of them are not secrets which makes this weird
• authors would have to specify them under use every time
• it does not carry the information that these are platform dependant

extending $context

Some runtimes might choose to extend the existing $context argument which would mean that the content of $context would differ from the result of the last export.as and if not expected could send invalid data in tasks

document: {}
do:
  - getPet:
      call: http
      with: {}
      export:
        as:
          petId: ${ .pet.id }
  - adoptPet:
      call: http
      with:
        body: ${ $context } # this would then suddenly contain unexpected data such as `$context.runtime.tenantId`

[matthias-pichler]

Interested to hear your thoughts @cdavernas @JBBianchi @fjtirado @ricardozanini

[cdavernas]

I propose to add a new argument called $runtime (alternative names could be $implementation or $engine) that can be filled with implementation runtime specific context

I think it's a great idea! It should IMHO be documented that the data and behavior encapsulated within that new argument might break the workflow when carried across implementations, as it's tightly bound to a specific runtime.

We also could define some common properties that all runtimes should implement, such as name, version, os, ...

I'd add that a viable alternative would be to add those data to a $constants argument. I do prefer your first proposal though, and would definitly avoid adding them to either $context or $secrets, for the reasons you enumerated.

[JBBianchi]

The distinction between standard and implementation-specific features can be somewhat unclear, but overall, I agree. Ultimately, this would make implementers more aware of commonly used runtime variables, may they provide them or not.

I'm in favor of a specific variable like $runtime. I don't think messing up with author-defined variables such as $context or $secrets is a good idea, it would introduce confusion.

[fjtirado]

@matthias-pichler-warrify
I actually prefer extending $context. I do not think sending extra data when the context change will cause any issue.

[matthias-pichler]

In my example, if the pet api has any input validation the request adoptPet would fail in one implementation and might not in another.

For this reason I am against extending $context. It also defeats the whole purpose of my proposal because it would not be immediately obvious which workflows are runtime agnostic and which not.

Tooling could even show hints for any workflows using $runtime that they are not universally compatible

[cdavernas]

@matthias-pichler-warrify I had not thought of the ability to determine whether or not a workflow is runtime specific. Great catch!

[fjtirado]

Or as intermediate proposal, add custom property to context (and put everything custom there)

[cdavernas]

Great idea, I personally love it!

[matthias-pichler]

A dedicated property on $context could work because it has a fixed name. I would however prefer to separate it completely from context because it would mean that $context is no longer the result of export.as. (it would be the result of export.as and some extra). This would also make the export.schema validation weird ... do users have to specify the custom property in theri export.schema?

Also $context could technically be anything (not just an object) e.g. a boolean ... how would we define a property on a boolean field?

[cdavernas]

@matthias-pichler-warrify You are perfectly right. Seems that we should indeed use a separate argument, mainly because, as you said, context can take any shape or form, and we should definitely not restrict it to a map type

[fjtirado]

yes, and now I think it more quietly, I agree with you guys, Lets define it as a separate property

[ricardozanini]

So since we have an agreement (+1 from my side too), can we have a summary, mark the answer as the correct one, and create an issue to move forward @matthias-pichler-warrify?

PS: I can do it, but since it's your requirement, I think it's fair if you do it.

[matthias-pichler]

Summary:

We will introduce a dedicated $runtime argument that will contain a set of default/standardized properties about the implementation (e.g. name & version) as well as a dedicated field (e.g. $runtime.metadata) to hold non-standard implementation specific values. Usage of this property will make it immediately visible that a given workflow definition might not be runtime agnostic.

Issue: #950
PR: #952