Initial/Common Connection Container Proposal

[egekorkan]

First just moving the existing points from HackMD. The comments that were given in yesterday's call were:

• @danielpeintner: The TD got complex for implementations and humans at the same time.
• @egekorkan : This should be used in complex TDs. A simple TD version should be provided. I will also provide the "expanded" version. (This is why this PR is now a draft)
• @lu-zero : The connection is a form in the end. If we keep it that way, the parser should be fine.

[TallTed]

small, editorial

[benfrancis]

I like the general direction this is going in.

I was curious how this would work for WebSockets, so as an exercise I tried to re-write the example Thing Description from my Web Thing Protocol WebSocket Sub-protocol strawman proposal. This is what I came up with:

{
  "@context": [
    "https://www.w3.org/2022/wot/td/v2.0"
  ],
  "id": "https://myweblamp.io",
  "title": "My Lamp",
  "connections": {
    "websocket": {
      "href": "wss://myweblamp.io",
      "subprotocol": "webthingprotocol",
      "security": "oauth2",
      "reusable": true
    }
  },
  "connection": "websocket",
  "securityDefinitions": {
    "oauth2": {
      "scheme": "oauth2",
      "flow": "code",
      "authorization": "https://myweblamp.io/oauth/authorize",
      "token": "https://myweblamp.io/oauth/token",
      "name": "jwt",
      "in": "query"
    }
  },
  "security": "oauth2",
  "properties": {
    "on": {
      "type": "boolean",
      "title": "On/Off",
      "description": "Whether the lamp is turned on",
      "forms": [
        {
          "op": [
            "readproperty",
            "writeproperty",
            "observeproperty",
            "unobserveproperty"
          ]
        }
      ]
    },
    "level": {
      "type": "integer",
      "title": "Brightness",
      "description": "The level of light from 0-100",
      "unit": "percent",
      "minimum": 0,
      "maximum": 100,
      "forms": [
        {
          "op": [
            "readproperty",
            "writeproperty",
            "observeproperty",
            "unobserveproperty"
          ]
        }
      ]
    }
  },
  "actions": {
    "fade": {
      "title": "Fade",
      "description": "Fade the lamp to a given level",
      "input": {
        "type": "object",
        "properties": {
          "level": {
            "type": "integer",
            "minimum": 0,
            "maximum": 100,
            "unit": "percent"
          },
          "duration": {
            "type": "integer",
            "minimum": 0,
            "unit": "milliseconds"
          }
        }
      },
      "forms": [
        {
          "op": [
            "invokeaction",
            "queryaction",
            "cancelaction"
          ]
        }
      ]
    }
  },
  "events": {
    "overheated": {
      "title": "Overheated",
      "data": {
        "type": "number",
        "unit": "degree celsius"
      },
      "description": "The lamp has exceeded its safe operating temperature",
      "forms": [
        {
          "op": [
            "subscribeevent",
            "unsubscribeevent"
          ]
        }
      ]
    }
  },
  "forms": [
    {
      "op": [
        "readallproperties",
        "readmultipleproperties",
        "writeallproperties",
        "writemultipleproperties",
        "observeallproperties",
        "unobserveallproperties",
        "queryallactions",
        "subscribeallevents",
        "unsubscribeallevents"
      ]
    }
  ]
}

Have I understood Ege's proposal correctly? If so feel free to use this example (or tweak it).

A couple of observations:

• I don't mind if the new member is called "connections" or "bases" (they are essentially form defaults), but I do question the idea of replacing subprotocol member of forms with protocol because these are still distinct concepts. E.g. In the example above, would protocol be wss or webthingprotocol? URIs are already a very well established way of serialising protocol connection information* (e.g. URI scheme) and I don't think that needs to be reinvented with a separate protocol member. I do think a connection/base definition needs a subprotocol member though.
• Just to highlight that a default href is semantically different to a base since in one case the default is replaced and in the other a base is extended. It needs to be possible to do both. A URI can be provided in a connection/base definition which is treated as a base URI but can be overridden with an absolute URI in a form. If the term "bases" is used then href is probably OK, but if the term "connections" is used perhaps href needs to be called base instead, to make this more explicit.

* In fact you might argue that URIs are the fundamental concept that makes the web the web and that linkability is super important.

[lu-zero]

• I don't mind if the new member is called "connections" or "bases" (they are essentially form defaults), but I do question the idea of replacing subprotocol member of forms with protocol because these are still distinct concepts. E.g. In the example above, would protocol be wss or webthingprotocol? URIs are already a very well established way of serialising protocol connection information* (e.g. URI scheme) and I don't think that needs to be reinvented with a separate protocol member. I do think a connection/base definition needs a subprotocol member though.

It isn't as clear as it were back in time :)

• http:// is a subprotocol of tcp or udp?
• https:// is a subprotocol of which version of tls?
• how to describe socket-io via websocket via http3 using mTLS with a specific CA ?

My proposal is to split the two concerns:

• one is having stackable form defaults (bases)
• the other is to have a place to keep both protocol configurations and connection configurations

This way you may say "protocol": "webthingprotocol" with webthingprotocol implying over websocket over anything both consumer and thing support, at least http-1.1 or you may restrict it more by being explicit.

[danielpeintner]

Just a side-comment:
Since consistency is important I would like to see the term connections renamed to connectionDefinitions. Doing so would nicely align it with the tuple security / securityDefinitions.

[benfrancis]

@lu-zero wrote:

It isn't as clear as it were back in time :)

• http:// is a subprotocol of tcp or udp?
• https:// is a subprotocol of which version of tls?
• how to describe socket-io via websocket via http3 using mTLS with a specific CA ?

Of course all protocols exist somewhere in a protocol stack but in the context of the World Wide Web, and the Web of Things, protocols are identified by URI schemes. URI schemes are (supposed to be) centrally registered with IANA to avoid name conflicts.

Anything defined on top of that layer can be considered a "sub-protocol", which is a formally defined concept in WebSockets (which also has a central IANA registry) but is also less formally used by WoT binding templates for other protocols and "should be understood together with the protocol indicated in href of the forms (or the base)".

A fundamental part of the Web of Things architecture is that protocol bindings are created using hypermedia controls, which rely on URIs. If we start to split protocol identifiers out of URIs into a separate mechanism then it's a step towards making an already large problem space even larger, by opening the door to protocols which don't have a URI scheme and are not included in the IANA registry. This is a step towards facilitating bindings for directly describing non-web and non-Internet protocols like Zigbee and Z-Wave which have so far been considered out of scope of the Web of Things. I feel strongly that the working group should not go in that direction, because the scope of the Web of Things already feels unreasonably large (especially with people pushing the boundaries by inventing informal URI schemes for protocols which don't natively define one). At least anything which goes in this direction should not be called the Web of Things.

My proposal is to split the two concerns:
* one is having stackable form defaults (bases)
* the other is to have a place to keep both protocol configurations and connection configurations

That's fine, but protocol bindings should use URIs and the scheme part of a URI (together with an optional subprotocol) unambiguously identifies the protocol being used.

This way you may say "protocol": "webthingprotocol" with webthingprotocol implying over websocket over anything both consumer and thing support, at least http-1.1 or you may restrict it more by being explicit.

In order to avoid ambiguity this would surely require the WoT Working Group to maintain its own protocol registry, in parallel to the one defined by IANA, which seems like a bad idea to me.

[lu-zero]

@lu-zero wrote:

It isn't as clear as it were back in time :)

• http:// is a subprotocol of tcp or udp?
• https:// is a subprotocol of which version of tls?
• how to describe socket-io via websocket via http3 using mTLS with a specific CA ?

Anything defined on top of that layer can be considered a "sub-protocol", which is a formally defined concept in WebSockets (which also has a central IANA registry) but is also less formally used by WoT binding templates for other protocols and "should be understood together with the protocol indicated in href of the forms (or the base)".

I'd rather spare 3 letters and potentially have a place different from the Form to hold the protocol and connection configuration details.

A fundamental part of the Web of Things architecture is that protocol bindings are created using hypermedia controls, which rely on URIs. If we start to split protocol identifiers out of URIs into a separate mechanism then it's a step towards making an already large problem space even larger, by opening the door to protocols which don't have a URI scheme and are not included in the IANA registry. This is a step towards facilitating bindings for directly describing non-web and non-Internet protocols like Zigbee and Z-Wave which have so far been considered out of scope of the Web of Things. I feel strongly that the working group should not go in that direction, because the scope of the Web of Things already feels unreasonably large (especially with people pushing the boundaries by inventing informal URI schemes for protocols which don't natively define one). At least anything which goes in this direction should not be called the Web of Things.

If I want to describe a Thing that uses mTLS and implements only http/3 I need a place to store that information. We already have the security* terms that do that so it is just expanding their scope and/or rename the terms.

Having per-scheme defaults is a possible the situation in which you have multiple protocols and each of them have different security schemes that may apply.

Alternatively we can leverage the stackable defaults approach and also remove the security-related defaults since they would be redundant.

My proposal is to split the two concerns:

• one is having stackable form defaults (bases)
• the other is to have a place to keep both protocol configurations and connection configurations

That's fine, but protocol bindings should use URIs and the scheme part of a URI (together with an optional subprotocol) unambiguously identifies the protocol being used.

No disagreement on this, but you have situations in which you need and have to put additional details.
Right now we already have the situation of binding-specific terms for the "protocol-specific verb", and if we want to go that way, stackable form defaults are sufficient.

But if we want to have more structure, and I'd argue the first step is to unify htv:methodName, cov:method, mqv:controlPacket, to have a single term holder.

This way you may say "protocol": "webthingprotocol" with webthingprotocol implying over websocket over anything both consumer and thing support, at least http-1.1 or you may restrict it more by being explicit.

In order to avoid ambiguity this would surely require the WoT Working Group to maintain its own protocol registry, in parallel to the one defined by IANA, which seems like a bad idea to me.

The current consensus is that in order to have a binding in our registry, the uri schemes it uses have to be registered with IANA.

[egekorkan]

This will be discussed in today's meeting. I observe that there is a discussion on the relationship between the definitions of protocol, href, URI scheme, subprotocol. For the first phase of the discussion, I would avoid going in that direction. Those can be defined under that reusable container with keywords we are free to define later on.

[egekorkan]

Call of today:

• The initial idea is to start with defining the container and come back to the discussion of redefining how protocols are defined in a TD and how the physical "connection" is reused and signalled in the TD.
• There is a question to the group on whether we should put connectionDefinitions, securityDefinitions and schemaDefinitions into one object, let's say commonDefinitions that can be referred to other places in the TD instance. This would be a lot like the components of the OpenAPI specs. See https://spec.openapis.org/oas/v3.1.0.html#components-object . While this is a "clean" design, it would require quite a bit of more processing effort from the Consumers. Of course, usage would not be mandatory and would make sense to use in complicated TDs.

[egekorkan]

Call of 10th of October:

• The mechanism overall is fine.
• The name should be picked. Some proposals: connections and connection, bases and base, formDefinitions and form (or defaultForm). This will be rediscussed.
• Luca: The form compositionality comes for free if we define the container values as form objects.
  • Ege: I feel weird about op and circularity. I see a bit of the point that op does not have to be defined but if it is defined with writeproperty, actions need to override it. So it makes sense for "property-heavy` TDs.
  • Cris: Circular dependencies is a bit of a concern

@relu91 and @lu-zero feel free to comment your points.

[lu-zero]

Call of 10th of October:

* The mechanism overall is fine.

* The name should be picked. Some proposals: connections and connection, bases and base, formDefinitions and form (or defaultForm). This will be rediscussed.

* Luca: The form compositionality comes for free if we define the container values as form objects.
  
  * Ege: I feel weird about op and circularity. I see a bit of the point that `op` does not have to be defined but if it is defined with `writeproperty`, actions need to override it. So it makes sense for "property-heavy` TDs.

op can be left unset as any other field, but a TD can have it set if most of its affordances have the same op.
The problem with circular references has to be tackled already because of ComboSecurityScheme so there is no additional logic needed, just has to be applied to one more item.

  * Cris: Circular dependencies is a bit of a concern

@relu91 and @lu-zero feel free to comment your points.

[egekorkan]

• Form Circularity:
  • @egekorkan I am more ok on this direction. If we add instructions on avoiding invalid TDs, it should be fine. This is nice for defining defaults for write and defaults for read.
• Reusability of Connection:
  • @lu-zero :Server may not have resources and close. Client may not need to keep it open. Cases where the Thing wants the Consumer to play nice. None of the protocols we currently have, have such a case where there is no graceful closing in the protocol level. Also it is protocol-specific.
• More Default Containers:
  • One proposal is added during the call. It will be revisited.
  • Should security in the root level be conserved
  • How do we handle "inline security" improvements that were collected by @JKRhb ?
  • The names of the new terms should be discussed. Can we have just security in the new container since there is no more naming conflict? @egekorkan says there can be namespace conflicts in the ontology.
  • The parsing algorithm should be specified thoroughly.

[benfrancis]

It would be nice if it was possible to provide inline connection metadata in the connection member in the case that there is only one connection, instead of the verbosity of separately defining connectionDefinitions and connection. This would be the same as the proposal to allow inline security definitions in 2.0.

[benfrancis]

Oh man, I don't look forward to trying to parse connection definitions which reference each other. Is this completely necessary? It's obviously a trade-off between verbosity in the Thing Description and parser complexity (and human readability), but you can't currently do this with securityDefinitions.

[lu-zero]

I want to have stackable defaults for forms and a container for the protocol specific terms. if there is consensus to conflate the two so be it, but...

[egekorkan]

I think this can be added later on based on the needs of the TD designers. This only makes sense if there are two defaults defined, e.g. DELETE for reading and POST for writing to a property and you have a many properties in the TD.

[benfrancis]

"definitions"?

[egekorkan]

Also makes sense. I will present it :)

[lu-zero]

This item hadn't been discussed yet, it boils down to either

• remove security and related terms and use Form as container for what we have in the securityDefinitions. (less tidy IMHO)
• rename security to connection and securityDefinitions to connectionDefinitions and use it to contain the connection setup terms.

"connectionDefinitions": { "no_sec+http3": {
    "scheme": "nosec",
    "htv:version": "3",
    "transport": "quick",
    "tls:cyphers": "...
  }
}

"formDefinitions": {
  "my_read_only_http3_properties": {
    "op": "readproperty",
    "connection": "no_sec+http3",
    "htv:methodName": "...
  } 
}

The reason why is that security is just a specialized kind of connection setup covering from trasport (tls) to application (oauth).

[egekorkan]

• MQTT broker related information (uri, mqtt version etc,) goes in the first container in this case.
• The second container is purely setting the defaults for forms. Can be renamed to formDefinitions
• Discussion: Should the keys of connectionDefinitions be limited to allowed/known uri schemes?
• Discussion: Are links influenced by these?

[egekorkan]

I will prepare a final example with the proposal from Luca of splitting the connection (security and other stuff) and defaults for forms. There will be also a simple markdown table with types etc. This can be used in the plugfest for testing and getting real-world experience.