Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSC2762: Allowing widgets to send/receive events #2762

Open
wants to merge 12 commits into
base: old_master
Choose a base branch
from
202 changes: 202 additions & 0 deletions proposals/2762-widget-event-receiving.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
# MSC2762: Allowing widgets to send/receive events

[MSC1236](https://github.com/matrix-org/matrix-doc/issues/1236) originally specified a Widget API
which supports widgets being able to receive specified events from the client, and for widgets to
be able to send more than stickers.

Sticker support is already specified for widgets, though support for text and image events has been
excluded from the initial specification, as has MSC1236's event receiving support. These components
have been excluded from the specification due to lack of documentation and lack of reference
implementation to influence the spec writing process.

This proposal aims to bring the functionality originally proposed by MSC1236 into the widget
specification with the accuracy and implementation validation required by modern MSCs.

## Prerequisite background

Widgets are relatively new to Matrix and so the terminology and behaviour might not be known to all
readers. This section should clarify the components of widgets that are applicable to this MSC without
going on a deep dive into widgets in general.

Widgets are embedded HTML/JS/CSS applications in a client which use the `postMessage` API to talk
to the client. This communication allows widgets to provide enhanced functionality such as sticker
pickers (when applied to a user) or performance dashboards (in rooms).

One of the first things that happens over this communication channel is a "capabilities negotiation"
where the client asks the widget what permissions it wants, and the widget replies with its ideal
set. The client then either decides or asks the user if the permissions requested are okay.

All communication over the channel is done in a simple request/response flow, using actions to
describe the request. For the capabilities negotiation, this would be the client sending the widget
a request with an `action` of `capabilities`, and the widget would respond to that request with a
response object.

The channel in which communication occurs is called a "session", where the session is "established"
after the capabilities negotiation. Sessions can only be terminated by the client.

The Widget API is split into two parts: `toWidget` (client->widget) and `fromWidget` (widget->client).
They are differentiated by where the request originates.

For a bit of background, stickers are gated by an `m.sticker` capability and have a `m.sticker`
action on the `fromWidget` API. If the widget was granted the capability and sent a valid request
to the client, the client would send an `m.sticker` event to the currently viewed room as the
user. This is all a bit confusing due to the naming of all the identifiers, but the principle
is that there's prior art for sending events from widgets.

## Proposal (sending events from widgets)

As mentioned above in the prerequisite background, sticker messages can currently be sent over the
Widget API but other events are not possible. To facilitate sending other event types to the room,
some new capabilities are introduced to allow clients to easily differentiate between custom
capabilities and custom event types (using the `m.sticker` convention could be confusing between a
capability of `com.example.event` and an event type of the same name).

The new capabilities are:

* `m.send.event:<event type>` (eg: `m.send.event:m.room.message`) - Used for sending room messages of
a given type.
* `m.send.state_event:<event type>` (eg: `m.send.state_event:m.room.topic`) - Used for sending state
events of a given type.
turt2live marked this conversation as resolved.
Show resolved Hide resolved

Being able to send other kinds of events (EDUs, account data, etc) is not currently proposed.
turt2live marked this conversation as resolved.
Show resolved Hide resolved

Clients SHOULD automatically deny `m.send.event` and `m.send.state_event` capability requests for
known event types which do not match the descriptor. For example, `m.send.event:m.room.topic` should
be denied, as should `m.send.state_event:m.room.message`.

As with capabilities negotiation already, the user SHOULD be prompted to approve these capabilities
if the widget requests them.

For `m.room.message` specifically, the `msgtype` must be appended to the capability following a `#`.
This ensures that widgets cannot interfere with encryption verification. It is expected that most
widgets looking to use this functionality will request the following:

* `m.send.event:m.room.message#m.notice`
* `m.send.event:m.room.message#m.text`
* `m.send.event:m.room.message#m.emote`

To actually send the event, widgets would use a new `fromWidget` request with action `send_event`
which takes the following shape:

```json
{
"api": "fromWidget",
"widgetId": "20200827_WidgetExample",
"requestid": "generated-id-1234",
"action": "send_event",
"data": {
"state_key": "",
"type": "m.room.topic",
"content": {
"topic": "Hello world!"
}
}
}
```

Under `data`, the `state_key` is omitted if the widget is not sending a state event. The other
properties of `data` are required.

The client is responsible for encrypting the event before sending, if required by the room. The widget
should not need to be made aware of encryption or have to encrypt events.

If the widget did not get approved for the capability required to send the event, the client MUST
send an error response (as required currently by the capabilities system for widgets). If the widget
was approved, the client MUST only send the event to the room the user is currently viewing.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"currently viewing" sounds like timing attacks waiting to happen. Should it be the room where the widget was spawned? Or is it expected that there can be "account widgets" that work cross-room? In that case we need protocols to ensure that delayed events don't end up in the wrong room.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are account widgets too. This MSC should probably do a better job of describing how room widgets work, which is indeed limiting them to where they are added.

The opportunity for a timing attack is so slim it's really not worth the extra overhead. If a stickerpicker sends an event and immediately changes room, the sticker might go to the wrong room, but the user would have had to switch in under 10ms in most cases. This is not a human-approachable response time.


The client SHOULD NOT modify the `type`, `state_key`, or `content` of the request unless required for
encryption. The widget is responsible for producing valid events - the client MUST pass through any
errors to the widget using the standard error response in the Widget API.
turt2live marked this conversation as resolved.
Show resolved Hide resolved

For added clarity, the client picks either the `/send` or `/state` endpoint to use on the homeserver
depending on the presence of a `state_key` in the request data. The client then forms a request using
the `type`, `state_key`, and `content` by matching those against the endpoint's parameters, after
encryption if required.

If the event is successfully sent by the client, the client sends the following response:

```json
{
"api": "fromWidget",
"widgetId": "20200827_WidgetExample",
"requestid": "generated-id-1234",
"action": "send_event",
"data": {
"state_key": "",
"type": "m.room.topic",
"content": {
"topic": "Hello world!"
}
},
"response": {
"room_id": "!room:example.org",
"event_id": "$example"
}
}
```

*Note: Widget API responses are a clone of the request with an added `response` field.*

Both fields of the `response` are required and represent the room ID in which the event was sent,
and the event ID of that event.

With this new approach, the `m.sticker` capability and associated action are deprecated in favour of
this MSC. If this proposal is able to land in the specification before the widgets spec has a first
release, the `m.sticker` approach described in the prerequisite background section is not to be
included in the release (existing clients may still support it for legacy purposes).

## Proposal (receiving events in a widget)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this section needs to specify behaviour for encrypted events.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should be covered somewhere in this proposal: the widget always receives decrypted events, and always sends unencrypted (to be encrypted by the client).

If you mean toDevice messages, those are #3819


In addition to being able to send events into the room, some widgets have an interest in reacting
to particular events that appear in the room. Using a similar approach to the sending of events,
a new capability matching `m.receive.event:<event type>` and `m.receive.state_event:<event type>`
are introduced, with the same formatting requirements as the `m.send.event` and `m.send.state_event`
capabilities above (ie: `m.receive.event:m.room.message#m.text`).

For each event type requested and approved, the client sends a `toWidget` request with action `event`
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should local echos be sent to widgets? I.e, if a widget sends an event, should this event be sent back to them?

Current behavior by Element is to echo an event after the send request to the homeserver succeeds. This is sent along with a response to the send message sent by the widget. There is no way for widgets to ignore local echos unless they maintain a list of IDs that they have sent and ignore events with those IDs when they are received.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they shouldn't receive the local echo but they should receive the server's echo (if the widget requested the capability to do so)

Copy link

@toger5 toger5 Dec 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would really like to have local echos for the events. Would it be beneficial to include local echos in this msc? Or can this be achieved easier in some (future) widget-sdk build on top of the matrix-widget-api and we can keep this msc simpler?

is sent to the widget with the `data` being the event itself. For example:

```json
{
"api": "fromWidget",
turt2live marked this conversation as resolved.
Show resolved Hide resolved
"widgetId": "20200827_WidgetExample",
"requestid": "generated-id-1234",
"action": "send_event",
"data": {
"type": "m.room.topic",
"sender": "@alice:example.org",
"event_id": "$example",
"room_id": "!room:example.org",
"state_key": "",
"origin_server_ts": 1574383781154,
"content": {
"topic": "Hello world!"
},
"unsigned": {
"age": 12345
}
}
}
```

The widget acknowledges receipt of this request with an empty `response` object.

The client SHOULD only send events which were received by the client *after* the session has been
established with the widget (after the widget's capabilities are negotiated). Clients are expected
to apply the same semantics as the send event capabilities: widgets don't receive `m.emote` msgtypes
unless they asked for it (and were approved), and they receive *decrypted* events.

## Security considerations

Because the widget can implicitly decrypt room history, it is absolutely imperative that clients
prompt for permission to use these capabilities even though the capabilities negotation does not
require this to be done. Clients which approve the capabilities proposed by this MSC without
asking the user first are strongly frowned upon. There are very few use cases where not asking for
the user's permission is valid.
turt2live marked this conversation as resolved.
Show resolved Hide resolved

## Unstable prefix

While this MSC is not present in the spec, clients and widgets should:

* Use `org.matrix.msc2762.` in place of `m.` in all new identifiers of this MSC.
* Only call/support the `action`s if an API version of `org.matrix.msc2762` is advertised.