Copying and pasting pills can result in UTF-16-mangled BOM insertion (Firefox/Linux)

[eternaleye]

Description

When copying and pasting pills, under certain circumstances a Unicode BOM (mangled by transformation to UTF-16) may be inserted into the message.

Steps to reproduce

• Tab-complete a user in a room at the beginning of the line (e.g. eternaleye)
• Type a word after the completion (e.g. Testing)
• Control-A
• Control-X
• Control-V
• Send message

The message will have converted the pill to plain text, and inserted a UTF-16-mangled BOM at the beginning of the line, e.g.

\xef\xbb\xbfeternaleye: Testing

If the BOM issue did not occur, this should instead be:

eternaleye: Testing

Optimally, the pill itself would be preserved, but that should (may already?) be a separate issue.

Exact source dump of such a message (Note: the BOM is not visible in the render, but can be seen via e.g. xxd):

{
  "content": {
    "body": "Matthew: Testing",
    "msgtype": "m.text"
  },
  "origin_server_ts": 1574804370125,
  "sender": "@eternaleye:matrix.org",
  "type": "m.room.message",
  "unsigned": {
    "age": 1578,
    "transaction_id": "m1574804369304.47"
  },
  "event_id": "$ygkxvbA5dtoIpBHw_i8Up9xrpkVP_S9gesSAJkqaMYg",
  "room_id": "!YTvKGNlinIzlkMTVRl:matrix.org"
}

Version information

• Platform: Web
• Browser: Firefox 66.0.3
• OS: Exherbo Linux, kernel 4.18.20
• URL: riot.im/develop
• matrix-react-sdk version: <local>
• riot-web version: caeae69-react-46846fcfd0b7-js-fcbfaac1fdd9
• olm version: 3.1.3

[eternaleye]

Another user reports experiencing the bug under Chrome on Linux; will update with the details once they're back at the computer in question.