Create pushrules with non-ascii ruleId fails on Python 2

[spantaleev]

Creating a new keyword notification seems to fail on Python 2 when the keyword contains non-ASCII text.

When trying to add a keyword called тест (cyrillic), Riot would send a PUT request with a payload of {"actions":["notify",{"set_tweak":"sound","value":"default"},{"set_tweak":"highlight"}],"pattern":"тест"} to /_matrix/client/r0/pushrules/global/content/%D1%82%D0%B5%D1%81%D1%82.

This fails with the following exception:

File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
result = g.send(result)
File "/usr/local/lib/python2.7/site-packages/synapse/http/server.py", line 316, in _async_render
callback_return = yield callback(request, **kwargs)
File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1613, in unwindGenerator
return _cancellableInlineCallbacks(gen)
File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
_inlineCallbacks(None, g, status)
--- <exception caught here> ---
File "/usr/local/lib/python2.7/site-packages/synapse/http/server.py", line 81, in wrapped_request_handler
yield h(self, request)
File "/usr/local/lib/python2.7/site-packages/synapse/http/server.py", line 316, in _async_render
callback_return = yield callback(request, **kwargs)
File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
result = g.send(result)
File "/usr/local/lib/python2.7/site-packages/synapse/rest/client/v1/push_rule.py", line 45, in on_PUT
spec = _rule_spec_from_path(request.postpath)
File "/usr/local/lib/python2.7/site-packages/synapse/rest/client/v1/push_rule.py", line 212, in _rule_spec_from_path
rule_id = path[0].decode('ascii')
exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xd1 in position 0: ordinal not in range(128)

Looks like Riot is using the new keyword for both the pattern (naturally..) and for the actual ruleId.

I don't know whether the {ruleId} can be anything according to the spec.. If it should be a safe ASCII string, then this may be a bug in Riot. Still, it's a bug in Synapse too, because failing with an exception doesn't sound ideal.

Looks like this is a regression since 02aa418 (#3823) / Synapse 0.33.5.

I'm reproducing it on Synapse 0.33.8 (Python 2). I haven't tested whether it works fine in Python 3.

[hawkowl]

Thanks for the report @spantaleev, I think I know what the problem is.

(For interest: it's because Twisted unescapes the path before giving it to us, rather than giving us the URL escaped version, which is ASCII safe. My bad.)

[richvdh]

fixed in #4248