describe cursor_pos ambiguity and bump protocol to 5.2 #262
Conversation
docs/messaging.rst
Outdated
| Many frontends, especially those implemented in javascript, | ||
| reported cursor_pos as the interpreter's string index, | ||
| which is not the same as the unicode character offset if the interpreter uses UTF-16 (e.g. javascript or Python 2 on macOS), | ||
| which stores astral-plane characters such as ``𝐚`` as surrogate pairs, |
There was a problem hiding this comment.
precise U+1D41A ? Because I'm going to assume somewhere it's not going to render correctly ?
docs/messaging.rst
Outdated
| - JupyterLab | ||
| - nteract | ||
| - CoCalc | ||
| - Jupyter Console and QtConsole with Python 2 on macOS |
There was a problem hiding this comment.
Confirmed on Windows Python 2 as well.
docs/messaging.rst
Outdated
| .. versionchanged:: 5.2 | ||
|
|
||
| Due to a widespread bug in many frontends, ``cursor_pos`` | ||
| in versions prior to 5.2 is ambiguous in the presence of astral-plane. |
There was a problem hiding this comment.
missing a noun:
in the presence of "astral-plane" characters
(added scare quotes around "astral-plane" since this is a slang term)
There was a problem hiding this comment.
maybe also link the github issue for the technical details?
|
In |
This ought to be the 'version' field in the request header, I believe. |
|
Currently |
|
|
|
Came back to this. I believe I've addressed comments now, and issued PRs against jupyter/notebook and jupyterlab to handle the transform and bump the version to 5.2. |
minrk
force-pushed
the
52-unicode
branch
2 times, most recently
from
f0081e6
to
7379294
Compare
docs/messaging.rst
Outdated
| taking up two indices instead of one, causing a unicode offset | ||
| drift of one per astral-plane character. | ||
| Not all frontends have this behavior, however, | ||
| and after JSON serialization information about encoding was used |
|
cross-ref: nteract/hydrogen#807 |
cursor_pos with characters after 0x10000 is ambiguous with respect to surrogate pairs in versions 5.1 and earlier due to widespread bugs in javascript frontends
cursor_pos with characters after 0x10000 is officially ambiguous with respect to surrogate pairs in versions 5.1 and earlier due to a widespread bug in javascript frontends.
Thus bumps the protocol to 5.2 with the sole change that cursor_pos is no longer ambiguous (i.e. clients that correctly implement cursor_pos in 5.1 can bump to 5.2 with no other changes).
cf #259
cross-refs for frontend implementations:
cc @stevengj @rgbkrk @Carreau @blink1073 @williamstein