Cursor position incorrect with non-European characters

[pepa65]

Hoping this is a bug that will be easy to fix:
When opening this UTF-8 sample file in diakonos and going with the cursor to the end of each line,
the cursor lines up with the end of the displayed characters, except in case of the Thai fonts (line 123-130). There the cursor is displayed way past the end of the characters. This makes it difficult to edit Thai texts.

[pepa65]

This report was for the Ubuntu LTS repo version of diakonos (0.9.0) and I found I couldn't build the latest version due to outdated Ruby (1.9.1). So I tried to install it in Ubuntu 15.10 - it's no longer in the repo (why?) so I installed ruby, cloned the git repo, tried to install curses "sudo gem install curses" but it failed to build 'gem native extention'... (Couldn't find ruby.h, so I installed ruby2.1-dev but it still could make a Makefile, 'probably lack of necessary libraries and/or headers') After installing libncurses5-dev and libtinfo-dev (I think) the curses gem did install, and I could run diakonos.

Loading the UTF-8 demo in diakonos rendered total rubbish...

[Pistos]

@pepa65 Thank you for the detailed bug report, as well as for being thorough. Can you provide a link to an example file with Thai text?

[Pistos]

I should also mention that Diakonos does not support languages whose characters are different width than normal Roman (English and European) characters.

[pepa65]

Sorry, meant to include this in the report:
http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt
It's just that diakonos renders the whole file well, except that the width of the Thai characters (line 123-130) get in the way. the problem occurs because one character is a consonant, and the vowel gets superimposed in the same space, so the editor thinks there are more characters then are actually rendered.

There must be a way to incorporate this in a future release?? Diakonos seems to have come such a long way since I last evaluated it a number of years ago.

Some editors do this correctly, notably JED. jupp, vim, nano, pico, le and joe. Others have the same problem as diakonos, like dex and e3.

EDIT: dex has now been fixed (July 2017).

[Pistos]

I've confirmed the behaviour you're seeing. However, I regret to report that languages like Thai are not supported by Diakonos at this time. I don't know if this would be easy to fix/add, but I welcome pull requests.

[pepa65]

I'm not quite up to working on this issue, but the people from dex (similar issue) came up with this:

• The problem is with combining characters
• Grapheme_Extend characters should have zero width, as they 'ride' on the character before it. See http://www.unicode.org/reports/tr44/#Grapheme_Extend
• The characters marked Mn on this page are zero-width: http://www.unicode.org/Public/8.0.0/ucd/extracted/DerivedCombiningClass.txt
• But, GLib has a g_unichar_iszerowidth() function that returns true for any character in the Grapheme_Extend category: https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html#g-unichar-iszerowidth
• Grapheme_Base characters and a following Grapheme_Extend character should stay together in selections. (For Backspace-delete, the Grapheme_Extend character usually gets deleted first!)

[pepa65]

The same problem was present in dex https://github.com/tihirvon/dex and reported there as issue 22, and @craigbarnes just found the way to fix it: tihirvon/dex#22 (comment) !!
I expect (hope?) the fix can be as easy for diakonos as well!