When moving by words with text review commands in Scintilla based editors NVDA treats spaces as separate words

[lukaszgo1]

Steps to reproduce:

1. open a Scintilla based editor (I've tested with Notepad++ and TeXnicCenter).
2. Write some text
3. Move navigator object to focus.
4. Start moving with Numpad 4 and 6 between words.

Expected behavior:

NVDA should report only words without spaces between them like in all other cases.

Actual behavior:

NVDA reports every space as a separate word.

System configuration:

NVDA version:
next-15133,1f3d46d2

NVDA Installed or portable:
Installed

Windows version:
Windows 8 x64

Name and version of other software in use when reproducing the issue:
Notepad++ v7.5.4 (64-bit), TeXnicCenter 2.02 Stable (64 bit)

Other questions:

Does the issue still occur after restarting your PC?
Yes
Have you tried any other versions of NVDA?
Yes from 2017,2 onward and it didn't work.

[LeonarddeR]

I can confirm this.

[lukaszgo1]

It seems that by default space isn't considered a word separator by Scintilla so when calculating word start with SCI_WORDSTARTPOSITION and word end with SCI_WORDENDPOSITION space is a separate word. There is a Scintilla message SCI_SETWHITESPACECHARS which should fix this. I don't know however how to send space using it.

[LeonarddeR]

Sending messages to controls in order to changed their behavior is honestly not what I'd prefer. If there are implementations that don't respect spaces as word separators, this should be covered as part of the implementation of that particular control. In Short, if navigating with control+arrows doesn't jump across spaces, enabling NVDA shouldn't change anything about this.

[lukaszgo1]

@LeonarddeR I'm afraid I do not understand your last comment. When navigating with ctrl+arrows the editor determines what is a word separator and what isn't. When working with object review it is an NVDA job to determine this and since in Scintilla the word separators aren't set to space by default when NVDA sends Scintilla messages asking about start and end of given word space is returned.

[lukaszgo1]

cc @DataTriny given your recent work on Scintilla.

[Adriani90]

and also cc: @francipvb

[LeonarddeR]

@lukaszgo1 commented on 24 May 2019, 14:02 CEST:

When navigating with ctrl+arrows the editor determines what is a word separator and what isn't.

This is true.

When working with object review it is an NVDA job to determine this.

This is not entirely true. When moving to the next word, NVDA asks the TextInfo object for the particular control to move to the next word. How a TextInfo object calculates word boundaries is mostly determined by the underlying TextINfo implementation and thus by the underlying control.

since in Scintilla the word separators aren't set to space by default when NVDA sends Scintilla messages asking about start and end of given word space is returned.

This actually makes sense. Moving with ctrl+arrows does include spaces, which is inconsistent.

[LeonarddeR]

I looked at the code and it looks like NVDA has a certain workaround in it to ensure that word offsets are correct, with the downside of space treated as a separator.

[LeonarddeR]

On closer inspection, at least for Notepad++, it is Scintilla that's failing here. Scintilla is definitely treating one space or a group of spaces as a word when using SCI_WORDSTARTPOSITION and SCI_WORDENDPOSITION.

[lukaszgo1]

Which is exactly what I am saying. According to this quote from the Scintilla documentation when we want Scintilla not to do that we have to set spaves as word separators ourselves.

SCI_SETWHITESPACECHARS(, const char *characters)
SCI_GETWHITESPACECHARS(, char *characters) → int
Similar to SCI_SETWORDCHARS, this message allows the user to define which chars Scintilla considers as whitespace. Setting the whitespace chars allows the user to fine-tune Scintilla's behaviour doing such things as moving the cursor to the start or end of a word; for example, by defining punctuation chars as whitespace, they will be skipped over when the user presses ctrl+left or ctrl+right. This function should be called after SCI_SETWORDCHARS as it will reset the whitespace characters to the default set. SCI_GETWHITESPACECHARS behaves similarly to SCI_GETWORDCHARS.

[LeonarddeR]

The problem is that the docs say:

for example, by defining punctuation chars as whitespace, they will be skipped over when the user presses ctrl+left or ctrl+right.

However, the movement behaviour of ctrl+left/rightArrow doesn't match with the offsets returned by SCI_WORDSTARTPOSITION and SCI_WORDENDPOSITION