Report comments in MS Word with UIA enabled

[jakubl7545]

Link to issue number:

closes #9285

Summary of the issue:

With UIA enabled it is impossible to read comments. This is because comments ar filtered by its name which appeared to be different across languages.
Also when there are more than 13 comments dates can't be retrieved because don't fit on the screen.

Description of how this pull request fixes the issue:

Instead of name comments are filtered by its role and AnnotationsPattern.
In situations when date is not on the screen only comment itself and author are reported.

Testing performed:

Tested with NVDA running from source with word document which has comments like the attached one.

Known issues with pull request:

None

Change log entry:

Section: Bug fixes
It is now possible to read comments in MS Word with UIA enabled.
test.docx

[zstanecic]

Before merging this, can you provide the try build?

[lukaszgo1]

@zstanecic the build from this PR is here
https://ci.appveyor.com/api/buildjobs/h0xaocd73ynxwnpa/artifacts/output%2Fnvda_snapshot_pr9631-17375%2C76080910.exe

[LeonarddeR]

I haven't tested this myself, but here are some comments based on just reading the code.

[LeonarddeR]

Why did you remove the ``if not obj.parent` here? This breaks if the object has no parent.

[jakubl7545]

I focused on replacing obj.name and missed it. Thanks for pointing this.

[LeonarddeR]

I think it would be good to have some documentation about how you came up with the if clause above.

[jakubl7545]

First I wanted to replace obj.name with something which doesn't change and decided for role which is grouping. In case it would be not enough I checked developer info for that object and discovered that comments have AnnotationPattern which may be significant for them. Then I found UIAElement.getCurrentPropertyValue checks for its presence. It requires one parameter which is propertyId. For AnnotationPattern it is 30118 but I didn't want to use that as itself it means nothing. Thats why as a parameter I used UIAHandler.UIA_IsAnnotationPatternAvailablePropertyId which provides that Id.
Thats how the process of creating looked like but I don't know how much of it should be documented. Maybe you have some ideas.

[LeonarddeR]

I think it is ok as it is. Thanks for providing the details here. Leaving this treat unresolved until @michaelDCurran or someone else has looked at it.

[LeonarddeR]

I think that it is more readable if you check obj.previous.previous first. Now, you are assigning an object to the dateObj variable and then, when it doesn't seem to be a dateObj, you're assigning the same object to authorObj.

For example:

		comment=obj.makeTextInfo(textInfos.POSITION_ALL).text
		tempObj = obj.previous.previous
		authorObj = tempObj or obj.previous
		author =authorObj.name
		if not tempObj:
			return dict(comment=comment,author=author)
		dateObj=obj.previous
		date=dateObj.name
		return dict(comment=comment,author=author,date=date)

[LeonarddeR]

Suggested change

	if len(commentInfo) ==2:
	if "date" not in commentInfo:

[LeonarddeR]

This lacks the description. Please reuse the description for the script for the object model implementation of Word.

Suggested change

	@script(gesture ="kb:NVDA+alt+c")
	@script(
	gesture="kb:NVDA+alt+c",
	# Translators: a description for a script that reports the comment at the caret.
	description=_("Reports the text of the comment where the System caret is located.")
	)

[LeonarddeR]

It really amazes me that this is not calling getCommentInfoFromPosition. Could you please change this in such a way that it uses that function and speaks a message based on what is in the dictionary? I think it is even possible to abstract this further and pas the result of CommentUIATextInfoQuickNavItem.label to ui.message.

If this sounds too complex, I"m happy to take this for you.

[jakubl7545]

It works with getCommentInfoFromPosition but when I tried CommentUIATextInfoQuickNavItem.label I got its adress in a memory or "Property object is not callable" error.

[LeonarddeR]

I think it would be best to create a new _getPresentableCommentInfoFromPosition function that returns the label. Ten, you can make the label property return what that helper function returns for the particular textInfo, and same for the script.

Does that clear things up for you?

[jakubl7545]

I hope I understood you well. If so, should this new function has its doc string?
I also added a message for no comments in script.

[zstanecic]

For me this build works as expected. Sorry for late reply.

[jakubl7545]

@LeonarddeR Could you have another look on that?

[LeonarddeR]

@jakubl7545: Could you merge current master into this?

[jakubl7545]

@LeonarddeR: done.

[LeonarddeR]

@jakubl7545 I'm sorry that it took me some time to come back to this. Could you plase merge master again and run scons lint base=origin/master to make sure this doesn't introduce linting errors?

[jakubl7545]

@LeonarddeR It's done. Sorry for such a long delay.

[LeonarddeR]

Thanks @jakubl7545 , looks good

[jakubl7545]

@michaelDCurran Would it be possible to merge this PR into 2019.3? It passed a review from @LeonarddeR so I think it's ready.

[codeofdusk]

Any updates on this PR? CC @michaelDCurran.

[lukaszgo1]

This has been reported several times and it looks like this PR is just awaiting review. @michaelDCurran Do you intent to look at this one?

[michaelDCurran]

I'm quite worried that this implementation might have broken with newer builds of Office 365. I shall give it a test and report back.

[cary-rowen]

What is the latest news of this PR and is it possible to be merged in 2020.4?
Thanks.

[lukaszgo1]

@michaelDCurran Ping. This has been asked about again in #11989

[cary-rowen]

@michaelDCurran Can you review this

[michaelDCurran]

This is not reporting comments for me in MS Word build 16.0.13707 nor has it been for many months. Can someone else confirm that this code still works in Microsoft Word for them? We need to also work out what MS Word version this broke in, and perhaps try and come up with a solution that works for both.

[michaelDCurran]

This pr does not work for me with Microsoft Word build 16.0.13707, nor has it worked with builds of MS Word that I have had for some months now. I am right now looking into how we can make this code more robust. At very least, as the annotation pattern is available, we can actually use that pattern to get the needed info directly now.

[michaelDCurran]

Here is an updated copy of getCommentInfoFromPosition which uses annotation properties directly, and does not need to walk as many objects. This code works on my 16.0.13707 build of MS Word:

def getCommentInfoFromPosition(position):
	"""
	Fetches information about the comment located at the given position in a word document.
	@param position: a TextInfo representing the span of the comment in the word document.
	@type L{TextInfo}
	@return: A dictionary containing keys of comment, author and date
	@rtype: dict
	"""
	val=position._rangeObj.getAttributeValue(UIAHandler.UIA_AnnotationObjectsAttributeId)
	if not val:
		return
	try:
		UIAElementArray=val.QueryInterface(UIAHandler.IUIAutomationElementArray)
	except COMError:
		return
	for index in range(UIAElementArray.length):
		UIAElement=UIAElementArray.getElement(index)
		UIAElement=UIAElement.buildUpdatedCache(UIAHandler.handler.baseCacheRequest)
		typeID = UIAElement.GetCurrentPropertyValue(UIAHandler.UIA_AnnotationAnnotationTypeIdPropertyId)
		if typeID == UIAHandler.AnnotationType_Comment:
			comment = UIAElement.GetCurrentPropertyValue(UIAHandler.UIA_NamePropertyId)
			author = UIAElement.GetCurrentPropertyValue(UIAHandler.UIA_AnnotationAuthorPropertyId)
			date = UIAElement.GetCurrentPropertyValue(UIAHandler.UIA_AnnotationDateTimePropertyId)
			return dict(comment=comment,author=author,date=date)

However, as MS Word now supports comment threads, much of the time, the comment reported will just be "comment thread started by Michael Curran" or similar. Users should really now start viewing comments in the comment thread pain.

[jakubl7545]

Can someone else confirm that this code still works in Microsoft Word for them

@michaelDCurran Yes, it works for me in MSO (16.0.13530.20368). It reads comment itself, author and date exactly as it should.

[cary-rowen]

It works fine for me: Microsoft 365 MSO (16.0.13530.20132) 64 Bit
@michaelDCurran

[AppVeyorBot]

• PR introduces Flake8 errors
• Build (for testing PR)

See test results for failed build of commit 52eb987190

[jakubl7545]

@michaelDCurran I've tried your code but it didn't work. Probably because AnnotationType_Comment is not available. Only AnnotationType_Unknown is returned. So I've combined both approaches to ensure that comments are read in different versions of Office. Could you check if it works for you now?

[michaelDCurran]

@jakubl7545 Yes, this is working for me. Thanks for your work on this.

[michaelDCurran]

@jakubl7545 I think there may be one or more review actions from @feerrenrut before he can also approve?

[feerrenrut]

All changes addressed

[cary-rowen]

In version Microsoft 365 MSO (16.0.14228.20200) 64-bit the pr reported an incorrect comment:
When I moved the cursor to the position containing the comment and pressed NVDA + Alt + C, NVDA reported the following:
"Comment: Resolved Comment Hint Button by Rowen Cary on 25 minutes ago"
According to the above, the content of the comment is: "Resolved Comment Hint" This is wrong and not the actual comment I wrote.
When I turned off the UIA mode in Word, everything returned to normal.
My language is Simplified Chinese, but I installed the English version of Office and can also reproduce this problem.

@feerrenrut
@michaelDCurran

[lukaszgo1]

@cary-rowen Please create a new issue for this and preferably attach a sample document reproducing the problem.