Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mouse tracking does not work well with display scaling. #6722

Closed
feerrenrut opened this issue Jan 12, 2017 · 4 comments · Fixed by #13254
Closed

Mouse tracking does not work well with display scaling. #6722

feerrenrut opened this issue Jan 12, 2017 · 4 comments · Fixed by #13254
Labels
bug displayScaling Display scaling and DPI awareness feature/mouse-tracking p4 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority triaged Has been triaged, issue is waiting for implementation.
Milestone
2022.4

Comments

@feerrenrut
Copy link
Contributor

feerrenrut commented Jan 12, 2017 •
edited
Loading

Using multiple monitors on Windows 10. Set different scaling percentages in the display settings dialog for each monitor.
Notice that the object that is under the mouse is not the object being read.

Setting both monitors to 100% results in the objects being reported correctly. Its worth noting that the objects reported may not even be in the same window as the mouse. For instance, the mouse may be over a notepad window, and the task bar is being reported.
@feerrenrut feerrenrut added bug feature/mouse-tracking p4 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority labels Jan 12, 2017
@jcsteh
Copy link
Contributor

jcsteh commented Jan 12, 2017

It's worth noting that NVDA reports as being DPI aware, but only of the system DPI. That is, the system doesn't have to scale it for values other than 100% if the DPI doesn't change, but if the DPI changes (or I assume if you're using multiple monitors with different DPI), the system will scale it to the system DPI value fetched when NVDA starts. For reference, we use SetProcessDPIAware. The new function is SetProcessDpiAwareness, which supports the new PROCESS_PER_MONITOR_DPI_AWARE awareness level.

That said, this doesn't quite explain what you're seeing. The point of DPI scaling is that the system is supposed to scale coordinates for unaware apps so they're reasonably accurate. The reason we switched to being DPI aware was because this was slightly inaccurate in some cases, but you shouldn't be seeing inaccuracy to the degree you're seeing.

My concern with becoming "per monitor DPI aware" is that we then have to figure out which monitor we're dealing with when we do have to scale coordinates. I'm also not sure if wx handles this, which could mean weirdness in our GUI if a user plugs in a different monitor or changes DPI at runtime. (Right now, the system will transparently scale our GUI if this happens.)

See also #3758, #3703.

@dkager
Copy link
Collaborator

dkager commented Jan 14, 2017

While this is on a single monitor, I ran into very similar problems in #4703.

@nishimotz nishimotz mentioned this issue May 11, 2018
Closed
@Adriani90
Copy link
Collaborator

cc: @LeonarddeR this might be also of interest for you at Babbage.

@LeonarddeR LeonarddeR added the displayScaling Display scaling and DPI awareness label May 2, 2019
@Qchristensen
Copy link
Member

Following a query about this from a user, I tested this again and found the issue is still present with NVDA 2022.2beta3 on Windows 11 (64-bit) Version: 21H2 (2009), Build: 22000.778

@seanbudd seanbudd linked a pull request Jul 14, 2022 that will close this issue
Use modern DPI awareness settings #13254
Merged
34 tasks
This was referenced Jul 14, 2022
Merged
Closed
@seanbudd seanbudd added the triaged Has been triaged, issue is waiting for implementation. label Jul 28, 2022
seanbudd pushed a commit that referenced this issue Aug 30, 2022
@VeselovAlex
Use modern DPI awareness settings (#13254)
28b4756 
Fixes #13370
Fixes #6722
Fixes #3875
Fixes #12070
Fixes #7083
Fixes #7915
Likely fixes #9531, otherwise close as stale/can't reproduce

### Summary of the issue:
When DPI for a monitor is not set to 100%, or when using multiple monitors with different DPI settings,
NVDA would:

- misplace highlight frames
- have inaccurate mouse tracking
- have inaccurate touch screen interaction

NVDA currently sets the DPI awareness via a Windows API call introduced in Windows Vista.
It is recommended to set DPI through the app manifest, rather than Windows API calls where possible.

Newer settings for DPI awareness have been introduced since Windows Vista.
Windows 8 introduced multiple monitor DPI awareness.
Windows 10 introduced a richer version of multiple monitor DPI awareness.

### Description of how this pull request fixes the issue:
Background docs: https://docs.microsoft.com/en-us/windows/win32/hidpi/high-dpi-desktop-application-development-on-windows#per-monitor-and-per-monitor-v2-dpi-awareness

When running as an executable, NVDA sets DPI awareness via the app manifest.
When running through source, NVDA sets DPI awareness via Windows API calls.
The most modern method available is used to set DPI awareness.

- For Windows 7, DPI awareness is unlikely to improve. There may be fixes from setting it via the app manifest instead of via Windows API calls.
- For Windows 8 and newer, NVDA has per monitor DPI awareness
- For Windows 10 1703 and newer, NVDA has [advanced per monitor DPI awareness](https://docs.microsoft.com/en-us/windows/win32/hidpi/dpi-awareness-context), including:
  - Child window DPI change notifications
  - Scaling of non-client area - All windows will automatically have their non-client area drawn in a DPI sensitive fashion. Calls to [EnableNonClientDpiScaling](https://docs.microsoft.com/en-us/windows/desktop/api/Winuser/nf-winuser-enablenonclientdpiscaling) are unnecessary.
  - Scaling of Win32 menus - All NTUSER menus created in Per Monitor v2 contexts will be scaling in a per-monitor fashion.
  - Dialog Scaling - Win32 dialogs created in Per Monitor v2 contexts will automatically respond to DPI changes.
  - Improved scaling of comctl32 controls - Various comctl32 controls have improved DPI scaling behavior in Per Monitor v2 contexts.
  - Improved theming behavior - UxTheme handles opened in the context of a Per Monitor v2 window will operate in terms of the DPI associated with that window.
@nvaccessAuto nvaccessAuto added this to the 2022.4 milestone Aug 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug displayScaling Display scaling and DPI awareness feature/mouse-tracking p4 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority triaged Has been triaged, issue is waiting for implementation.
Projects
None yet
Milestone
2022.4
Development

Successfully merging a pull request may close this issue.

Use modern DPI awareness settings VeselovAlex/nvda
8 participants
@LeonarddeR @dkager @feerrenrut @jcsteh @seanbudd @nvaccessAuto @Qchristensen @Adriani90