[arch, plug] fix paths that contain unicode char for plugInfo.json #1987
Conversation
nvidia-jomiller
force-pushed
the
dev_unicode_path_fix
branch
from
fdbe0e3
to
cc35e46
Compare
|
It looks like the build for Windows on Azure is hanging. It's working correctly on Windows 10 but will investigate when time permits. The line where this seems to be hanging is here: |
|
Filed as internal issue #USD-7554 |
nvidia-jomiller
force-pushed
the
dev_unicode_path_fix
branch
2 times, most recently
from
00634cc
to
a7513e6
Compare
arch: ArchGetAddressInfo was using GetModuleFileName instead of GetModuleFileNameW on Windows. This prevented the objectPath to be correctly returned should the module found at the address contained a unicode character. plug: _ReadPlugInfoObject did not properly handle the case of a UTF-8 encoded string on Windows. This would result in the plugInfo.json file not being loaded if it resided at a path containing a unicode character
nvidia-jomiller
force-pushed
the
dev_unicode_path_fix
branch
from
a7513e6
to
cfc54f6
Compare
| # To keep things consistent we'll force UTF-8 for non-interactive | ||
| # streams (which is recommended by Microsoft). See: | ||
| # https://docs.python.org/3.6/library/sys.html#sys.stdout | ||
| # https://docs.microsoft.com/en-us/windows/win32/intl/code-page-identifiers |
There was a problem hiding this comment.
I was able to track down the build issue with Windows Azure and it has something to do with what default encoding is set for the terminal running build_usd.py. I made a small change to GetLocale() in build_usd.py to force UTF-8 on non-interactive terminals for Windows.
Specifically sys.stdout.encoding on Windows will use the ANSI codepage for non-interactive streams and Microsoft recommends to use UTF-8 or UTF-16 over ANSI codepages
| # for Windows | ||
| pxr_create_test_module(TestPlugModuleUnicode | ||
| INSTALL_PREFIX PlugPlugins | ||
| DEST_DIR TestPlugMödüleÜnicöde |
There was a problem hiding this comment.
It's worth pointing out that DEST_DIR will be a created directory which contains unicode characters to verify the fix
|
I know this isn't a very high priority bug, but it would be great if this could get reviewed and merged into Thanks! |
|
We're scheduling this for the next release, @nvidia-jomiller - thanks! |
|
@nvidia-jomiller I was wondering if you've tried this under Windows 11? Although the fix seems to address the specific case in the issue title, I haven't been able to get "unicode user name in path" case mentioned in the usd-interest message working without further work in addition to the changes you propose here, and I'm wondering if it's a Windows 10 vs 11 thing. |
|
@meshula I haven't tried under Windows 11 as my Windows machine is still on Windows 10. I assume you're testing on Windows 11? What additional work did you need to do with the usd-interest case to get it working? |
|
TfGlob failed to discover the pluginfo.json files and I made a fix that I'm not particularly excited about. If you are interested in seeing some of the issues around unicode in filenames versus toolchains (not USD specifically), I have been collecting findings here... https://github.com/meshula/m-n |
Description of Change(s)
arch: ArchGetAddressInfo was using GetModuleFileName instead of GetModuleFileNameW on Windows. This prevented the objectPath to be correctly returned should the module found at the address containing a unicode character.
plug: _ReadPlugInfoObject did not properly handle the case of a UTF-8 encoded string on Windows. This would result in the plugInfo.json file not being loaded if it resided at a path containing a unicode character
Fixes Issue(s)