Poetry should write .pth files in sys.path with the locale-defined encoding

[shoucandanghehe]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Windows 11 22H2 22622.436
• Poetry version: 1.1.14
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/shoucandanghehe/84fa45ddecba402d284bd22e7e141c11

Issue

This looks similar to #5985, but it should not be the same error
I tried removing all python environments and reinstalling poetry, but it didn't work
Below is the error log

nonebot-plugin-tetris-stats on  main [!⇡] via 🐍 v3.10.5 
❯ poetry install -vvv

  Stack trace:

  12  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\clikit\console_application.py:131 in run
       129│             parsed_args = resolved_command.args
       130│
     → 131│             status_code = command.handle(parsed_args, io)
       132│         except KeyboardInterrupt:
       133│             status_code = 1

  11  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\clikit\api\command\command.py:120 in handle
       118│     def handle(self, args, io):  # type: (Args, IO) -> int
       119│         try:
     → 120│             status_code = self._do_handle(args, io)
       121│         except KeyboardInterrupt:
       122│             if io.is_debug():

  10  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\clikit\api\command\command.py:163 in _do_handle
       161│         if self._dispatcher and self._dispatcher.has_listeners(PRE_HANDLE):
       162│             event = PreHandleEvent(args, io, self)
     → 163│             self._dispatcher.dispatch(PRE_HANDLE, event)
       164│
       165│             if event.is_handled():

   9  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\clikit\api\event\event_dispatcher.py:22 in dispatch
        20│
        21│         if listeners:
     →  22│             self._do_dispatch(listeners, event_name, event)
        23│
        24│         return event

   8  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\clikit\api\event\event_dispatcher.py:89 in _do_dispatch
        87│                 break
        88│
     →  89│             listener(event, event_name, self)
        90│
        91│     def _sort_listeners(self, event_name):  # type: (str) -> None

   7  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\console\config\application_config.py:119 in set_env
       117│
       118│         env_manager = EnvManager(poetry)
     → 119│         env = env_manager.create_venv(io)
       120│ 
       121│         if env.is_venv() and io.is_verbose():

   6  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\env.py:659 in create_venv
        657│ 
        658│         cwd = self._poetry.file.parent
     →  659│         env = self.get(reload=True)
        660│
        661│         if not env.is_sane():

   5  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\env.py:522 in get
        520│                 return self.get_system_env()
        521│
     →  522│             return VirtualEnv(venv)
        523│
        524│         if env_prefix is not None:

   4  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\env.py:1375 in __init__
       1373│         # from inside the virtualenv.
       1374│         if base is None:
     → 1375│             self._base = Path(self.run_python_script(GET_BASE_PREFIX).strip())
       1376│
       1377│     @property

   3  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\env.py:1163 in run_python_script
       1161│ 
       1162│     def run_python_script(self, content, **kwargs):  # type: (str, Any) -> str
     → 1163│         return self.run(self._executable, "-W", "ignore", "-", input_=content, **kwargs)
       1164│
       1165│     def _run(self, cmd, **kwargs):

   2  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\env.py:1152 in run
       1150│         bin = self._bin(bin)
       1151│         cmd = [bin] + list(args)
     → 1152│         return self._run(cmd, **kwargs)
       1153│
       1154│     def run_python(self, *args, **kwargs):

   1  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\env.py:1453 in _run
       1451│     def _run(self, cmd, **kwargs):
       1452│         kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
     → 1453│         return super(VirtualEnv, self)._run(cmd, **kwargs)
       1454│
       1455│     def get_temp_environ(

  EnvCommandError

  Command C:\Users\17486\AppData\Local\pypoetry\Cache\virtualenvs\nonebot-plugin-tetris-stats-bRFMNshD-py3.10\Scripts\python.exe -W ignore - errored with the following return code 1, and output:
  Fatal Python error: init_import_site: Failed to import the site module
  Python runtime state: initialized
  Traceback (most recent call last):
    File "C:\Program Files\Python310\lib\site.py", line 617, in <module>
      main()
    File "C:\Program Files\Python310\lib\site.py", line 600, in main
      known_paths = venv(known_paths)
    File "C:\Program Files\Python310\lib\site.py", line 532, in venv
      addsitepackages(known_paths, [sys.prefix])
    File "C:\Program Files\Python310\lib\site.py", line 387, in addsitepackages
      addsitedir(sitedir, known_paths)
    File "C:\Program Files\Python310\lib\site.py", line 226, in addsitedir
      addpackage(sitedir, name, known_paths)
    File "C:\Program Files\Python310\lib\site.py", line 179, in addpackage
      for n, line in enumerate(f):
  UnicodeDecodeError: 'gbk' codec can't decode byte 0x80 in position 10: illegal multibyte sequence
  input was : import sys

  if hasattr(sys, "real_prefix"):
      print(sys.real_prefix)
  elif hasattr(sys, "base_prefix"):
      print(sys.base_prefix)
  else:
      print(sys.prefix)


  at ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\env.py:1195 in _run
      1191│                 output = subprocess.check_output(
      1192│                     cmd, stderr=subprocess.STDOUT, **kwargs
      1193│                 )
      1194│         except CalledProcessError as e:
    → 1195│             raise EnvCommandError(e, input=input_)
      1196│
      1197│         return decode(output)
      1198│
      1199│     def execute(self, bin, *args, **kwargs):

[shoucandanghehe]

Update:
I noticed
C:\Users\17486\AppData\Local\pypoetry\Cache\virtualenvs\nonebot-plugin-tetris-stats-bRFMNshD-py3.10\Lib\site-packages (my virtualenv path)
There is a nonebot_plugin_tetris_stats.pth file under the path, poetry uses UTF-8 encoding to save it, but the python tries to decoding it using GBK
Because my path contains Chinese, so python reports an error
Now I remove the Chinese from the path, poetry works fine. (because UTF-8 encoded English can be decoded using GBK)

[neersighted]

Had to do a fair bit of spelunking to figure out if there was any defined encoding, and who was in the wrong here:

https://github.com/python/cpython/blob/cac2e8a51fe6412b7bc864274a60927ececfd833/Lib/site.py#L173-L175

Newer versions of CPython are thankfully explicit and spell this out -- Poetry needs to write this file using the default encoding. This is a fairly trivial code change, and the test is harder to write (but should not be that hard -- an existing test can be modified to check encoding and make sure LOCALE is respected).

[SkandaPrasad-S]

Has anyone picked this up? I can pick it up if no one has

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.