Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

utils.env._full_python_path output encoding on Windows #7313

Closed
2 of 4 tasks
Winand opened this issue Jan 6, 2023 · 2 comments · Fixed by #8060
Closed
2 of 4 tasks

utils.env._full_python_path output encoding on Windows #7313

Winand opened this issue Jan 6, 2023 · 2 comments · Fixed by #8060
Labels
kind/bug Something isn't working as expected status/triage This issue needs to be triaged

Comments

@Winand
Copy link
Contributor

Winand commented Jan 6, 2023

  • Poetry version: 1.3.1
  • Python version: 3.9.13
  • OS version and name: Windows 11 22H2
  • pyproject.toml: gist
  • I am on the latest stable Poetry version, installed using a recommended method.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have consulted the FAQ and blog for any relevant entries or release notes.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

I want to use poetry along with pyenv-win.
When I set prefer-active-python = true and call poetry install I get "Command cannot be found" error:

PS F:\python\projects\tutorial-hypermodern-python-winand> poetry install
El sistema no puede encontrar la ruta especificada.

Command 'C:\Users\Àíäðåé\.pyenv\pyenv-win\versions\3.8.10\python.exe -c "import sys; print('.'.join([str(s) for s in sys.version_info[:3]]))"' returned non-zero exit status 1.

Username in the output has broken encoding (Àíäðåé instead of Андрей). I've found out that poetry tries to get current interpreter using utils.env._full_python_path function which returns broken path in my case.

I've found two possible solutions:

  • Add universal_newlines=True / text=True to subprocess.check_output call. That fixes only paths which can be encoded with local encoding - cp1251.
  • Encode output to UTF-8 in child process:
subprocess.check_output(
    list_to_shell_command(
        # [python, "-c", '"import sys; print(sys.executable)"']
        [python, "-c", '"import sys; sys.stdout.buffer.write(sys.executable.encode())"']
    ),
    shell=True,
).strip()
@Winand Winand added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Jan 6, 2023
@Winand
Copy link
Contributor Author

Winand commented Jan 6, 2023

Path encoding is also a problem for poetry install.
poetry thinks that pyenv's 3.9.13 base environment is a virtual environment and installs everything there. (actually 3.8.10 is selected in pyenv)

PS F:\python\projects\tutorial-hypermodern-python-winand> poetry env info

Virtualenv
Python:         3.9.13
Implementation: CPython
Path:           C:\Users\Андрей\.pyenv\pyenv-win\versions\3.9.13
Executable:     C:\Users\Андрей\.pyenv\pyenv-win\versions\3.9.13\python.exe
Valid:          True

System
Platform:   win32
OS:         nt
Python:     3.8.10
Path:       C:\Users\Àíäðåé\.pyenv\pyenv-win\versions\3.9.13
Executable: python

If I patch utils.env.GET_BASE_PREFIX:

GET_BASE_PREFIX = """\
import sys

if hasattr(sys, "real_prefix"):
    sys.stdout.buffer.write(sys.real_prefix.encode())
elif hasattr(sys, "base_prefix"):
    sys.stdout.buffer.write(sys.base_prefix.encode())
else:
    sys.stdout.buffer.write(sys.prefix.encode())
"""

poetry env info still does not show 3.8.10 but poetry install creates a new virtual environment in .venv folder as expected (prefer-active-python, in-project are set)

PS F:\python\projects\tutorial-hypermodern-python-winand> poetry env info   

Virtualenv
Python:         3.9.13
Implementation: CPython
Path:           NA
Executable:     NA

System
Platform:   win32
OS:         nt
Python:     3.9.13
Path:       C:\Users\Андрей\.pyenv\pyenv-win\versions\3.9.13
Executable: C:\Users\Андрей\.pyenv\pyenv-win\versions\3.9.13\python.exe

dimbleby added a commit to dimbleby/poetry that referenced this issue Jun 4, 2023
dimbleby added a commit to dimbleby/poetry that referenced this issue Jun 4, 2023
radoering pushed a commit to dimbleby/poetry that referenced this issue Jun 18, 2023
radoering referenced this issue Jun 18, 2023
cf python-poetry#7313, python-poetry#7643
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Something isn't working as expected status/triage This issue needs to be triaged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant