pip wheel fails without useful error message when --wheel-dir not writable

[timb07]

Environment

• pip version: 18.0
• Python version: 3.7
• OS: macOS 10.13.6

Description

When wheel is invoked specifying a non-writable destination directory with --wheel-dir, it fails but doesn't give an error indicating the cause of the failure.

$ pip3 wheel --wheel-dir=not_writable_directory .
Processing /home/user/example_pkg
Building wheels for collected packages: example-pkg
  Running setup.py bdist_wheel for example-pkg ... done
  Running setup.py clean for example-pkg
Failed to build example-pkg
ERROR: Failed to build one or more wheels

Adding -vvv to the pip3 command line provides more output but still no useful error message.

Expected behavior

I would expect to see something like:

Failed to write not_writable_directory/example_pkg-0.0.1-py3-none-any.whl: permission denied

How to Reproduce

1. Get any wheel-installable package source (e.g. pip or the minimal example at https://packaging.python.org/tutorials/packaging-projects/)
2. Make a directory that isn't writable: mkdir not_writable_directory; chmod 555 not_writable_directory
3. Then run pip3 wheel --wheel-dir=not_writable_directory .
4. The wheel doesn't build, but no useful error message is given.

Output

See above.

Note

This was originally raised as a bug in wheel: pypa/wheel#246

[pradyunsg]

This issue is a good starting point for anyone who wants to help out with pip's development -- it's simple and the process of fixing this should be a good introduction to pip's development workflow.

There's an existing PR for this, with an additional change suggested there.

[sbidoul]

This issue is partially solved in master, following #7285.
Now it fails without warning only if the cache directory is not writable, because pip wheel now caches first and then copies to wheel-dir, and that copy operation logs errors:

pip/src/pip/_internal/wheel_builder.py

Lines 488 to 496 in ef20309

	shutil.copy(
	os.path.join(output_dir, wheel_file),
	self._wheel_dir,
	)
	except OSError as e:
	logger.warning(
	"Building wheel for %s failed: %s",
	req.name, e,
	)

[pb-cdunn]

We are having lots of problems with pip-19.3.1 (pip3) when multiple users write into the wheel-dir.

• This is not a race condition; we are not working simultaneously.
• It's not strictly permissions either. When we give each other write-perms, there is a still a problem. But removing each other's file solves the problem.

The absence of an error message is the biggest problem. We could probably work around this if only we knew it was related to this other ownership bug.

[pradyunsg]

I think it's pretty clear that we want to be printing a better error message in this situation. PRs are welcome for fixing this!

[deveshks]

Hi @pradyunsg , @sbidoul

I do see that as per

pip/src/pip/_internal/commands/wheel.py

Lines 182 to 189 in 720c77a

	try:
	shutil.copy(req.local_file_path, options.wheel_dir)
	except OSError as e:
	logger.warning(
	"Building wheel for %s failed: %s",
	req.name, e,
	)
	build_failures.append(req)

and #5740 (comment)

The wheel command now fails with

WARNING: Building wheel for example-pkg-YOUR-USERNAME-HERE failed: 
[Errno 13] Permission denied: '~/non_writable_dir/example_pkg_YOUR_USERNAME_HERE-0.0.1-py3-none-any.whl'

So are we looking for a better error message than this? If yes, what details would such an error message cover?

[sbidoul]

So are we looking for a better error message than this?

No. To me, the error message is fine in pip 20.
The issue with read-only cache directories has been resolved in pip 20 too (#7489).

So I think we can now close this issue.

[deveshks]

If that is the case, we can close the issue, although I don't have permission to do that.

[pradyunsg]

Done! Thanks @deveshks and @sbidoul for investigating and figuring out that we fixed this! ^>^