Need an escape hatch for corrupted cache.

[eirrgang]

Description

If the pip wheel_cache json file is corrupted, users get a confusing error and there does not seem to be a recovery mode other than finding and removing the cache.

pip cache purge does not resolve such a problem.

It is not completely clear how the file became corrupted in this case, but it may have been related to an instance of a disk quota being exceeded.

Expected behavior

• The JSON decoder exception should be caught.
• Either pip should automatically remove / regenerate the cache data file or pip should advise the user on how to proceed.

In this case, the exception propagated from pip/_internal/cache.py::WheelCache.record_download_origin(), which called DirectUrl.from_json(). Letting the JSONDecoderError propagate through DirectUrl.from_json() seems appropriate. It also seems reasonable to let it propagate through WheelCache.record_download_origin().

I think one of the following would be an appropriate place for additional error handling.

• pip/_internal/wheel_builder.py::build() produces a build_failures list, but does not catch exceptions for conversion to "build_failures".
• pip/_internal/commands/install.py::InstallCommand.run() calls build in a try block, but only OSError is caught.

It also seems reasonable that pip cache purge could reinitialize state files more completely.

pip version

main, as of 20 April 2023

Python version

all

OS

all

How to Reproduce

pip install a package from a source distribution, producing a cachable "wheel", but with invalid JSON files in the pip cache.

Output

Building wheels for collected packages: ...
  Stored in directory: /.../.cache/pip/wheels/e4/50/f2/fddc95515d81964ea105d53c75210d0c94388d80faf5e0b797
ERROR: Exception:
Traceback (most recent call last):
  File "/.../lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 169, in exc_logging_wrapper
    status = run_func(*args)
  File "/.../lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
    return func(self, options, args)
  File "/.../lib/python3.9/site-packages/pip/_internal/commands/install.py", line 417, in run
    _, build_failures = build(
  File "/.../lib/python3.9/site-packages/pip/_internal/wheel_builder.py", line 334, in build
    wheel_cache.record_download_origin(cache_dir, req.download_info)
  File "/.../lib/python3.9/site-packages/pip/_internal/cache.py", line 261, in record_download_origin
    origin = DirectUrl.from_json(origin_path.read_text())
  File "/.../lib/python3.9/site-packages/pip/_internal/models/direct_url.py", line 231, in from_json
    return cls.from_dict(json.loads(s))
  File "/opt/cray/pe/python/[3.9.12.1/lib/python3.9/json/__init__.py](http://3.9.12.1/lib/python3.9/json/__init__.py)", line 346, in loads
    return _default_decoder.decode(s)
  File "/opt/cray/pe/python/[3.9.12.1/lib/python3.9/json/decoder.py](http://3.9.12.1/lib/python3.9/json/decoder.py)", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/opt/cray/pe/python/[3.9.12.1/lib/python3.9/json/decoder.py](http://3.9.12.1/lib/python3.9/json/decoder.py)", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[sbidoul]

@eirrgang thanks for reporting! With #12060, this causes a warning instead of an error.