[Bug]: importlib.resources - Repository file opening: open_text is deprecated. Use files() instead

[SylviaWhittle]

Checklist

• Re-run analysis with topostats process --core 1.
• Describe the bug.
• Include the configuration file.
• Copy of the output.
• The exact command that failed. This is what you typed at the command line, including any options.
• TopoStats version, this is reported by topostats --version
• Operating System and Python Version

Describe the bug

When creating a fresh Python 3.11 environment and attempting to run any topostats command, files fail to load when loaded in this manner:

plotting_dictionary = pkg_resources.open_text(topostats, "plotting_dictionary.yaml")

This seems to be because of importlib no longer allowing use of the read_text() function. The error log suggests using files() instead.

[Solution]
I think I have a working solution. I tried it last week and it worked:

• Import files from importlib

from importlib.resources import files

• Use files to load the local repo files instead of read_text. Eg:

default_config = (files(__package__) / "default_config.yaml").read_text()

Copy of the output

❯ topostats create-config
Exception ignored in: <_io.FileIO name='/Users/sylvi/topo_data/cats/libby_shelterin_examples/libby_shelterin_examples-2024-04-22-09-12-58.log' mode='ab' closefd=True>
Traceback (most recent call last):
  File "/Users/sylvi/Documents/TopoStats/topostats/logs/logs.py", line 72, in setup_logger
    return logger
ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/sylvi/topo_data/cats/libby_shelterin_examples/libby_shelterin_examples-2024-04-22-09-12-58.log' mode='a' encoding='UTF-8'>
Exception ignored in: <_io.FileIO name='/Users/sylvi/topo_data/cats/libby_shelterin_examples/libby_shelterin_examples-2024-04-22-09-12-58.log' mode='ab' closefd=True>
Traceback (most recent call last):
  File "/Users/sylvi/Documents/TopoStats/topostats/logs/logs.py", line 72, in setup_logger
    return logger
ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/sylvi/topo_data/cats/libby_shelterin_examples/libby_shelterin_examples-2024-04-22-09-12-58.log' mode='a' encoding='UTF-8'>
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/envs/topo-main/bin/topostats", line 8, in <module>
    sys.exit(entry_point())
             ^^^^^^^^^^^^^
  File "/Users/sylvi/Documents/TopoStats/topostats/entry_point.py", line 386, in entry_point
    args.func(args)
  File "/Users/sylvi/Documents/TopoStats/topostats/io.py", line 129, in write_config_with_comments
    config = pkg_resources.open_text(__package__, "default_config.yaml").read()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/topo-main/lib/python3.11/importlib/resources/_legacy.py", line 18, in wrapper
    warnings.warn(
DeprecationWarning: open_text is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.

Include the configuration file

No config file was used, I was trying to create one using topostats --create-config

To Reproduce

Create a fresh environment of Python 3.11. (I think this also happens on lower versions? I encountered it on Friday and IIRC I tried a 3.9 environment too).

Attempt to create a config file using topostats --create-config

TopoStats Version

Git main branch

Python Version

3.11

Operating System

MacOS M1/M2 (post-2021)

Python Packages

asttokens==2.4.1
cheap-repr==0.5.1
contextlib2==21.6.0
contourpy==1.2.1
cycler==0.12.1
decorator==5.1.1
executing==2.0.1
fonttools==4.51.0
h5py==3.11.0
igor==0.3
igor2==0.5.6
imageio==2.34.0
ipython==8.23.0
jedi==0.19.1
joblib==1.4.0
kiwisolver==1.4.5
lazy_loader==0.4
matplotlib==3.8.4
matplotlib-inline==0.1.7
networkx==3.3
numpy==1.26.4
packaging==24.0
pandas==2.2.2
parso==0.8.4
pexpect==4.9.0
pillow==10.3.0
prompt-toolkit==3.0.43
psutil==5.9.8
ptyprocess==0.7.0
pure-eval==0.2.2
pyfiglet==1.0.2
Pygments==2.17.2
pyparsing==3.1.2
pyspm==0.6.1
python-dateutil==2.9.0.post0
pytz==2024.1
PyYAML==6.0.1
ruamel.yaml==0.18.6
ruamel.yaml.clib==0.2.8
schema==0.7.5
scikit-image==0.23.1
scikit-learn==1.4.2
scipy==1.13.0
seaborn==0.13.2
six==1.16.0
snoop==0.4.3
stack-data==0.6.3
threadpoolctl==3.4.0
tifffile==2024.4.18
topofileformats==0.1.0
-e git+ssh://[email protected]/AFM-SPM/TopoStats.git@e4c8f05d6304446e981c55cc028ca995475fc82e#egg=topostats
tqdm==4.66.2
traitlets==5.14.3
typing_extensions==4.11.0
tzdata==2024.1
wcwidth==0.2.13

[SylviaWhittle]

I can catch this one later today 👍 (leaving it a few hours in case there are suggestions or input)

[ns-rse]

Looks straight-forward to address as there is useful information about the deprecation.

[ns-rse]

Confirmed...

Environment

(tmp-d918eb399b838e4) ❱ python --version            
Python 3.11.8
(tmp-d918eb399b838e4) ❱ pip freeze
accessible-pygments==0.0.4
alabaster==0.7.16
anyascii==0.3.2
anyio==4.3.0
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
astroid==3.1.0
asttokens==2.4.1
async-lru==2.0.4
attrs==23.2.0
autopep8==2.0.4
Babel==2.14.0
beautifulsoup4==4.12.3
black==24.4.1
bleach==6.1.0
cattrs==23.2.3
certifi==2024.2.2
cffi==1.16.0
cfgv==3.4.0
charset-normalizer==3.3.2
cheap-repr==0.5.1
click==8.1.7
comm==0.2.2
contextlib2==21.6.0
contourpy==1.2.1
coverage==7.5.0
cycler==0.12.1
debugpy==1.8.1
decorator==5.1.1
defusedxml==0.7.1
dill==0.3.8
distlib==0.3.8
docstring-to-markdown==0.15
docutils==0.20.1
execnet==2.1.1
executing==2.0.1
fastjsonschema==2.19.1
filelock==3.13.4
filetype==1.2.0
flake8==7.0.0
fonttools==4.51.0
fqdn==1.5.1
h11==0.14.0
h5py==3.11.0
httpcore==1.0.5
httpx==0.27.0
identify==2.5.36
idna==3.7
igor2==0.5.6
imageio==2.34.1
imagesize==1.4.1
importlib_metadata==7.1.0
iniconfig==2.0.0
ipykernel==6.29.4
ipython==8.23.0
ipython-genutils==0.2.0
ipywidgets==8.1.2
isoduration==20.11.0
isort==5.13.2
jedi==0.19.1
jedi-language-server==0.41.4
Jinja2==3.1.3
joblib==1.4.0
json5==0.9.25
jsonpointer==2.4
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
jupyter==1.0.0
jupyter-console==6.6.3
jupyter-events==0.10.0
jupyter-highlight-selected-word==0.2.0
jupyter-lsp==2.2.5
jupyter-nbextensions-configurator==0.6.3
jupyter_client==8.6.1
jupyter_contrib_core==0.4.2
jupyter_contrib_nbextensions==0.7.0
jupyter_core==5.7.2
jupyter_server==2.14.0
jupyter_server_terminals==0.5.3
jupyterlab==4.1.6
jupyterlab_pygments==0.3.0
jupyterlab_server==2.27.1
jupyterlab_widgets==3.0.10
jupyterthemes==0.20.0
jupytext==1.16.1
kiwisolver==1.4.5
lazy_loader==0.4
lesscpy==0.15.1
loguru==0.7.2
lsprotocol==2023.0.1
lxml==5.2.1
Markdown==3.6
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib==3.8.4
matplotlib-inline==0.1.7
mccabe==0.7.0
mdit-py-plugins==0.4.0
mdurl==0.1.2
mistune==3.0.2
mypy==1.10.0
mypy-extensions==1.0.0
myst-parser==3.0.0
nbclient==0.10.0
nbconvert==7.16.3
nbformat==5.10.4
nbmultitask==0.1.0
nest-asyncio==1.6.0
networkx==3.3
nodeenv==1.8.0
notebook==7.1.3
notebook_shim==0.2.4
numpy==1.26.4
numpydoc==1.7.0
overrides==7.7.0
packaging==24.0
pandas==2.2.2
pandocfilters==1.5.1
parso==0.8.4
pathspec==0.12.1
pexpect==4.9.0
pillow==10.3.0
platformdirs==4.2.1
pluggy==1.5.0
ply==3.11
pockets==0.9.1
pre-commit==3.7.0
prometheus_client==0.20.0
prompt-toolkit==3.0.43
psutil==5.9.8
ptyprocess==0.7.0
pure-eval==0.2.2
pycodestyle==2.11.1
pycparser==2.22
pydata-sphinx-theme==0.15.2
pydocstyle==6.3.0
pyfiglet==1.0.2
pyflakes==3.2.0
pygls==1.3.1
Pygments==2.17.2
pylint==3.1.0
pylsp-mypy==0.6.8
pyparsing==3.1.2
pyright==1.1.360
pyspm==0.6.1
pytest==7.4.4
pytest-cov==5.0.0
pytest-durations==1.2.0
pytest-github-actions-annotate-failures==0.2.0
pytest-lazy-fixture==0.6.3
pytest-mock==3.14.0
pytest-mpl==0.17.0
pytest-pylint==0.21.0
pytest-regtest==2.1.1
pytest-xdist==3.5.0
python-dateutil==2.9.0.post0
python-json-logger==2.0.7
python-lsp-jsonrpc==1.1.2
python-lsp-ruff==2.2.0
python-lsp-server==1.11.0
pytoolconfig==1.3.1
pytz==2024.1
pyupgrade==3.15.2
PyYAML==6.0.1
pyzmq==26.0.2
qtconsole==5.5.1
QtPy==2.4.1
readline==6.2.4.1
referencing==0.35.0
requests==2.31.0
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rope==1.13.0
rpds-py==0.18.0
ruamel.yaml==0.18.6
ruamel.yaml.clib==0.2.8
ruff==0.4.1
ruff-lsp==0.0.53
schema==0.7.5
scikit-image==0.23.2
scikit-learn==1.4.2
scipy==1.13.0
seaborn==0.13.2
Send2Trash==1.8.3
six==1.16.0
sniffio==1.3.1
snoop==0.4.3
snowballstemmer==2.2.0
soupsieve==2.5
Sphinx==7.3.7
sphinx-autoapi==3.0.0
sphinx-autodoc-typehints==2.1.0
sphinx-markdown-tables==0.0.17
sphinx-multiversion==0.2.4
sphinx-rtd-theme==2.0.0
sphinxcontrib-applehelp==1.0.8
sphinxcontrib-devhelp==1.0.6
sphinxcontrib-htmlhelp==2.0.5
sphinxcontrib-jquery==4.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-mermaid==0.9.2
sphinxcontrib-napoleon==0.7
sphinxcontrib-qthelp==1.0.7
sphinxcontrib-serializinghtml==1.1.10
stack-data==0.6.3
tabulate==0.9.0
terminado==0.18.1
threadpoolctl==3.4.0
tifffile==2024.4.24
tinycss2==1.3.0
tokenize-rt==5.2.0
toml==0.10.2
tomli==2.0.1
tomlkit==0.12.4
topofileformats==0.1.0
-e git+ssh://[email protected]/AFM-SPM/TopoStats.git@e4c8f05d6304446e981c55cc028ca995475fc82e#egg=topostats
tornado==6.4
tqdm==4.66.2
traitlets==5.14.3
types-python-dateutil==2.9.0.20240316
typing_extensions==4.11.0
tzdata==2024.1
ujson==5.9.0
uri-template==1.3.0
urllib3==2.2.1
virtualenv==20.26.0
wcwidth==0.2.13
webcolors==1.13
webencodings==0.5.1
websocket-client==1.8.0
whatthepatch==1.0.5
widgetsnbextension==4.0.10
yapf==0.40.2
zipp==3.18.1

Error

❱ topostats create-config  
Exception ignored in: <_io.FileIO name='/home/neil/work/git/hub/AFM-SPM/TopoStats/TopoStats-2024-04-25-10-13-59.log' mode='ab' closefd=True>
Traceback (most recent call last):
  File "/home/neil/work/git/hub/AFM-SPM/TopoStats/topostats/statistics.py", line 8, in <module>
    LOGGER = setup_logger(LOGGER_NAME)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/neil/work/git/hub/AFM-SPM/TopoStats/TopoStats-2024-04-25-10-13-59.log' mode='a' encoding='UTF-8'>
Exception ignored in: <_io.FileIO name='/home/neil/work/git/hub/AFM-SPM/TopoStats/TopoStats-2024-04-25-10-13-59.log' mode='ab' closefd=True>
Traceback (most recent call last):
  File "/home/neil/work/git/hub/AFM-SPM/TopoStats/topostats/processing.py", line 31, in <module>
    LOGGER = setup_logger(LOGGER_NAME)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/neil/work/git/hub/AFM-SPM/TopoStats/TopoStats-2024-04-25-10-13-59.log' mode='a' encoding='UTF-8'>
Traceback (most recent call last):
  File "/home/neil/.virtualenvs/tmp-d918eb399b838e4/bin/topostats", line 8, in <module>
    sys.exit(entry_point())
             ^^^^^^^^^^^^^
  File "/home/neil/work/git/hub/AFM-SPM/TopoStats/topostats/entry_point.py", line 386, in entry_point
    args.func(args)
  File "/home/neil/work/git/hub/AFM-SPM/TopoStats/topostats/io.py", line 129, in write_config_with_comments
    config = pkg_resources.open_text(__package__, "default_config.yaml").read()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/resources/_legacy.py", line 18, in wrapper
    warnings.warn(
DeprecationWarning: open_text is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.