Use rich for logging, tracebacks, printing, progressbars

[tomaarsen]

Closes #1843

Hello!

Pull Request overview

• Use rich for logging, tracebacks, printing and progressbars.
• Add rich as a dependency.
• Remove loguru as a dependency and remove all mentions of it in the codebase.
• Simplify logging configuration according to the logging documentation.
• Update logging tests.

Before & After

rich is a large Python library for very colorful formatting in the terminal. Most importantly (in my opinion), it improves the readability of logs and tracebacks. Let's go over some before-and-afters:

Printing, Logging & Progressbars

Before:

After:

Note that for the logs, the repeated information on the left side is removed. Beyond that, the file location from which the log originates is moved to the right side. Beyond that, the progressbar has been updated, ahd the URL in the printed output has been highlighted automatically.

Tracebacks

Before:

After:

---

Before:

After:

---

Before

After

Notes

Note that there are some changes in the logging configuration. Most of all, it has been simplified according to the note from here. In my changes, I only attach our handler to the root logger and let propagation take care of the rest.

Beyond that, I've set rich to 13.0.1 as newer versions experience a StopIteration error like discussed here.

I've replaced tqdm with rich Progressbar when logging. However, I've kept the tqdm progressbar for the Weak Labeling for now.

One difference between the old situation and now is that all of the logs are displayed during pytest under "live log call" (so, including expected errors), while earlier only warnings were shown.

What to review?

Please do the following when reviewing:

1. Ensuring that rich is correctly set to be installed whenever someone installs argilla. I always set my dependencies explicitly in setup.py like here or here, but the one for argilla is empty, and pyproject.toml is used instead. I'd like for someone to look this over.
2. Fetch this branch and run some arbitrary code. Load some data, log some data, crash some programs, and get an idea of the changes. Especially changes to loggers and tracebacks can be a bit personal, so I'd like to get people on board with this. Otherwise we can scrap it or find a compromise. After all, this is also a design PR.
3. Please have a look at my discussion points below.

Discussion

rich is quite configurable, so there's some changes that we can make still.

1. The RichHandler logging handler can be modified to e.g. include rich tracebacks in their logs as discussed here. Are we interested in this?

2. The rich traceback handler can be set up to include local variables in its traceback:

   Click to see a rich traceback with local variables

   

   Are we interested in this? I think this is a bit overkill in my opinion.

3. We can suppress frames from certain Python modules to exclude them from the rich tracebacks. Are we interested in this?

4. The default rich traceback shows a maximum of 100 frames, which is a lot. Are we interested in reducing this to only show the first and last X?

5. The progress bar doesn't automatically stretch to fill the full available width, while tqdm does. If we want, we can set expand=True and it'll also expand to the entire width. Are we interested in this?

6. The progress "bar" does not need to be a bar, we can also use e.g. a spinner animation. See some more info here. Are we interested in this?

---

Type of change

• Refactor (change restructuring the codebase without changing functionality)

How Has This Been Tested

I've updated the tests according to my changes.

Checklist

• I have merged the original branch into my forked branch

• I added relevant documentation

• follows the style guidelines of this project

• I did a self-review of my code

• I added comments to my code

• I made corresponding changes to the documentation

• My changes generate no new warnings

• I have added tests that prove my fix is effective or that my feature works

• Tom Aarsen

[tomaarsen]

It seems to have some issues installing rich. In particular, I think it's struggling to read the __version__ of argilla without triggering errors that rich hasn't been installed yet. This issue persists regardless of whether rich==13.0.1 is in the - pip: section of the environment_dev.yml or not.

It'll have to be investigated some more.

[codecov]

Codecov Report

Base: 91.86% // Head: 91.82% // Decreases project coverage by -0.04% ⚠️

Coverage data is based on head (a3495e5) compared to base (c83ec9e).
Patch coverage: 85.71% of modified lines in pull request are covered.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #2350      +/-   ##
===========================================
- Coverage    91.86%   91.82%   -0.04%     
===========================================
  Files          161      161              
  Lines         7892     7869      -23     
===========================================
- Hits          7250     7226      -24     
- Misses         642      643       +1     

Flag	Coverage Δ
pytest	91.82% <85.71%> (-0.04%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/argilla/__init__.py	86.66% <60.00%> (-13.34%)	⬇️
src/argilla/logging.py	88.46% <80.00%> (+1.18%)	⬆️
src/argilla/client/client.py	87.55% <100.00%> (+0.05%)	⬆️
...gilla/labeling/text_classification/label_errors.py	86.58% <0.00%> (-3.66%)	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[tomaarsen]

In order to allow rich to be installed, I had to:

• Allow argilla.__version__ to be accessed without having rich installed by hiding the imports under try-except ModuleNotFound.
• Update spacy from 3.1.0 to 3.5.0 due to a dependency conflict between spacy==3.1.0 and rich==13.0.1 over the typing-extensions version.

I'll solve the merge conflicts.

This is ready for review.

[frascuchon]

So nice work!

Thanks @tomaarsen