Automatically add metadata to Hugging Face Hub repos when uploading projects

[juhoinkinen]

With this PR, when running annif upload:

• if README.md (Model Card) does not exist in the destination repository, then README.md is created with default contents and some metadata of the uploaded projects,
• if README.md exists, its metadata are updated as necessary.

Closes #790.

The metadata includes these:

language:
- <language-code tags automatically obtained from the uploaded projects>
tags:
- annif   # custom tag
pipeline_tag: text-classification  # HFH tag

The Model Card text content is very minimal; it has just the repo name as the heading and info about how to download projects from the repo, see an example in https://huggingface.co/juhoinkinen/Annif-models-upload-testing.

[juhoinkinen]

About @osma's suggestions in #790 (comment):

For example it could include the Annif version used for training, the backend, vocabulary name and size, possibly some of the hyperparameters / configuration settings as well.

• Annif version:
  • The Annif version used for training is not stored anywhere at the moment; the version performing the upload is not necessarily the same. This kind of metadata should be first stored somewhere, for which there is the issue Store metadata of project training #329
• Backend, vocabulary name and other project configuration:
  • These are available in the <project-id>.cfg files, accessible from the Files and versions tab, e.g. https://huggingface.co/NatLibFi/FintoAI-data-YSO/blob/main/yso-en.cfg, so I think they are not worth putting to the Model Card.

[sonarcloud]

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[codecov]

Codecov Report

Attention: Patch coverage is 96.66667% with 4 lines in your changes missing coverage. Please review.

Project coverage is 99.60%. Comparing base (3b5f7a1) to head (55b0ffc).
Report is 30 commits behind head on main.

Files with missing lines	Patch %	Lines
annif/hfh_util.py	91.66%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #793      +/-   ##
==========================================
- Coverage   99.64%   99.60%   -0.05%     
==========================================
  Files          91       93       +2     
  Lines        6817     7048     +231     
==========================================
+ Hits         6793     7020     +227     
- Misses         24       28       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[juhoinkinen]

@CodiumAI-Agent /review

[CodiumAI-Agent]

PR Reviewer Guide 🔍

(Review updated until commit 845f53d)

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Key issues to review Error Handling The function upsert_modelcard lacks error handling for potential failures during the push_to_hub operation. Consider adding try-except blocks to handle exceptions that might arise during the push operation, ensuring that the function can gracefully handle errors and provide meaningful feedback to the user. Configuration Error Handling The error handling in _read_config might not provide clear feedback to the user since it directly raises ConfigurationException with err.message, which might not be defined. It's recommended to ensure that the exception message is informative and user-friendly.

[juhoinkinen]

The project fixture does not provide vocab_lang so for these tests it is just set here, not super clean maybe?

[juhoinkinen]

Possible Bug:
Ensure that the upsert_modelcard function handles cases where project language data might be missing or malformed. > The current implementation assumes that proj.vocab_lang is always available and valid.

Good point by the AI, but I think the project language is always set if this point is reached...?

[CodiumAI-Agent]

Persistent review updated to latest commit 845f53d

[juhoinkinen]

I added an automatically updating Projects section to the modelcard, like this: https://huggingface.co/juhoinkinen/Annif-models-upload-testing#projects

[sonarcloud]

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud