Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add additional repo card utils from modelcards repo #940

Merged
merged 63 commits into from
Sep 2, 2022

Conversation

nateraw
Copy link
Contributor

@nateraw nateraw commented Jul 6, 2022

This PR aims to add repo card utilities built out in nateraw/modelcards to this repo. Note that there are some lingering issues we may want to resolve and update both there and here before merging.

Some todos:

  • Update existing repo card utils to use new dataclasses + ensure backwards compatibility
  • Add default model card and include it in package here
  • Do not merge until default model card is updated to latest spec we defined offline
  • Add tests from modelcards to this repo
    • Add additional tests for private repos
    • (maybe?) Add additional tests that support alternative base endpoint (i.e. staging env)
  • Check on min/max versions of Jinja2 and update accordingly in setup.py
  • Check all docstrings work as intended (original repo tested with doctests, I don't think we do that here?)
  • Add section to docs that explains usage

Latest walkthrough of this PR in Colab: Open In Colab

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Jul 6, 2022

The documentation is not available anymore as the PR was closed or merged.

@nateraw
Copy link
Contributor Author

nateraw commented Jul 12, 2022

This PR is ready for a first round of reviews. The docstrings may be out of date a bit (which is why I'll leave this as draft), but the tests are all in. The example colab in my message above shows the usage.

I don't want to add anything in docs for now in case there are big changes from review I need to make.

@nateraw nateraw requested a review from adrinjalali July 12, 2022 16:19
Copy link
Member

@LysandreJik LysandreJik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution @nateraw, excited to see it land in hfh.

I've added a few high-level comments so that it complies with the rest of the codebase for it to be merged. I didn't dive in the code to try it out (yet).

Thanks for providing a notebook to play around with; could you also add this to the doc, perhaps in its own file? Having it be documented alongside some nice code examples like you have in the notebook will make it even easier to play with.

You can add a new file here: https://github.com/huggingface/huggingface_hub/tree/main/docs/source

And link it from the _toctree.yml file which is in that same folder.

setup.py Outdated Show resolved Hide resolved
src/huggingface_hub/repocard.py Outdated Show resolved Hide resolved
src/huggingface_hub/repocard.py Outdated Show resolved Hide resolved
src/huggingface_hub/repocard_data.py Outdated Show resolved Hide resolved
tests/test_modelcard.py Outdated Show resolved Hide resolved
Copy link
Member

@julien-c julien-c left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Played with it a little bit, I really like it!

This looks to be in good shape.

Aside from small comments, I wonder why you didn't keep the notion of ModelCard inheriting from RepoCard (which applies to all repo types) which i quite liked in https://github.com/nateraw/modelcards ?
And same for CardData, IMO we should have a repo-type-agnostic implem of CardData and then children dataclasses that define the relevant fields

My other question was, do we really want to bundle the (or a?) model card template in the library, or should we host it on the Hub and load the template from there too.
I think there might be advantages and drawbacks to both approaches.

src/huggingface_hub/repocard_data.py Outdated Show resolved Hide resolved
src/huggingface_hub/repocard_data.py Show resolved Hide resolved
src/huggingface_hub/repocard.py Outdated Show resolved Hide resolved
Copy link
Contributor

@osanseviero osanseviero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR! It's very cool! 🔥 I left some questions and high-level suggestions, but this is in a great state already!

  • First of all, thanks for sharing the colab! It makes it super easy and nice to play with the API very quickly. I think a lot of it can be converted to some nice docs 🦾
  • I agree with @julien-c on having different classes for each type of repo card, with ModelCard, SpacesCard, etc.
  • I think having the main template bundled in the library is nice, so I would keep as is. If people want to provide new templated in the future we could have some additional functionality to use templates from the Hub, which might motivate people to create their own templates, but I think for now focusing in the current one is better!
  • I didn't review the tests yet as they are probably going to change a bit :)

Thanks again! 🚀

setup.py Outdated Show resolved Hide resolved
src/huggingface_hub/repocard.py Outdated Show resolved Hide resolved
src/huggingface_hub/repocard.py Outdated Show resolved Hide resolved
src/huggingface_hub/repocard.py Outdated Show resolved Hide resolved
src/huggingface_hub/repocard.py Show resolved Hide resolved
src/huggingface_hub/repocard_data.py Outdated Show resolved Hide resolved
src/huggingface_hub/repocard_data.py Outdated Show resolved Hide resolved
src/huggingface_hub/repocard_data.py Show resolved Hide resolved
src/huggingface_hub/repocard_data.py Outdated Show resolved Hide resolved
src/huggingface_hub/repocard_types.py Show resolved Hide resolved
nateraw added a commit that referenced this pull request Jul 25, 2022
* Preserve newlines in the existing card files

* make style

* ✅ add test

* 💄 apply style

Co-authored-by: nateraw <[email protected]>
nateraw added a commit that referenced this pull request Jul 26, 2022
* 🚧 wip

* 🚧 wip

* 🚧 wip

* 💄 apply style

* 🚧 wip
nateraw added a commit that referenced this pull request Aug 22, 2022
* Preserve newlines in the existing card files

* make style

* ✅ add test

* 💄 apply style

Co-authored-by: nateraw <[email protected]>
nateraw added a commit that referenced this pull request Aug 22, 2022
* 🚧 wip

* 🚧 wip

* 🚧 wip

* 💄 apply style

* 🚧 wip
@nateraw nateraw marked this pull request as ready for review August 23, 2022 06:03
@nateraw
Copy link
Contributor Author

nateraw commented Aug 23, 2022

Ok folks, I've:

  • Updated tests to unittest.TestCase instead of pytest
  • Refactored to have Repocard be parent of ModelCard and DatasetCard
  • Refactored to have CardData be parent of ModelCardData and DatasetCardData
  • Updated to have Jinja2 not be installed by default, only as extra
  • Added docs (both a guide + reference documentation)

@codecov
Copy link

codecov bot commented Aug 23, 2022

Codecov Report

Merging #940 (6d4129e) into main (0f2c286) will increase coverage by 0.62%.
The diff coverage is 95.70%.

@@            Coverage Diff             @@
##             main     #940      +/-   ##
==========================================
+ Coverage   82.18%   82.80%   +0.62%     
==========================================
  Files          35       35              
  Lines        3593     3752     +159     
==========================================
+ Hits         2953     3107     +154     
- Misses        640      645       +5     
Impacted Files Coverage Δ
src/huggingface_hub/__init__.py 78.37% <ø> (ø)
src/huggingface_hub/file_download.py 85.41% <72.72%> (-0.39%) ⬇️
src/huggingface_hub/repocard.py 94.37% <95.04%> (-1.08%) ⬇️
src/huggingface_hub/repocard_data.py 98.38% <98.38%> (ø)
src/huggingface_hub/hf_api.py 87.48% <0.00%> (+0.61%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Copy link
Contributor

@osanseviero osanseviero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool PR! This is very very nice, and the documentation for it is very helpful!

docs/source/how-to-model-cards.mdx Outdated Show resolved Hide resolved
docs/source/how-to-model-cards.mdx Outdated Show resolved Hide resolved
docs/source/how-to-model-cards.mdx Outdated Show resolved Hide resolved
docs/source/how-to-model-cards.mdx Show resolved Hide resolved
docs/source/how-to-model-cards.mdx Outdated Show resolved Hide resolved
docs/source/how-to-model-cards.mdx Outdated Show resolved Hide resolved
setup.py Outdated Show resolved Hide resolved
src/huggingface_hub/file_download.py Show resolved Hide resolved
tests/test_repocard.py Outdated Show resolved Hide resolved
tests/test_repocard_data.py Show resolved Hide resolved
@Wauplin Wauplin modified the milestones: v0.9, v0.10 Aug 24, 2022
@nateraw
Copy link
Contributor Author

nateraw commented Sep 1, 2022

Ok folks, I'm ready to call this good to go. Any small updates to docs/etc can be done in future small PRs, as long as there's nothing major lingering that anybody is super opinionated about.

Feel free to give this a final look over.


@Wauplin, as a sanity check, can you please double check I didn't mess anything up when I rebased? one of my most recent commits ended up showing me codecov twice, causing tests to fail for some reason, but it resolved itself when I pushed to this branch again.

Should be fine, just would like another pair of eyes on it. Thank you!!

Copy link
Contributor

@meg-huggingface meg-huggingface left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!!

docs/source/how-to-model-cards.mdx Outdated Show resolved Hide resolved
docs/source/how-to-model-cards.mdx Outdated Show resolved Hide resolved
docs/source/package_reference/cards.mdx Outdated Show resolved Hide resolved
docs/source/package_reference/cards.mdx Show resolved Hide resolved
docs/source/package_reference/cards.mdx Outdated Show resolved Hide resolved
docs/source/package_reference/cards.mdx Outdated Show resolved Hide resolved
src/huggingface_hub/repocard_data.py Outdated Show resolved Hide resolved
docs/source/package_reference/cards.mdx Outdated Show resolved Hide resolved
@Wauplin
Copy link
Contributor

Wauplin commented Sep 2, 2022

@nateraw I double-checked everything and I think we are good to go.
Thanks again for the huge work done in this PR both in the implementation and in documenting everything !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants