Skip to content

Commit

Permalink
Update datasets-download-stats.md (#1469)
Browse files Browse the repository at this point in the history
* Update datasets-download-stats.md

* same title for models

* Apply suggestions from code review

Co-authored-by: Julien Chaumond <[email protected]>

---------

Co-authored-by: Julien Chaumond <[email protected]>
  • Loading branch information
lhoestq and julien-c authored Oct 24, 2024
1 parent 1634d90 commit dcd8b8c
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 4 deletions.
6 changes: 3 additions & 3 deletions docs/hub/datasets-download-stats.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Datasets Download Stats

## How are download stats generated for datasets?
## How are downloads counted for datasets?

Counting the number of downloads for datasets is not a trivial task, as a single dataset repository might contain multiple files, from multiple subsets and splits (e.g. train/validation/test) and sometimes with many files in a single split. To solve this issue and avoid counting one person's download multiple times, we treat all files downloaded by a user within a 5-minute window as a single dataset download. This counting happens automatically on our servers when files are downloaded (through GET or HEAD requests), with no need to collect any user information or make additional calls.
Counting the number of downloads for datasets is not a trivial task, as a single dataset repository might contain multiple files, from multiple subsets and splits (e.g. train/validation/test) and sometimes with many files in a single split. To solve this issue and avoid counting one person's download multiple times, we treat all files downloaded by a user (based on their IP address) within a 5-minute window as a single dataset download. This counting happens automatically on our servers when files are downloaded (through GET or HEAD requests), with no need to collect any user information or make additional calls.

## Before Setpember 2024
## Before September 2024

The Hub used to provide download stats only for the datasets loadable via the `datasets` library. To determine the number of downloads, the Hub previously counted every time `load_dataset` was called in Python, excluding Hugging Face's CI tooling on GitHub. No information was sent from the user, and no additional calls were made for this. The count was done server-side as we served files for downloads. This means that:

Expand Down
2 changes: 1 addition & 1 deletion docs/hub/models-download-stats.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Models Download Stats

## How are download stats generated for models?
## How are downloads counted for models?

Counting the number of downloads for models is not a trivial task, as a single model repository might contain multiple files, including multiple model weight files (e.g., with sharded models) and different formats depending on the library (GGUF, PyTorch, TensorFlow, etc.). To avoid double counting downloads (e.g., counting a single download of a model as multiple downloads), the Hub uses a set of query files that are employed for download counting. No information is sent from the user, and no additional calls are made for this. The count is done server-side as the Hub serves files for downloads.

Expand Down

0 comments on commit dcd8b8c

Please sign in to comment.