Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate a json with extended synonym as we build taxonomies #10742

Open
alexgarel opened this issue Aug 28, 2024 · 1 comment · Fixed by #10744
Open

Generate a json with extended synonym as we build taxonomies #10742

alexgarel opened this issue Aug 28, 2024 · 1 comment · Fixed by #10744
Assignees
Labels
API READ All READ APIs include Product, Search… API Refactor API Issues related to the Open Food Facts API. More specific labels exist & should be used (API WRITE…) Data export We export data nightly as CSV, MongoDB… See: https://world.openfoodfacts.org/data 🧬 Taxonomies https://wiki.openfoodfacts.org/Global_taxonomies

Comments

@alexgarel
Copy link
Member

Problem

As we build taxonomies,
we export a json with taxonomy. This is very useful for third party applications that deals with open food facts data, and wants to run some analysis based on taxonomies.
For example, this is used by robotoff and search-a-licious.

As reported on https://wiki.openfoodfacts.org/Taxonomy_access, we currently have two versions: one with only synonyms, and one with additional properties.
But there is no export with extended synonyms (were we replace synonyms by synonyms), whereas it would be very useful to search-a-licious.

Proposed solution

Export a <taxonomy_name>.extended.json, which contains a extended_synonyms property with extended synonyms.

My guess is to avoid putting properties in it (to avoid file being too massive), one can download .extended and .full version if needed, the merge is easy to do.

Code pointers

It happens in Tags.pm, in build_taxonomies build_tags_taxonomy (at the end)

@alexgarel
Copy link
Member Author

https://static.openfoodfacts.org/data/taxonomies/categories.extended.json is not there @stephanegigandet even if the other files were generated Oct 3 in /srv/off/html/data/taxonomies

@alexgarel alexgarel reopened this Oct 9, 2024
@teolemon teolemon added 🧬 Taxonomies https://wiki.openfoodfacts.org/Global_taxonomies Data export We export data nightly as CSV, MongoDB… See: https://world.openfoodfacts.org/data API Issues related to the Open Food Facts API. More specific labels exist & should be used (API WRITE…) API READ All READ APIs include Product, Search… API Refactor labels Oct 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API READ All READ APIs include Product, Search… API Refactor API Issues related to the Open Food Facts API. More specific labels exist & should be used (API WRITE…) Data export We export data nightly as CSV, MongoDB… See: https://world.openfoodfacts.org/data 🧬 Taxonomies https://wiki.openfoodfacts.org/Global_taxonomies
Projects
Development

Successfully merging a pull request may close this issue.

3 participants