Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Allow places's categories to be NULL (although categories.primary mus…
…t still exist if any categories.alternate are present) (#231) * Make categories an optional field for place In regards to the longstanding NULL category issue from OvertureMaps/tf-place-data#123 Reporting back on the longstanding issue of 2.5M overture places with missing categories, needing a fix in advance of General Availability **tl;dr - I want to change the schema to allow for NULL categories.** My first finding is that ~half come from microsoft, and half come from Meta. Someone from Microsoft should investigate and/or fix their half. In terms of why Meta's half is missing, it's because they only have extremely generic categories that aren't specific enough to be meaningful, and so those categories don't exist in the category mapping. They're generic like "Local", "Product Service", and "Health Beauty" - so generic that they're essentially meaningless. In some large fraction of places, it seems like they have this category merely so that they can be classified as a place. (In our internally-enforced category hierarchy, some categories are disallowed from being places. "Local" is the most generic category that is still allowed to be a place.) So our options are: 1) add a mapping to some inappropriate category for the generic internal categories 2) add these new generic categories to the Overture category ontology 3) change the schema to allow categories to be NULL 4) remove all category-less places from the release I like option 3. As Jeff Underwood reported, the quality of these category-less places is quite low (which matches my investigation today), so I'm ok to remove them, but some of them are good and with better signals collected on them then we could classify them even better. I prefer to maintain the option to collect signals on them to then improve them, which I view as a large portion of why we're in the Overture project to begin with. I discussed this with @jwass and @jenningsanderson , who supported making categories into an optional field. In particular, @jwass liked how keeping them there invites feedback for actually assigning an appropriate category... and elevates the importance of having a system to collect and respond to that feedback. * Don't require main in place's categories * Require categories but allow categories.main to be NULL * Missing categories.main counterexample now permissible * main to primary * Allow places.primary to be NULL * Require categories.primary but allow it to be NULL * Allow categories to be null, but require categories.primary if categories is non-NULL * Set categories counter-example to have NULL categories. * Re-add missing categories counter-example, based on places schema refinement
- Loading branch information