Skip to content

Commit

Permalink
Allow places's categories to be NULL (although categories.primary mus…
Browse files Browse the repository at this point in the history
…t still exist if any categories.alternate are present) (#231)

* Make categories an optional field for place

In regards to the longstanding NULL category issue from OvertureMaps/tf-place-data#123

Reporting back on the longstanding issue of 2.5M overture places with missing categories, needing a fix in advance of General Availability

**tl;dr - I want to change the schema to allow for NULL categories.**

My first finding is that ~half come from microsoft, and half come from Meta. Someone from Microsoft should investigate and/or fix their half.

In terms of why Meta's half is missing, it's because they only have extremely generic categories that aren't specific enough to be meaningful, and so those categories don't exist in the category mapping.

 They're generic like "Local", "Product Service", and "Health Beauty" - so generic that they're essentially meaningless. In some large fraction of places, it seems like they have this category merely so that they can be classified as a place. (In our internally-enforced category hierarchy, some categories are disallowed from being places. "Local" is the most generic category that is still allowed to be a place.)

So our options are:
1) add a mapping to some inappropriate category for the generic internal categories
2) add these new generic categories to the Overture category ontology
3) change the schema to allow categories to be NULL
4) remove all category-less places from the release

I like option 3. As Jeff Underwood reported, the quality of these category-less places is quite low (which matches my investigation today), so I'm ok to remove them, but some of them are good and with better signals collected on them then we could classify them even better. I prefer to maintain the option to collect signals on them to then improve them, which I view as a large portion of why we're in the Overture project to begin with.

I discussed this with @jwass and @jenningsanderson , who supported making categories into an optional field. In particular, @jwass liked how keeping them there invites feedback for actually assigning an appropriate category... and elevates the importance of having a system to collect and respond to that feedback.

* Don't require main in place's categories

* Require categories but allow categories.main to be NULL

* Missing categories.main counterexample now permissible

* main to primary

* Allow places.primary to be NULL

* Require categories.primary but allow it to be NULL

* Allow categories to be null, but require categories.primary if categories is non-NULL

* Set categories counter-example to have NULL categories.

* Re-add missing categories counter-example, based on places schema refinement
  • Loading branch information
Dritte authored Jul 10, 2024
1 parent 31aa871 commit 7451a4f
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 1 deletion.
30 changes: 30 additions & 0 deletions examples/places/place-null-categories.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
geometry:
coordinates:
- 0
- 0
type: Point
id: overture:places:place:1
properties:
addresses:
- freeform: 770 Broadway, Floor 8
locality: New York
- country: US
freeform: '770 Broadway #802'
locality: New York
region: US-NY
brand:
name: Example
wikidata: Q1000
emails:
- [email protected]
phones:
- +32 1207
socials:
- https://www.twitter.com/example
theme: places
type: place
update_time: '2024-06-12T12:20:08-06:00'
version: 1
websites:
- https://www.example.com
type: Feature
1 change: 0 additions & 1 deletion schema/places/place.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ properties:
- "$ref": https://geojson.org/schema/Point.json
properties:
unevaluatedProperties: false
required: [categories]
allOf:
- "$ref": ../defs.yaml#/$defs/propertyContainers/overtureFeaturePropertiesContainer
- "$ref": ../defs.yaml#/$defs/propertyContainers/namesContainer
Expand Down

0 comments on commit 7451a4f

Please sign in to comment.