-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Non-schema Change] Add sql query files for mappings #258
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just curious - does converting to lower case and trimming extra spaces greatly improve the mapping? Do we have other data cleaning mechanisms in place?
I'm all for the element_at() change, but is the lower(trim(...)) really necessary? This feels like correcting OSM data that should instead be fixed at the source. |
We do currently "correct/transform" the height values. We convert values with "m" "meters" "feet" "inches" and even ' and '' as units into a standardized height in meters. By adding this code we will convert: Also, we currently use trim and lower in the names conversion. I thought it made sense to use it any place that we are comparing strings from OSM to a list of string values. |
Ah, right. I was actually thinking you were trimming or lowering that actual tags themselves, not the values. Yeah this makes total sense. |
This adds the queries that map to the mapping that exist today. We have decided that SQL statements are better than the config yaml files that are there now so this converts those yaml files to the SQL files. These will be used by the building pipeline in the tf-data-platform repo once they are merged.
I also plan on removing the old config files once the tf-data-platform code has been switched over to use these new files.
I also fixed the subtype for bridge structures to be set to "transportation" if it is not filled in by other means.