Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Standardize scraped location to city, state, country and latlongs #37

Closed
Sangarshanan opened this issue Oct 29, 2019 · 7 comments · Fixed by #102
Closed

Standardize scraped location to city, state, country and latlongs #37

Sangarshanan opened this issue Oct 29, 2019 · 7 comments · Fixed by #102
Labels
enhancement New feature or request

Comments

@Sangarshanan
Copy link
Contributor

Sangarshanan commented Oct 29, 2019 •
edited
Loading

Could use Nominatim
@vinayak-mehta vinayak-mehta changed the title Nesting the location entity which would be Geocoded to city, state, country and latlong Geocoding scraped location to parse city, state and country Oct 29, 2019
@vinayak-mehta vinayak-mehta added enhancement New feature or request hacktoberfest labels Oct 29, 2019
@vinayak-mehta vinayak-mehta changed the title Geocoding scraped location to parse city, state and country Geocoding scraped location to parse city, state, country and latlongs Oct 29, 2019
@vinayak-mehta
Copy link
Owner

vinayak-mehta commented Oct 29, 2019 •
edited
Loading

This could be a conrad util that everyone can use in their crawlers.

@vinayak-mehta
Copy link
Owner

vinayak-mehta commented Oct 29, 2019 •
edited
Loading

Or a code snippet that anyone can copy|paste and use directly.

@vinayak-mehta
Copy link
Owner

We could then give users nearby conferences using their location.

@vinayak-mehta vinayak-mehta pinned this issue Nov 3, 2019
@vinayak-mehta vinayak-mehta changed the title Geocoding scraped location to parse city, state, country and latlongs Standardize scraped location to parse city, state, country and latlongs Nov 3, 2019
@vinayak-mehta vinayak-mehta changed the title Standardize scraped location to parse city, state, country and latlongs Standardize scraped location to city, state, country and latlongs Nov 3, 2019
@vinayak-mehta
Copy link
Owner

We can use https://geopy.readthedocs.io with MapBox!

@vinayak-mehta
Copy link
Owner

vinayak-mehta commented Nov 4, 2019 •
edited
Loading

Alas, data returned from the MapBox API can't be stored because of their terms of service :(

@Sangarshanan
Copy link
Contributor Author

Could check out Here API or TomTom Both of em have a free tier and are pretty good.

@Sangarshanan Sangarshanan mentioned this issue Jan 4, 2020
Merged
@vinayak-mehta vinayak-mehta mentioned this issue Mar 30, 2020
Open
@kunaljubce
Copy link

kunaljubce commented Apr 11, 2020 •
edited
Loading

Hi,

I am starting to work on this. Should I develop this as a custom module placed in the conrad/ folder, which can then be imported into the crawlers?

@vinayak-mehta vinayak-mehta unpinned this issue Jul 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[MRG] Add nominatim geocoder to standardize location
3 participants
@vinayak-mehta @Sangarshanan @kunaljubce