Location based auto discovery

[joostlek]

As I was on my bike listening to the HA release party, I came up with a new way of getting people in touch with useful data.

Location based auto discovery

When setting up Home Assistant, you automatically get notified about devices on your network or devices found with Bluetooth.
While Home Assistant is a good platform for automating stuff with devices, it can also be used for reading and displaying public data.
In this proposal I want to describe the idea of auto discovery of integrations that integrate with external services that have a bound location to them.

What integrations would be valid?

This is not a definition carved in stone, so feel free to suggest other definitions.

To give a list:

• Power companies
  • opower
  • Electric Kiwi
• Transport companies
  • Nederlandse spoorwegen
  • Trafikverket train
• Telecom providers
  • Aussie broadband
• Garbage collection
  • Twente milieu
  • ROVA
• Other location based information provided by government:
  • PEGEL ONLINE

One thing to keep in mind is the amount of integrations that get added to HA after being discovered.
For example discovered network devices would almost always be relevant for the user.
For bluetooth devices, it would depend on the amount of detected bluetooth devices of neighbours or visitors.

The relevancy of integrations recommended by location based auto discovery would be lower.
For example telecom providers, if I use a different provider, I have no use of the recommended integration.
This is also applicable to utility companies.

How can we define the relevancy

Currently we only know the coordinates of the user and their country.
I want to propose to utilize both since they both have pros and cons.

Country based

For example, PEGEL ONLINE would have Germany as country with relevance.
This would be a good way of recommending a country wide integration.

If you would want to use this for for example Twente Mileu it wouldn't be ideal, since the actual part where the integration is relevant is small.

To give another example, if you would select the United states of america as country, and it automatically recommends all opower virtual integrations even though only one would be relevant.

Example manifest.json:

{
  "domain": "pegel_online",
  "name": "PEGELONLINE",
  "location": {
    "country": [
      "DE"
    ]
  }
}

Bounding box

To take Twente Mileu as example again. You could create a bounding box around the relevant area.

We could also allow multiple bounding boxes to enhance the relevancy.

This would be ideal for a local government or local company that has an integration.
The bounding box wouldn't be great for integrations like PEGEL ONLINE, since countries are shaped weird and it would take or a lot of bounding boxes, or it would cover a lot of other countries.
(To give an example, here's Germany)

Example manifest.json:

{
  "domain": "twentemileu",
  "name": "Twente Milieu",
  "location": {
    "coordinates": [
      {
        "min_latitude": 52.135891,
        "max_latitude": 52.467274,
        "min_longitude": 6.430009,
        "max_longitude": 7.033065
      }
    ]
  }
}

When do we check?

Since location is something that changes rarely in a session, this should only be checked at startup.

What integration should not use this?

This shouldn't be a way for companies to push their product to customers in areas.
For example if there's an integration that acts like a hub for powerdata (for utility companies that may or may not have an integration)/
They should not be able to use this to promote users to sign up at their service to collect more data.

General notes

• I think the UI must have a different way of nudging location based services.
  Since the likelyhood of users installing them is lower than normal discovered integrations, it doesn't make sense to show it as big as currently is happening.
• We could go more specific if we have more data
  For example in the netherlands you could work with zipcodes.
  If we could store more relevant location data we could recommend more specific, this isn't a necessary point to have before implementing.
  Another option would be to go for regions, in America it would be based on states and in the netherlands on the regions.

cc @Lash-L

[dgomes]

weather providers are also location based.

I like the idea!

[Lash-L]

That's a good point, I didn't even realize HomeAssistant had region specific weather integrations!

Examples:

Deutscher Wetterdienst (DWD) Weather Warnings

National Weather Service (NWS)

Trafikverket Weather Station

[jbouwh]

May be for the location specific discovery coordinates and a radius works better. For larger area's this seems not ideal to me. But I guess It could be checked if there is a service near by. Not sure if this would be practical to manage in the manifest.json though. Further we should watch out not to get into privacy issues if we are going to detect services, when a user did not yet conclude yet to use them. These services might be able to collect location info, so we should not just connect to them unless discovery is part of an integration the use choose to install, or the service should supply static support information upon the locations that are supported.

[joostlek]

For OpenSky I have a function implemented to get a bounding box from a point and a radius. This was a hassle to find out since there aren't a lot of ways to do this. I used reverse vincenty (if I say the name right). The accuracy of the function is good in the center (0,0) but the further you go the less accurate it gets.

In OpenSky I use this to get a box and not a circle. I am not aware about how heavy the function is, but having to execute this calculation for every integration that has location relevancy might get a bit too much.

[codyc1515]

I agree these should not be set-up & configured automatically, but HA already does that for integrations that I personally don't want, notably Google TTS.

[jbouwh]

Besides of country filtering, location filtering based on coordinates works best when the location coordinates are passed to a service that filters from a database calculating the distance and make a top-n-list. But that will also bring privacy concerns. It will also bring issues if we fetch a full location list and process it locally.

[Lash-L]

I know it's not super optimized just checking bounding boxes, but we could cache which ones we have checked. We only need to check the location on startup if one of the two is true

1)the integration holding that bounding box has updated(specifically the manifest but that may be more difficult)

2. if the user changes their location

3. a new integration was added to core that has a bounding box

If we have already checked the bounding box, we don't need to check it again unless one of those is true. So we can store a cache of what integrations need to be checked and we wouldn't be dealing with much overhead.

Alternatively, we could use something like an rtree or shapely sttree and that would allow us to easily check all the polygons at once and then only check the actual intersection of the tree shows it as a valid answer.

Both of those options are relatively lightweight and could be done 100% locally.

As far as privacy concerns go, the checking of location should happen 100% within the core code and the integration discovery should only get called with true saying it was in the bounding box, but not where in the box.

Could also add a setting toggle "let home assistant recommend integrations that are related to my area"

[balloob]

I reached out to my friend @grischard at OpenStreetMap. He suggested to use GeoJSON, an existing standard for capturing regions. Regions can be drawn/checked on websites like https://geojson.io/, but even better: cities etc can be exported from OSM directly from a website like https://osm-boundaries.com/Map

For checking if an instance is inside the shape, we can leverage a Ray Casting algorithm (inspiration).

[joostlek]

After a quick skim of that file, only countries and American states are listed if I see correctly. Are there any good options when we want to target a smaller place? City/Region?

From a technical point of view, would it maybe be beneficiary to extract the collection of zones into a separate repo like we do with the intent language? Is that .osm file a standard or should we create our own? In that case I like the idea of creating an easy (YAML-ish) way of defining polygons and then compiling and optimizing that into something we can use in core.

[grischard]

Yeah, that file fits the needs of the JOSM project, and has subdivisions for only a few countries: the US, Spain, India, China... Home Assistant has different needs, and would ship a different set of boundaries.

.osm is standardised, supported by tools like ogr2ogr and openstreetmap editors, but specific to the OpenStreetMap project - I just thought that the way that file includes a simplified version of every country would be a good example. Over at https://github.com/osmlab/editor-layer-index/ we use geojson instead; it comes with the advantage of having lots of existing libraries. I'd warn against reinventing new ways of defining polygons - good formats and tools already exist. Compiling source geojsons into something used in core sounds like a good idea.

Fwiw, https://answers.launchpad.net/ubuntu/lunar/armhf/python3-shapely/1.8.5-1 seems to have shapely for armhf.

[Misiu]

I've implemented a simple function that uses a ray-casting algorithm from https://www.youtube.com/watch?v=RSXM9bgqxJM
in my PR: home-assistant/core#105729
I think it could be used as a starting point.

[grischard]

Raycasting doesn't deal well with enclaves/exclaves, but a "donut" like https://www.openstreetmap.org/relation/62504 could then simply be split by connecting the outside to the inside ring, to make it a C-shaped simple polygon.

[Misiu]

If shapely isn't available, does the old PyGEOS compile on armhf?

GEOS should work on armhf libgeos/geos#947 (comment)
Shapely uses GEOS, so maybe it will work/it works.
I've created a discussion in the Shapely repo asking about armhf support (shapely/shapely#2079), but sadly no reply yet.

How to test if the recent version (2.0.5) works? I can change my PR to use Shapely, push it to Core, and check CI status.
If there is a package for Ubuntu can't we use it?

[codyc1515]

I think it absolutely makes sense to surface integrations geographically.

The existing discovery mechanism through Devices and Services > Add Integration today requires you to have knowledge of a particular brand / service having an HA integration already. Even today, I am still finding new ones that I was not previously aware of (despite having scrolled the list myself many times).

When there is a new release which adds relevant integrations for your locality, there is no surfacing of these either (outside of the release notes / blog).

Bearing in mind that we are only talking about core integrations here. If we have already agreed to incorporate a particular integration into core, why would we not want people to know about it?

Surfacing these integrations through a discovery flow seems the best way to do it, with an option to ignore and not be reminded again for that integration.