RFC: Entity Registry

[balloob]

Problem

Entities in Home Assistant are non-deterministic. If you have 2 devices with the same name and they load each time in a different order, the entity ids between the two will switch. One will be "normal" one based on the name, ie light.kitchen and the other one will have a digit appended: light.kitchen_2.

This is annoying. It also means that when we rename things, the entity id changes. It also means we can't simply rename entity ids. We had a plan about this before but it was flaky too.

Introducing the entity registry

So I have a new proposal: the Entity Registry.

Entities will be registered in this by the following data:

• domain: component that they belong to
• integration: platform that added the device
• unique id: property on entity that uniquely identifies the device (usually some serial number)

Whenever an entity gets added that has a unique ID, we look it up in the registry to see what the entity ID has to be. If it doesn't exist, we use the existing strategy to create an entity ID and add it to the registry. This registry is saved to a file.

This gives us:

• Persistent entity IDs
• Easy option for user to change entity IDs

The overhead of this would be minimal. It would only be executed whenever a device is added to the registry. To keep writes during startup to a minimum, the entity registry shouldn't write to a file until the startup phase is over. If Home Assistant is running and a new entity is registered, it should debounce 5 seconds to make sure no other devices are being added.

File format

File format proposal would be YAML file named entities.yaml:

<entity_id>:
  platform: <platform name>
  unique_id: <unique_id>

Domain name can be derived from the entity id. Keying it by entity id will guarantee us that entity ids will remain unique.

Example entry:

light.kitchen:
  platform: hue
  unique_id: zxcjndajk12312123

Entities without a unique ID

They would follow the current logic and not be stored in the registry.

Future ideas

We can start using this file to store other things too:

• name, icon, picture overrides?
• last seen (to allow cleanup of the registry)
• entity config?
• migrate Device tracker known devices to this format?

Why not use customize?

We don't want to use customize because it writes all information to the state machine. That's not where configuration belongs and also means that we already have an entity id.

[cgarwood]

I agree, something like this would be useful. Might also be able to move the zwave_device_config settings to this as well.

[tschmidty69]

This has bitten me a few times so I think this is a great idea. The ability to truly rename an entity ID (not just friendly name) would be very useful

[tinloaf]

I like the idea a lot! Two thoughts:

• I'm not sure whether we need (or should have) "last seen" in this file. We can get a reasonable approximation of it from the recorder, and having "last seen" would make the registry something that is written to a lot (at which point a YAML file is probably not the right choice anymore?)
• If the registry changes only rarely, it might be a super-cool idea to (at run time) compile it into a static JSON / JavaScript file, put that into the folder of static front end files and include that file in the front end. This way, the front end wouldn't need to hit any API end points for all of the information (like icons, friendly names, …) included in the registry.

[balloob]

Good point about last seen. I think that name/icon would still be written to the state machine (and this thus doesn't have to be available for frontend). They just wouldn't perse be configured from customize. My problem with customize is that it is a too powerful tool without any config validation. A great hack that has gotten us pretty far, now has to be made less important by moving its responsibilities to more proper places (like the Alexa config).

[emlove]

I like this a lot. Seems like a solid solution to this problem. I'd also point out that the closer I look known_devices.yaml is almost exactly an implementation of this idea, with only small differences.

It's a little scary combining user configuration and auto written updates though. Right now known_devices.yaml gets away with this by only ever appending. Since this is a brand new feature though, I think we should try and plan for UI initiated entity_id renames or configuration changes that hass would have to write back to the file. We might want to keep entity configuration separate, and let this just tackle the problem of persistent entity IDs. Once we have that guarantee, configuration is a lot easier.

Again, since this is a new feature with no baggage, I'd say any entity configuration that goes though this system should pass through a voluptuous validator.

[balloob]

Yeah, I would not expect a user to spend time in the registry. Instead I want to allow it to be managed from the UI by adding a mini config panel to the entity more info dialog. The config panel will have different write targets: entity id and name will be here, Z-Wave entity config to Z-Wave etc.

[pvizeli]

I like this idea

[DubhAd]

It would be really useful if this could support renaming entities while running, rather than having to restart. That's one of the major pain points for Z-Wave currently (other than OZW not releasing updates since June 2017). Even if it's not a feature now, not having it designed out would be great.

[andrey-git]

Yeah, the more we can change without restarting HA the better.

[balloob]

If we would store name and icon in here, we could have entity objects listen to changes on the entity registry.

[abmantis]

Really nice idea! Just some toughs:

• How would cleanup be done? We could have some panel in HA with all the entities in the registry sorted by "last seen" (from the recorder), for example, where the user could delete devices (and maybe rename too).
• Would the file be editable directly? Since many things are still done with config files (and many users still prefer that than using the current automation/script UIs), I think it is important to allow the registry to be edited directly. Still, if the implementation is a lot better if it doesn't allow that, I understand.

[mhofman]

How would this work for platforms that have multiple entities for a single physical device.
Would the unique Id be a combination of the device serial number and subsystem id? (E.g. temperature and motion on a Z-Wave multi-sensor)
What about devices that don't have an exposed serial number, but an "address" in a hub (e.g. light in a Lutron system).

I haven't seen the concept of URIs used much in HA, but maybe this could help steer the platforms to use a more uniform unique identifier. The scheme would be the platform. The hierarchical part, i.e. the "unique id", while obscure to the core, can easily represent a structured address for the entity inside the platform.

For platforms that integrate with service discovery, the URI host part could be the discovered hostname, however this could also cause issues for devices that are both discovered and configured manually.

For example:

• zwave://home_id/node_id/value_index
• lutron-caseta://lutron-01b5da7c.local./zone_id/device_id
• google-cast://9006ee6e-eb0a-4a04-acce-2b82e6673239.local.

[balloob]

@abmantis files would be editable directly, but you will have to restart Home Assistant after making changes. The goal of the registry is to be administered via a UI.

I still think "Last Seen" is valuable. It would not be filled with the current time stamp but it will be updated only when the entity is added to Home Assistant. So if you start HA at 8am, restart at 4pm and another time at 9pm, last seen would only be one of those values (depending if the entity was seen during that run)

Note that the initial version of the registry (per #11979) will not have the last seen functionality.

@mhofman it's up to every integration to figure out their unique ID situation. If an integration can uniquely identify a device and it can uniquely identify the parts within the device, combining those 2 pieces of information will give each part a unique id.

In my current implementation, the uniqueness of an entity is judged based on component, platform and unique ID. This could for example be light, hue and abcdefgh.

[stiansoevik]

I hope international characters can be allowed in entity names. In most computer related cases, I prefer English, but with home automation, I would like to refer things by what I call them in my native language.

Another thing I have been wondering about, which maybe could be up for discussion at some point - why such emphasis on the component in names? Say, light.kithen_roof, light.livingroom_sofa have most focus on the light component. As a user, it would be more logical to group by room (or other user defined category, maybe groups could implicitly be defined by entity names?), and don't include component at all (or only when needed for clarity): kitchen.light_roof, kitchen.light_table, livingroom.tv, etc.

[abmantis]

@stiansoevik I don't really see having international characters in entity ids, unless it does not have any implementation impact. It is only for config, not UI, and the config is all in english anyway.

The issue (for me) with removing the component from the entity id is that it makes it less obvious what that entity is. Having light.kitchen_ceiling makes it obvious that the entity is a light, and that you can use the light component services on it.

[balloob]

Don't conflate entity names and entity IDs. Names can have international characters, entity IDs don't.

Also, an entity ID is composed of .<object_id>, so we can't remove the domain or else we don't know to what domain it belongs.

Going to lock this issue now as this is not the place to discuss these things.