Configuring Zigbee2MQTT for use with Adaptive Lighting

[th3w1zard1]

Last updated: October 2024

This guide serves to configure Zigbee2MQTT's configuration.yaml for use with Adaptive Lighting.
This guide is meant for a supervised HA install with the use of the Zigbee2MQTT addon. I cannot promise it will be relevant to users not using the HA addon.

See the bottom of this post for a quick copy paste of all required/recommended options.

Please note the device_options in the configurations below if defined in configuration.yaml will set these values for all devices, which you may not want. Everything in device_options are correct config options you can set in devices.yaml to be used for specific devices.

Required options for Home Assistant Addon specifically:

Note: some of these options are already default.

homeassistant:
  discovery_topic: 'homeassistant'
  status_topic: 'homeassistant/status'
mqtt:
  force_disable_retain: false
advanced:
  cache_state: true
  legacy_api: false
device_options:
  disabled: false # default
  homeassistant: true

Required options for Adaptive Lighting specifically

homeassistant:
mqtt:
advanced:
  cache_state_persistent: true
  cache_state_send_on_startup: false
  report: true # default
device_options:
  optimistic: true
  filtered_cache: ["brightness", "rgb_color", "hs_color", "xy_color", "color_temp_kelvin", "color_temp", "color", "color_mode"]

You also must configure your light's power_on_state to match the last known state.

Required Adaptive Lighting options: Why?

cache_state_persistent: true and cache_state_send_on_startup: false

From Zigbee2MQTT docs:

Optional: persist cached state, only used when cache_state: true (default: true)
Optional: send cached state on startup, only used when cache_state_persistent: true (default: true)

We want Zigbee2MQTT to store all of the last known states for restarts etc, but we do not want these sent to Home Assistant on startup. In future updates, Adaptive Lighting will assume whenever brightness/color info is provided in the service data, the light should be set to manual control.

report: true (default)

From Zigbee2MQTT docs:

Optional: Enables report feature (default true), this feature is DEPRECATED since reporting is now setup by default when binding devices. Docs can still be found here: https://github.com/sjorge/zigbee2mqtt.io/blob/4d8f358dade21b2a382fd18aec182b9bda633db1/docs/information/report.md

Adaptive Lighting needs to be able to listen to state change events in Home Assistant in order to properly detect manually controlled events.

filtered_cache

From Zigbee2MQTT docs:

This will ignore the unchanged attributes whenever a state update is pushed to HASS. Needed in future updates when Adaptive Lighting will assume brightness/color info in light.turn_on are manually controlled events.

Additional Options you may need:

retrieve_state: true

From Zigbee2MQTT docs:

(DEPRECATED) Retrieves the state after setting it. Should only be enabled when the reporting feature does not work for this device.

optimistic: true

From Zigbee2MQTT docs:

Publish optimistic state after set, e.g. when a brightness change command succeeds Zigbee2MQTT assumes the brightness of the device changed and will publish this (default true)

For me, this option is broken on my lights which I've reported here. When it's disabled, state changes are never reported to Home Assistant. This needs to be true if you're having that issue.

Both Required and Recommended Options (copy/paste version)

homeassistant:
  legacy_entity_attributes: true
mqtt:
  force_disable_retain: false
advanced:
  cache_state: true
  cache_state_persistent: true
  cache_state_send_on_startup: false
  report: true
  legacy_api: false
device_options:
  disabled: false
  homeassistant: true
  optimistic: true
  filtered_cache: ["brightness", "rgb_color", "hs_color", "xy_color", "color_temp_kelvin", "color_temp", "color", "color_mode"]

Resources:
Configuration | Zigbee2MQTT
Devices and Groups | Zigbee2MQTT
FAQ | Zigbee2MQTT

Further Configuration

Please see this top tier reply further down in this thread.

[fbloemhof]

Nice! I implemented this yesterday, did not have all these settings in place yet.

The report: true setting causes that a restart of Z2M is very slow, it takes ages before the webinterface is available because it seems very busy (re)configuring the reporting for all devices?

I had some weird behavior with unpredictable state changes etc., same as described in #829. Then I found this topic.

[djurny]

As a tip, for reporting, you can set these options per device as well, to make sure not all devices start reporting. Same goes for most device_options, they can be set per device.
Groetjes,

[djcrawleravp]

Is this still working?

[basnijholt]

I'm using zigbee2mqtt without any problems.

[djcrawleravp]

I'm stuck in ZTM version 1.33.1 because, when automating my lights using Node-RED, this issue occurs:

Upon detecting absense, the lights are supposed to enter sleep mode for 30 seconds and then turn off. However,

when presence is detected again, the lights turn on in sleep mode and then switch back to adaptive mode.

In later versions, the lights turn on in sleep mode but fail to transition to adaptive mode (sometimes they succeed after a couple of seconds, sometimes they just stay in sleep mode).

And that issue only occurr on versions greater than 1.33.1

THis is how my config look like:

mqtt:
  base_topic: zigbee2mqtt
  keepalive: 60
  reject_unauthorized: true
  force_disable_retain: false
  version: 4
advanced:
  transmit_power: 5
  homeassistant_legacy_entity_attributes: false
  legacy_api: false
  legacy_availability_payload: false
  log_syslog:
    app_name: Zigbee2MQTT
    eol: /n
    host: localhost
    localhost: localhost
    path: /dev/log
    pid: process.pid
    port: 514
    protocol: udp4
    type: '5424'
  channel: 25
  log_level: error
  cache_state: true
  cache_state_persistent: true
  cache_state_send_on_startup: false
availability: true
device_options:
  disabled: false
  homeassistant: true
  optimistic: true
  filtered_cache: ["brightness", "rgb_color", "hs_color", "xy_color", "color_temp_kelvin", "color_temp", "color", "color_mode"]
homeassistant:
  legacy_entity_attributes: true

[R3Cardoso]

This options don't work any more.
Got my zigbee2mqtt on version 1.40.1-1 and lights only adapt after turning on, not before.

[djcrawleravp]

I just solved my problem, It wasn't Adaptive Lightning.

You can check this issue

[djurny]

Hi there,
My whole setup is based on controlling my lighting with Adaptive Lighting, with zigbee2mqtt being the main light source handler. For me it was not just about setting up zigbee2mqtt with AL, but the lights controlled by zigbee2mqtt required some quirks to make sure that it all worked nicely together.

Make sure the AL interval is big enough to make steps that your lights can handle.

I had my interval set to 30 seconds, but got a lot of spurious manual_control events from Ikea Zigbee bulbs. After some digging into why this is happening it seemed like AL was trying to set a color temperature that did not translate to a mired value difference for my Ikea bulbs. This led AL to believe the lights were manually controlled and AL stopped adapting the lights.
After reading the documentation and increasing the interval to 90 seconds (which leads to a bigger change in color_temp in mireds), the spurious manual_control events have not popped up anymore.

Make sure to stop any ongoing transition if you have Ikea bulbs.

Be sure to read the documentation on how to setup AL for Ikea bulbs (..yeah yeah I know).
In my setup, I use some generic script (per AL entity) to turn lights on/off and dim/undim them. Some other HASS automations deal with dimming AL controlled bulbs when the area is vacant for some time (based on occupancy). When occupancy is detected, AL controlled bulbs are undimmed.
Every now and then, I would see that the Ikea bulbs would not respond to an 'off' or 'undim' command. Based on some HAFO (hacking around and finding out), it seemed that sending a color_temp slightly off from the current (target) value would stop transitions for E14 Ikea bulbs. This seemed to work for a while, but using the guide actually works best.
Make sure that if you turn AL controlled bulbs off, or undim them, to stop any ongoing transactions to make sure your lights react.

Have proper color transitions for your lights.

After starting out with WiZ connected lights, I was used to lights transitioning from off to on with the color temperature already set properly.
This changed with the Zigbee bulbs from both Ikea TRÅDFRI and Philips Hue. The starting color temperature seemed to be whatever it was set to when the bulb was turned off. This is not a biggie if you turn off your bulbs in the late evening and flip them back on in the early morning (both have low color_temps), but this started to annoy me.
The solution from @sjorge worked well (see ref), for all the Zigbee bulbs I had in my setup. Enable the execute_when_off option for color_options for your bulbs and send the color_temp you want before you turn on the light (#1022 (comment)). In my configuration I decided to enable execute_when_off after a bulb is turned off.

zigbee2mqtt's transition and AL's initial_transition and adapt_delay.

I've had some odd off-to-on transition effects on some of my bulbs, where it would transition slowly at first then quicklly to the 'setpoint'.
This was resolved by making sure to either use an initial_transition in AL that is equal to the transition in zigbee2mqtt, or to use an AL adapt_delay that is slightly longer then z2m's transition.
What also seemed to be the case, is that disabling manual_control will result in a change of light settings, with - I think - the transition from the AL switch configuration, whilst interrupting any transition that is taking place.

With the caveats above, my AL works well with zigbee2mqtt and both Ikea and Hue bulbs.

Hope this all helps someone out there.
Groetjes,

Ref Koenkk/zigbee2mqtt#13156 (reply in thread) (@sjorge)
Ref #1022 (comment)

[R3Cardoso]

Thanks for the help, but I wasn't able to resolve anything.
I'm not sure where I need to put the code? should be on all automations?
//snip - repeat: for_each: '{{ state_attr(entity_light, "entity_id") }}' sequence: - service: mqtt.publish data_template: topic: 'zigbee2mqtt/{{state_attr(repeat.item, "friendly_name")}}/set/color_options' payload: | { "execute_if_off": "true" } - service: mqtt.publish data_template: topic: 'zigbee2mqtt/{{state_attr(repeat.item, "friendly_name")}}/set' payload: | { "color_temp": "{{turn_on_quirk_color_temp_mired_current}}", "transition": "0" } //snip

About transition I got 0 on zigbee2mqtt and adapt_delay, on initial_transition got 1 because it helps disguise the change.

[djurny]

Hi there,
I think you are referring to your issue Zigbee2mqtt to fast for adapt?

To 'prime' the color_temp of your lights, which means set them to the wanted color_temp before you turn them on, you would have to make a script in homeassistant. You can read up here on what that entails.

As an basic example, your script would look as follows (this is a dry example, have not tested it whatsoever):

script:
  some_script_name_that_makes_sense:
    variables:
      ## change these to suit your situation
      name_light: 'kitchen'
      name_adaptive_lighting: 'kitchen'
      transition: '1'
      ## 
      entity_id_light: 'light.{{name_adaptive_lighting}}'
      entity_id_adaptive_lighting: 'adaptive_lighting.{{name_adaptive_lighting}}'
      mqtt_topic: 'zigbee2mqtt/{{name_light}}/set'
    sequence:
    - action: mqtt.publish
      data_template:
        payload: |-
            {
              "color_temp": "{{state_attr(entity_id_adaptive_lighting, 'color_temp_mired')}}"
            }
        topic: '{{mqtt_topic}}'
    - delay:
        hours: 0
        minutes: 0
        seconds: 0
        milliseconds: 250
    - light.turn_on
      data:
        transition: '{{transition}}'
      target:
        entity_id: '{{entity_id_light}}'

Note that for the script to work:

• your light's name in home assistant should be the same as the friendly_name in zigbee2mqtt.
• the adaptive lighting entity must have the same name as your light's name.

There is no simpler way to do this a.f.a.i.k.

Hope that helps,
Groetjes,

[R3Cardoso]

Thanks for the explanation, but I don't have the same light name on zigbee2mqtt and HA(this was easy fix) but for AL I got only some entity to control several lights. For exemple got a entity call bedrooms that control all the lights on bed rooms.

And as far I also understand, I would need to have a script to every single light I got. :/ a lot of work.

I will wait for zigbee2mqtt are able to do light Philips Hue box was able to do it. If philips hue box do it, zigbee2mqtt sooner or later will be able too.

[djurny]

To keep this topic to the point, I have posted a comment in your issue here: #996 (comment).