adaptive_lighting.apply does not set color temperature when "turn_on_lights" is set to "false"

[djurny]

Please confirm that you have completed the following steps:

• I have updated to the latest Adaptive Lighting version available in HACS.
• I have reviewed the Troubleshooting Section in the README.
• (If using Zigbee2MQTT) I have read the Zigbee2MQTT troubleshooting guide in the README.
• I have checked the V2 Roadmap and open issues to ensure my issue isn't a duplicate.

Required information for bug reports:

Please include the following information in your issue.

Issues missing this information may not be addressed.

1. Debug logs captured while the issue occurred. See here for instructions on enabling debug logging:

2024-07-05 16:47:49.966 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: '_update_attrs_and_maybe_adapt_lights' called with context.id='01J22MR12E:al:JBQW:ntrv:WU' lights: 'None', transition: '4.0', force: 'False'
2024-07-05 16:47:49.978 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: filtered_lights: '[]'
2024-07-05 16:48:08.549 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Called 'adaptive_lighting.apply' service with '{'entity_id': ['switch.adaptive_lighting_adapt_color_hallway'], 'turn_on_lights': False, 'adapt_brightness': False, 'adapt_color': True, 'lights': [], 'prefer_rgb_color': False, 'transition': 2.0}'
2024-07-05 16:48:20.468 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: '_update_attrs_and_maybe_adapt_lights' called with context.id='01J22MRYVM:al:JBQW:ntrv:WV' lights: 'None', transition: '4.0', force: 'False'
2024-07-05 16:48:20.490 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: filtered_lights: '[]'
2024-07-05 16:48:21.556 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Called 'adaptive_lighting.apply' service with '{'entity_id': ['switch.adaptive_lighting_adapt_color_hallway'], 'turn_on_lights': True, 'adapt_brightness': True, 'adapt_color': False, 'lights': [], 'prefer_rgb_color': False, 'transition': 2.0}'
2024-07-05 16:48:21.563 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: execute_cancellable_adaptation_calls with data: AdaptationData(entity_id=light.bulbs_ikea_e26_1100lm_no_11_12, context_id=01J22MRZXN:al:JBQW:srvc:WW, sleep_time=2.001, force=True, max_length=1, which=brightness, initial_sleep=False)
2024-07-05 16:48:21.564 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: Scheduling 'light.turn_on' with the following 'service_data': {'entity_id': 'light.bulbs_ikea_e26_1100lm_no_11_12', 'brightness': 255, 'transition': 2.0} with context.id='01J22MRZXN:al:JBQW:srvc:WW'
2024-07-05 16:48:21.932 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: '_update_attrs_and_maybe_adapt_lights' called with context.id='01J22MS09B:al:JBQW:lght:WX' lights: '['light.bulbs_ikea_e26_1100lm_no_11_12']', transition: '2.0', force: 'True'
2024-07-05 16:48:21.946 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: filtered_lights: '['light.bulbs_ikea_e26_1100lm_no_11_12']'
2024-07-05 16:48:21.947 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: Calling _adapt_light from _update_attrs_and_maybe_adapt_lights: 'light.bulbs_ikea_e26_1100lm_no_11_12' with transition 2.0 and context.id=01J22MS09B:al:JBQW:lght:WX
2024-07-05 16:48:21.954 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: Setting color_temp of light light.bulbs_ikea_e26_1100lm_no_11_12
2024-07-05 16:48:21.955 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: execute_cancellable_adaptation_calls with data: AdaptationData(entity_id=light.bulbs_ikea_e26_1100lm_no_11_12, context_id=01J22MS09B:al:JBQW:lght:WX, sleep_time=1.001, force=True, max_length=2, which=both, initial_sleep=False)
2024-07-05 16:48:22.009 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: Scheduling 'light.turn_on' with the following 'service_data': {'entity_id': 'light.bulbs_ikea_e26_1100lm_no_11_12', 'brightness': 255, 'transition': 1.0} with context.id='01J22MS09B:al:JBQW:lght:WX'
2024-07-05 16:48:23.040 DEBUG (MainThread) [custom_components.adaptive_lighting.switch] Hallway: Scheduling 'light.turn_on' with the following 'service_data': {'entity_id': 'light.bulbs_ikea_e26_1100lm_no_11_12', 'color_temp_kelvin': 3840, 'transition': 1.0} with context.id='01J22MS09B:al:JBQW:lght:WX'

2. Your Adaptive Lighting configuration:

- name: Hallway
  lights:
    - light.hallway
  interval:                     30
  transition:                   4
  initial_transition:           2
  min_brightness:               40
  max_brightness:               100
  min_color_temp:               2700
  max_color_temp:               4000
  prefer_rgb_color:             false
  sleep_brightness:             1
  sleep_rgb_or_color_temp:      color_temp
  sleep_color_temp:             2000
  #sleep_rgb_color:
  sleep_transition:             30
  transition_until_sleep:       true
  #sunrise_time:
  min_sunrise_time:             07:00:00
  #max_sunrise_time:
  sunrise_offset:               3600
  #sunset_time:
  #min_sunset_time:
  #max_sunset_time:
  sunset_offset:                3600
  brightness_mode:              tanh
  brightness_mode_time_dark:    7200
  brightness_mode_time_light:   0
  take_over_control:            True
  detect_non_ha_changes:        True
  autoreset_control_seconds:    0
  only_once:                    false
  adapt_only_on_bare_turn_on:   false
  separate_turn_on_commands:    True
  send_split_delay:             1
  adapt_delay:                  0
  skip_redundant_commands:      false
  intercept:                    False
  multi_light_intercept:        False
  include_config_in_attributes: True

3. (If using Zigbee2MQTT), provide your configuration files (remove all personal information before posting):

Zigbee2MQTT:

  '0x*':
    friendly_name: Bulb IKEA E26 1100lm No.11
    description: TRADFRI LED bulb E26/27 1100/1055/1160 lumen, dimmable, white spectrum, opal white
    homeassistant:
      name: Bulb IKEA E26 1100lm No.11
    transition: 1
    disabled: false
...
  '11':
    friendly_name: Hallway ceiling section No.2
    homeassistant:
      name: Bulbs IKEA E26 1100lm No.11+12
    devices:
      - 0x*/1
      - 0x*/1
    transition: 1

Homeassistant:

- platform:   group
  name:       Hallway
  entities:
    - light.bulbs_ikea_e26_1100lm_no_11_12

4. Describe the bug and how to reproduce it:

In short: it seems that adaptive_lighting.apply with turn_on_lights = false does not actually send any commands to (in my case) Zigbee2MQTT in case the bulb is OFF.
Some more elaboration:
I would like to set the color temperature of (Ikea) bulbs that are off, before turning the bulb on. Normal case is that the Ikea bulb will adjust brightness before color temperature. A workaround I wanted to implement is to set the color temperature first (when bulb is off), before adaptive_lighting.apply'ing with adapt_brightness = true + turn_on_lights = true to turn on the bulb(s).
My attempt was to do a adaptive_lighting.apply with adapt_color = true + adapt_brightness = false + turn_on_lights = false, but this does not seem to change the color temperature of the bulb. The Z2M logging also does not show that there is a message being sent to the bulb when the service is called with those attributes.

6. Steps to reproduce the behavior:

Observed behavior/effect:
Use Z2M frontend to change color temperature to warmest.
Use Z2M frontend or HASS to turn off bulb.
Call adaptive_lighting.apply as follows to change color temperature while bulb is OFF:

Call adaptive_lighting.apply as follows to turn on the bulb w/out color temperature change

Bulb does not change color temperature back to what adaptive lighting should have set it to.
Also, there is no Z2M activity observed for the first adaptive_lighting.apply that would set color temperature while bulb is off, only for the second adaptive_lighting.apply messages to the bulb are observed.
Expected behavior/effect:
Zigbee2MQTT has exposed "Color options - Execute if off". This option allows to set the color temperature to the bulb in case it's off. Using the Z2M frontend, this seems to be exactly what the option is enabling:

• Set the brightness & color temp to dim & warm
• Turn the bulb off
• Set the color temp to the coolest
• Turn the bulb on
  The Z2M logging will show

info 2024-07-05 16:07:57MQTT publish: topic 'zigbee2mqtt/Bulb IKEA E26 1100lm No.11', payload '{"brightness":227,"color_mode":"color_temp","color_options":{"execute_if_off":true},"color_temp":255,"last_seen":"2024-07-05T16:07:57-07:00","level_config":{"on_level":"previous"},"linkquality":126,"power_on_behavior":null,"state":"OFF","update":{"installed_version":65554,"latest_version":65554,"state":"idle"},"update_available":null}'

Note that "state" is set to "OFF" in the Zigbee message here and when the bulb is turned on there is no color temperature transition occurring.

[djurny]

Workaround based on this comment: Add an mqtt.publish message to enable execute_if_off and mqtt.publish a message to set the color_temp wanted before the light is turned on.

//snip
        - repeat:
            for_each: '{{ state_attr(entity_light, "entity_id") }}'
            sequence:
            - service: mqtt.publish
              data_template:
                topic: 'zigbee2mqtt/{{state_attr(repeat.item, "friendly_name")}}/set/color_options'
                payload: |
                  {
                    "execute_if_off": "true"
                  }
            - service: mqtt.publish
              data_template:
                topic: 'zigbee2mqtt/{{state_attr(repeat.item, "friendly_name")}}/set'
                payload: |
                  {
                    "color_temp": "{{turn_on_quirk_color_temp_mired_current}}",
                    "transition": "0"
                  }
//snip

This workaround will only work when using Zigbee lights + zigbee2mqtt. In my case, the Philips Hue and Ikea bulbs luckily support the execute_if_off feature for setting the color temperature.

Related issues:

• HUE suppress color transition from off state Koenkk/zigbee2mqtt#13156 (reply in thread)
• Setting color temperature when bulb is off does not work home-assistant/core#121347 (comment)