MQTT settings for Boolean Format are changing API V3 Output

[tp1de]

I opened an ioBroker Forum Thread for asking other users to test the ems-esp adapter.
I got replies and recognized the following issue:

The MQTT Boolean Format settings are changing the rest api output. This is independent if MQTT is active or not.

When format 1/0 is chosen the API returns numeric values for boolean and enum fields.
All other formats return the respective text strings.

MQTT settings should not have an impact on API format and return values for enum / boolean should be always numbers.
The same for settings (changing values) !

[proddy]

MQTT boolean format should not affect the API, I agree, and should be fixed.

But we should define how to handle booleans in the API. It was in my original proposal for the V3 API but not implemented yet. It should be true/false I think, not 1 or 0 ?

[proddy]

also related, the API should not return the enumm or min/max if it's a boolean type

// http://ems-esp/api/boiler/heatingactivated

{
  "name": "heatingactivated",
  "fullname": "heating activated",
  "value": true,
  "type": "boolean",
  "min": 0,
  "max": 1,
  "enum": [
    false,
    true
  ],
  "unit": " ",
  "writeable": true
}

[tp1de]

I would recommend 1/0 for booleans. Graphs are easier with numeric values.
But anyhow I could adjust the adapter accordingly for booleans.

Enums are more important to be numeric !

[tp1de]

I use numeric values for booleans within the ioBroker adapter to easyly integrate them into flot graphs.

Important for me is while changing the State-Value in ioBroker the http post to ems-esp should accept numeric and/or boolean values.

[proddy]

Googling it the OpenAPI standard for booleans is true/false. Also in the JSON spec so I'll implement that way. If you want 1/0 then we'll need to build in a formatter setting as we did with MQTT

[tp1de]

If you want 1/0 then we'll need to build in a formatter setting as we did with MQTT

Not necessary. I already built this transformation (boolean -> number) into the ioBroker Adapter for http get.
But would you accept 1/0 for posts or do I need to adjust?

[tp1de]

By the way: km200 gateway returns numeric values (0/1) for boolean states. I think to be consistant with IP-gateways from Bosch-Group might be a strong argument as well.

[proddy]

I think the KM200 is just old tech. Bosch also use the OpenAPI standard for all their modern devices. The spec is on https://developer.bosch.com/web/bosch-thermotechnology-device-api/devices

[tp1de]

Will be interesting im IP-Gateways will be open with documented API. Until now Bosch was refusing to open specs. Hopefully they will do soon. I can live the booleans but just tell me if you will accept 1/0 for posts.

Since I integrate km200 and ems-esp with today's status I will transform in the ioBroker adapter to numerical values to be consistant unless changed by Bosch in the future.

[proddy]

I know Bosch are working on hard on this. I met their Chief Expert (Georg Grütter) this week who's driving their Innersource/Opensource initiative. https://www.bosch.com/research/know-how/open-and-inner-source/

For POSTs EMS-ESP will continue to accept all formats (1/0, true/false, yes/no, "1","0") so you're ok there.

[tp1de]

I think the KM200 is just old tech. Bosch also use the OpenAPI standard for all their modern devices. The spec is on https://developer.bosch.com/web/bosch-thermotechnology-device-api/devices

As far as I understood this Bosch API will be a (priced) cloud service. I hope that local LAN request to devices like km200 will not be blocked in the future.

I know Bosch are working on hard on this. I met their Chief Expert (Georg Grütter) this week who's driving their Innersource/Opensource initiative

Have you got any feedback if Bosch plans to open documentation on ems telegrams? Or what would this initiative mean for the ems-esp project?

For POSTs EMS-ESP will continue to accept all formats (1/0, true/false, yes/no, "1","0") so you're ok there.

Good to hear, so I do not need to adjust.

[tp1de]

I tested b8 version. There are some changes on boolean but they are inconsistant. Enum still unchanged.

In a single field with API the values true/false changed as discussed. (thermostat.hc1.summermode = true)
Reading a device (e.g. thermostat) the values as defined in mqqt setting still are there (thermostat.hc1.summermode = 1)

On enum values there are no changes. So numeric values only with mqtt settings 0/1

[proddy]

I missed the single values, I'll change that.

enum is how I designed and wrote it. It represents a list of valid options. In the code it's actually called options and I can't remember where the enum came in so I may rename it later to list or options or range?

This is how it looks now:

http://ems-esp/api/boiler/maintenance
{
  "name": "maintenance",
  "fullname": "maintenance scheduled",
  "value": "off",
  "type": "enum",
  "enum": [
    "off",
    "time",
    "date"
  ],
  "writeable": true
}

[proddy]

Reading a device (e.g. thermostat) the values as defined in mqqt setting still are there (thermostat.hc1.summermode = 1)

I can't reproduce for single values. I get for example:

http://ems-esp/api/boiler/wwcirc
{
  "name": "wwcirc",
  "fullname": "ww circulation active",
  "circuit": "ww",
  "value": false,
  "type": "boolean",
  "writeable": true
}

can you provide a screenshot or browser capture for summermode? I don't have an RC300 so can't test

[tp1de]

What ever the name is enum, lsit, options or range.
In km200 they are name "allowedValues" which represent a list of values always starting with 0 then 1,2,3 etc.
The http get return is a numeric value. That's what I need as well: Numeric values !

I just recognized that in ems-esp some of the "allowedValues" / "enums" start with 1 rather then 0.
(e.g. thermostat.building with 1,2,3). So I adjusted that in the Adapter.

The ioBroker Adapter has 2 loops:
1st reading all devices and then making single http gets to get the field attributes (initialize states)
Then every 15 secons reading "values" per field from the devices (one hhtp get per device returns all field values).

The field values I need as numeric values. Actually I get them by setting in mqqt 1/0 for boolean values.
How will this work after the adjustments?

[tp1de]

http://ems-esp/api/boiler/wwcirc I get the same result.
http://ems-esp/api/boiler returns 0 for wwcirc. Depends on mqtt settings

[tp1de]

just to add:
boolean values can be true or false but device and single field values should be the same
allowedValues / enums should be numeric (index to table of allowedValues)

[proddy]

The enums (or whatever we end up calling it) always return strings. This is how the data is interpreted and displayed to the end-user in the console, web and MQTT payloads. For most people using the KM200's internal numerical mapping will make no sense at all. Remember the EMS-ESP is not a replacement for the KM200. And it's not just for Bosch either.

For example, The thermostat's heating type outputs off, radiator, convector and floor.

You can see all the enums at

EMS-ESP32/src/locale_EN.h

Line 299 in e99c920

// thermostat lists

What I could do is include the index (starting at 0) in the API output along side the string value?

[MichaelDvP]

The bool format was, and in v2 it is, a global option placed in project-settings. On 23 Feb. proddy decided to show on web and console only on/off and moved the option to mqtt-settings. But all the time the api is also affected by the option.

IMO it is the best to move the option back to global settings and rename it to api/mqtt bool and enum format. It is not very common that people using api and mqtt with different formats. And the discussion here shows that there are different preferences for the best format.

• The naming enum was introduced in Improvements to the API #46 with the single value output and nobody complains, so it is not changed.
• The enums shows numbers for single value (emsdevice.cpp, line 683) and in info (line 873) if format 0/1 is selected.
• The bool format for single values is now removed, but ' info' still uses bool format (line 844) for all outputs, except console. tp1de uses the info to read the values and sigle values only for configuration.
• some enums does not have a known zero option. RC300 building is 1-3 for light, meduim, heavy, in RC35 the same settings are 0-2. Maybe RC300 have a off option, but no RC300 user have reported it. This confusing numbering is from the Bosch-telegram, not from emsesp. With the latest changes RC300 will show a empty option ["","light","medium","heavy"].

[tp1de]

@MichaelDvP This is a good proposal to use global option settings for output format.
I was happy with the 0/1 mqtt settings and the respective api response.
It was just a surprise that mqtt settings have an impact on api.

At the end enum values should correspond to km200 allowedValues numbers.
The text is anyhow different. But values correspond are giving an additional quality check that ems-esp is working fine.

I would just recommend that the global settings then should be valid for mqtt, api and webui. For enums webui should show value and text e.g. 1 (light), 2 (medium) .... for building. There is no empty option btw...

And try to define if post commands are independent of the settings and accept all formats or just the one set in global settings. In the moment the ioBroker adapter sends numeric values in post commands. This is working fine and for enums this is actually defined by min / max values. e.g. the thermostat.building has a min of 1 and max of 3.

[proddy]

remember the reason why I added the Boolean Format in the first place was because stupid Domoticz only accepted "ON" and "OFF" in MQTT. That could be a reason to keep API and MQTT boolean formats separate.

Also the webUI is supposed to be user-friendly. Show "Heating" as "on" is a lot more intuitive than showing it as "true". The rendering happens in the web code (and translated) and is not affected by any of the EMS-ESP's boolean settings.

[MichaelDvP]

remember the reason why I added the Boolean Format in the first place was because stupid Domoticz only accepted "ON" and "OFF" in MQTT.

Oh, i remembered the capital ON/OFF for OpenHAB in #713 and that the bool format was introduced before in #509, but i don't know the trigger for #509.
But nevertheless i stay with the opinion that someone who uses MQTT with OpenHAB or something else don't use api with different format simultaneously. Adding extra options for mqtt and api is possible, but useless.
And i agree with you that web/console should stay with fixed on/off and enum-texts.

[proddy]

ok, so changes are

• move the MQTT Boolean Format to the main settings page and use the setting also in the API
• enums stay as texts, and we're not adding an index field to the API response

[tp1de]

• enums stay as texts, and we're not adding an index field to the API response

yes for single fields but min/max needs to be specified.
no for device api response. Should return values, when global settings are 0/1 for boolean or maybe it is better having a seperate selector for enums in global settings. ..... and http post should accept values as well

[tp1de]

@proddy You have closed this issue. But as far as I can see the global settings for API formats are not yet implemented in v3.1.2b0.
And min/max info for enums is not available as well - e.g. for thermostat/building min should be 1 since enum number start with 1.

I have to admit that I lost the overview what is already implemented and what not in respect to API V3.

[proddy]

this issue fixed the "MQTT settings for Boolean Format are changing API V3 Output".

For your changes either create a new issue or add to #65

I don't plan to implement anything new in the API for a while since we're all arguing on what it should be. I'll eventually implement #65 and document it, when I decide how it should be.