Expose restart and max_current through mqtt

[mathieucarbou]

Using the RAPI MQTT topic to set the max_current_soft ($SC) does not correctly refresh the UI. It's best to expose it. Also, restart of the gateway was missing.

[mathieucarbou]

FYI I am currently working on a HA integration that I will share once finished and these missing stuff merged.

[jeremypoulter]

Have you seen https://github.com/firstof9/openevse? We would recommend using the HTTP API if you wish to do your own integration. The HTTP API is much more feature complete and is better suited for request->response interactions.

[KipK]

What is the use case of setting max_current_soft over mqtt? It is mostly a one time setup so I wonder. Le mer. 3 mai 2023, 13:39, Jeremy Poulter ***@***.***> a écrit :

…

Have you seen https://github.com/firstof9/openevse? We would recommend using the HTTP API if you wish to do your own integration. The HTTP API is much more feature complete and is better suited for request->responce interactions. — Reply to this email directly, view it on GitHub <#620 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADXKVKMYLESSPCXNA3GLHLXEI7YHANCNFSM6AAAAAAXUGTZ4A> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[mathieucarbou]

Have you seen https://github.com/firstof9/openevse? We would recommend using the HTTP API if you wish to do your own integration. The HTTP API is much more feature complete and is better suited for request->response interactions.

Hi,

I've searched for a while for a good integration and didn't find it! I will definitely try it.

Do you know if it is using Websocket or MQTT behind ? HTTP polling is highly inefficient and does not support pushs.

[mathieucarbou]

What is the use case of setting max_current_soft over mqtt? It is mostly a one time setup so I wonder.

I am on 4.2.2, and sometimes when I restart he box, the max current is set to 6A. I am not using any RAPI calls though. I didn't find a way yet to systematically reproduce though, but it happens after restart.

But I didn't implement that to fix this, I implemented that to have a parity with the rest api, which allows to change it and also restart the box.

Besides, I strongly prefer going through mqtt that http calls for responsiveness.

Also, setting the max current through the RAPI MQTT topic leads to the UI not correctly being refreshed.

And another point is that without this, it's not possible to get a component in HA to configure this value backed by mqtt.

[chris1howell]

HTTP is fine but ONLY works on a local network without port forwarding (not secure) or complicated VPNs. MQTT is much more useful as it can be used local and over the internet. HTTP will not be suitable for users with more than one location, commercial, or fleet setups MQTT or OCPP will be required. All the cloud providers support MQTT for IoT integrations. If OpenEVSE were to develop a smartphone app it would likely use MQTT not HTTP.

…

On Wed, May 3, 2023, 7:39 AM Jeremy Poulter ***@***.***> wrote: Have you seen https://github.com/firstof9/openevse? We would recommend using the HTTP API if you wish to do your own integration. The HTTP API is much more feature complete and is better suited for request->responce interactions. — Reply to this email directly, view it on GitHub <#620 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN5QH3C7SARTYRL6X5KV3TXEI7YHANCNFSM6AAAAAAXUGTZ4A> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[KipK]

What is the use case of setting max_current_soft over mqtt? It is mostly a one time setup so I wonder.

I am on 4.2.2, and sometimes when I restart he box, the max current is set to 6A. I am not using any RAPI calls though. I didn't find a way yet to systematically reproduce though, but it happens after restart.

But I didn't implement that to fix this, I implemented that to have a parity with the rest api, which allows to change it and also restart the box.

Besides, I strongly prefer going through mqtt that http calls for responsiveness.

Also, setting the max current through the RAPI MQTT topic leads to the UI not correctly being refreshed.

And another point is that without this, it's not possible to get a component in HA to configure this value backed by mqtt.

Could you check when your max_current caps at 6A the /monitoring/commands section on UI(v2) if it's not another service limiting it over a claim.

[jeremypoulter]

@mathieucarbou

Do you know if it is using Websocket or MQTT behind ? HTTP polling is highly inefficient and does not support pushs.

It uses the Websocket to get the status I believe.

@chris1howell

MQTT has its problems too, it is really designed to broadcast messages on a one-to-many basis, and you need a certain degree of hackery to do proper one-to-one API like calls.

[mathieucarbou]

Have you seen https://github.com/firstof9/openevse? We would recommend using the HTTP API if you wish to do your own integration. The HTTP API is much more feature complete and is better suited for request->response interactions.

Hi,

I've searched for a while for a good integration and didn't find it! I will definitely try it.

Do you know if it is using Websocket or MQTT behind ? HTTP polling is highly inefficient and does not support pushs.

So I've installed it, and it is actually quite limited compared to what I've done (and in only 1 yaml file).

• The vehicle sensors are not working

• No way to enable / disable Current Shaper
• No way to add an offset for the solar divert grid usage, to offset the injection on purpose to allow to go higher in charge current. I..e to allow a grid import. This is different than the ratio. I.e. I offset the grid export by -400W / -500W, to make openevse box think is has that available on top.
• No way to override but without starting a charge: when overriding there are 3 possibilities:
  • override: status=active, with some current settings
  • override: status=disabled, with some current settings
  • override: clear
• No way to control the max charge current dynamically (i.e. could be used to limit solar divert charge current, or cap the charge current when overriding). I.e. might be nice to allow an override during the day, but no more than 13A, and allow up to 16 or 32A during the nigh (costs less). The max limit could be changed dynamically based on peak hours for example.

[mathieucarbou]

MQTT has its problems too, it is really designed to broadcast messages on a one-to-many basis, and you need a certain degree of hackery to do proper one-to-one API like calls.

Yes totally agree. Though, IMO, these sorts of blocking calls should be avoided... Especially in HA which is largely event-based. I.e. in the integration I did, I don't need any blocking call. Components are answering updates from MQTT and updates are pushed through the */set topics, which is exactly a non blocking request-response, far better IMO than a blocking one ;-)

[mathieucarbou]

What is the use case of setting max_current_soft over mqtt? It is mostly a one time setup so I wonder.

I am on 4.2.2, and sometimes when I restart he box, the max current is set to 6A. I am not using any RAPI calls though. I didn't find a way yet to systematically reproduce though, but it happens after restart.
But I didn't implement that to fix this, I implemented that to have a parity with the rest api, which allows to change it and also restart the box.
Besides, I strongly prefer going through mqtt that http calls for responsiveness.
Also, setting the max current through the RAPI MQTT topic leads to the UI not correctly being refreshed.
And another point is that without this, it's not possible to get a component in HA to configure this value backed by mqtt.

Could you check when your max_current caps at 6A the /monitoring/commands section on UI(v2) if it's not another service limiting it over a claim.

@KipK : so I have the GUI v1, but I've checked the /claims endpoint and the 2 claims I have are the current shaper and solar divert. But I didn't pay attention to the settings of the current shaper, if for example when it happens its values are low for some reasons... I will check.

About the V2 GUI... I was waiting to use it since I saw it is still in development ? It is stable enough to switch to it ? If so, is there a quick wait to change the submodule hash that points to the V2 gui when being on the master branch of the gateway ? Just wondering what is the procedure. I used to to a git submodule update then a npm install

[KipK]

Yes we're migrating the UI for next release. You can already use it on current stable there's 2 builds one for each ui.

So that was what I was expecting you have a claim targeting max_current at 6A, probably Shaper ( you can check the /claims/target endpoint it will show u only the ones that has the priority )
I think you don't need max_current_soft, thus one is only set one time and should not move. You need to use a claim or override to set your charge_current ( you shouldn't use the max_current if you have shaper activated as it will have priority on this one )

[KipK]

I'm not sure about exposing config settings over MQTT. HTTP API is a better candidate for that parts of properties that don't need to be updated often ( they're written to flash so if you change max_current_soft with some automations it will hit the flash each time. )

[mathieucarbou]

Ok! I will verify all that.

[mathieucarbou]

I'm not sure about exposing config settings over MQTT. HTTP API is a better candidate for that parts of properties that don't need to be updated often ( they're written to flash so if you change max_current_soft with some automations it will hit the flash each time. )

A lot of configs are already exposed through MQTT and some in RW mode (i..e shaper, divert mode, etc). Which is nice by the way because when we integrate some components in HA backed by MQTT, like a slider, switch, etc they need to be able to push back to an MQTT the resulting user interaction. And still be updated by mqtt topic changes.

max_current has its value in mqtt. This is an information that is useful to be displayed / tracked. Like the maximum power for shaper.

It is also far easier to bind some HA components with mqtt (each HA component has an MQTT flavor which can answer to user interaction).

As a ref, here is my integration: https://gist.github.com/mathieucarbou/92a3d5e0dc38d6b68aa1bdaf153a80c5

I really hope this PR (or some adjustments of it) will be merged because I can't see any reason why we wouldn't want to fill the gap in the mqtt layer, which is already nice and provides an easy and fast way to integrate OpenEVSE.

A nicer way would be to add some MQTT topics in /homeassistant topic for automatic discovery. I know some project doing that: HA is then able to automatically create devices with their list of sensors based on what's discovered in MQTT.

Example:

[KipK]

shaper & divert mode on mqtt are volatile like their related http endpoint.
They are not the same value from the config, they are temporary switch and doesn't survive reboot.

max_current has its value in mqtt. This is an information that is useful to be displayed / tracked. Like the maximum power for shaper.

I get your point on read accessing the value on mqtt ( the current charge current slider in UI2 use max_current_soft as the max value of the slider , so for ex to mimic this with mqtt, it is needed )
edit: thinking about that, we should expose the whole /config endpoint to a /config topic

I think there's confusion between max_current_soft and max_current.
max_current_soft ( store in wifi module ) is software equivalent to max_current_hard ( stored in openevse module ) but can be set lower as a setup. It's not intended to be modulated.
max_current should then be used if needed over a claim or override
rule is charge_current <= max_current <= max_current_soft <= max_current_hard
max_current_soft is here to prevent max_current going over it.

The risk of adding settable config over mqtt is misuage of users who will use this value in their automation scripts instead of setting it over claim as it will look simpler to do this way.

What do you think ?

edit: I like your HASS integration based on mqtt btw

A nicer way would be to add some MQTT topics in /homeassistant topic for automatic discovery. I know some project doing that: HA is then able to automatically create devices with their list of sensors based on what's discovered in MQTT.

There was a ticket for this ( #359 ), not sure it was intended to close it, I've just reopened it.

[KipK]

to build with the new gui you need to export 2 env var:

export GUI_NAME=../openevse-gui-v2 ( this point to the v2 gui dir )
export PLATFORMIO_BUILD_FLAGS='-D DISABLE_WIFI_PORTAL -D WEB_SERVER_ROOT_PAGE_INDEX'
it won't be needed after migration

the gui needs to be built first before compiling the fw ( npm run build )

the git is here: https://github.com/OpenEVSE/openevse-gui-v2

[mathieucarbou]

shaper & divert mode on mqtt are volatile like their related http endpoint. They are not the same value from the config, they are temporary switch and doesn't survive reboot.

Oh, right! I did't pay attention to that! That's a bummer because its configuration-related though (from a UX point of view). I understand the technical reason behind, but still, as a user who wants to integrate in HA, I would expect the switch to survive a reboot since this is a setting and as part of the many sensors exposed, these are those susceptible to be part of some logic, automations or else in HA. So if their value is reverted back unexpectedly, there is no state anymore, so it becomes more useless because cannot be counted on :-(

FYI I tested and after a reboot, solar divert mode is kept (normal / eco), but current shaper always restarts activated yes. So the issue is only for current shaper.

max_current has its value in mqtt. This is an information that is useful to be displayed / tracked. Like the maximum power for shaper.

I get your point on read accessing the value on mqtt ( the current charge current slider in UI2 use max_current_soft as the max value of the slider , so for ex to mimic this with mqtt, it is needed ) edit: thinking about that, we should expose the whole /config endpoint to a /config topic

/config and /config/set with a partial payload of only the updated fields ? if so, that would be nice yes. A real endpoint to read and update the config remotely through mqtt.

I think there's confusion between max_current_soft and max_current. max_current_soft ( store in wifi module ) is software equivalent to max_current_hard ( stored in openevse module ) but can be set lower as a setup. It's not intended to be modulated. max_current should then be used if needed over a claim or override rule is charge_current <= max_current <= max_current_soft <= max_current_hard max_current_soft is here to prevent max_current going over it.

That's how I understood it too. But like I said, after a reboot, it happened several times that the Max Current configured on the website has changed to a value below 16A in the drop-down. So I am not sure that this is a one-time setup and fixed and persisted and kept read-only. Otherwise the value would never change after a reboot.

I am not able to reproduce as of now, but seeing that, that's one of the reason I have exposed this value, because I needed a way from HA to control it and set it to a correct value.

Peak and off-peak can be handled by an override. I..e use solar divert during the day, force a charge but not more than 13A during the day if needed, but during the night, allow up to 16A. All that can be handled by an override claim.

I really played a lot with the settings while doing my HA integration so it is also possible that I bumped into a corner case that won't happen normally. I will monitor if I see that happening again,

Nonetheless, still, this is a config and my argument remains that any config should be remotely accessible (read and write), as this is also why a home automation system is made for... When not at home, I don't want to have to go to the OpenEVSE website to click on a configuration when everything that is susceptible to be changed should be integrated in HA.

The risk of adding settable config over mqtt is misuage of users who will use this value in their automation scripts instead of setting it over claim as it will look simpler to do this way.

What do you think ?

I agree ;-)
And any IT system WILL be used at any point in the worst possible way and no one can prevent that! Even if there is a doc.

I don't think you raise a child by preventing him to fall by being always behind him. Get him a bike, he will fall, try again, won't fall after...

So the idea is not to prevent dumb users to do dumb things. The idea is to allow better integrations to be made and empower users with the ability to achieve what they need, providing basic rules are respected. Yes the claim system has to be used to set maximum current temporarily. But also, config should be monitored and configurable remotely ;-) I completely understand the distinction between claims/override and settings, which max_current_soft is (and you can confirm that if you look at my HA integration code).

BUT still, this is a config, associated to a value that is IMO important to track, monitor and update accordingly as needed - but as a config, like activating or not OCPP, Current shaper, etc, setting the Level (2 or 1), etc. All these settings should be accessible and configurable remotely IMO :-)

[mathieucarbou]

npm run build

A git submodule update --init was also required ;-)

I've built and installed the gui V2 (master) with the gateway (master) plus my changes. That's awesome!

[KipK]

npm run build

A git submodule update --init was also required ;-)

I've built and installed the gui V2 (master) with the gateway (master) plus my changes. That's awesome!

Yes there's a submodule for the Timezone ( posix-tz-db ), I forgot it on the readme.

[KipK]

An important point on why I'm a little wavered to expose writable config on mqtt is I'd like to add rights management to credentials ( i.e user vs admin ) and make the config part/endpoints only accessible to admin

Integrating settable config settings over the same mqtt account used by automations ( considered as users ) would deserve this security ain't it ?

[mathieucarbou]

An important point on why I'm a little wavered to expose writable config on mqtt is I'd like to add rights management to credentials ( i.e user vs admin ) and make the config part/endpoints only accessible to admin

Integrating settable config settings over the same mqtt account used by automations ( considered as users ) would deserve this security ain't it ?

There are ways to do that.
If you have user management in place, you can generate for the admin user a private (and resettable) MQTT API keys, and each payload sent on topics requiring admin rights should have this api key.

openevse/config/set would take a json:

{"key": "api_key", "max_current_soft": 16}

The idea is that this key is only visible in the website, should be copied by the integration and put in the Home Assistant secret file or config or whatever so that Home Assistant can use this key when sending payloads.

[KipK]

I guess most users don't even know how to setup ACL in topics with mosquitto but we could add some doc then.

So then we need a PR to expose the whole /config /config/set

@jeremypoulter what do you think on this ?

[jeremypoulter]

Yeah, that would work.

[KipK]

after a little chat with Mathieu, I suggest to expose instead "max_current" = getMaxCurrent() in /status + main mqtt topic.
This has more sense in /status to get live limitation ( either "max_current_soft" or from a "max_current" claim then )

This would simplify a bit UI too, currently it checks claims first then use max_current_soft for the main page charge_current slider max value.

And I was almost sure it was already exposed there but seems it is not and now remember why It was done this way on ui2 :)

[mathieucarbou]

PR udpated ;-)

[mathieucarbou]

@KipK : I added a third commit to fix a MQTT update bug: the divert_active flag, which allows to know whether the solar divert is authorising a charge or not, is not sent after a restart of the gateway, which causes the HA sensor to become unavailable.

With these 3 commits, my integration works perfectly and I am pretty happy with it.

Box online

Box offline

• Current Shaper

  • Enable / Disable
  • Max Current shows the maximum charge current allowed

• Solar Divert

  • Enable / Disable
  • Allow Charge shows when Solar Divert is allowing a charge, and with which Charge Current
  • Offset decreases or increases the exported power seen by the module. A use case of that is to offste by -2A for example to allow OpenEVSE to go higher in charge current. Useful if we want the charge power to go above the exported power. The opposite is also possible, for examle +2A would reserve ~450W of grid export

• Energy meters

• Charge Session

• Charge Control

  • Set Manuel or Auto mode
  • Force Charge with a selected current
  • Force Pause
  • Charge Current shows the current amperage used. When the slider is moved, we automatically switch in manual mode
  • Max current shows the computed max current depending on current shaper or overrides/claims

• Gateway

  • Restart the gateway
  • Check if it is running or offline

• Vehicle

  • Plugged or not
  • External vehicule infos