Grid Import/Export (aka net consumption/production) sensors missing from Enphase Whole Home Backup setup

[joergbattermann]

The problem

As of core-2024.3.0 there's sensors for grid import/export (net production and net consumption) sensors for Enphase systems, however for ours, these sensors do not show up (not even as disabled).

Our setup in particular is a 'Whole Home Backup' one consisting of an Enphase System Controller 3, Combiner 5, and 5P batteries. The microinverters for our solar panels are IQ8As.

Enphase Enlighten and also the local Enphase Envoy interface show the net power (so grid import/export wattage) numbers, see i.e.:

I've attached the HA diagnostic file (with serial numbers XXXXed out), for the correct CT wiring diagram as per Combiner 5 documentation see page 45 in their quick installation guide.

Initial discussion re: these sensors not appearing started incorrectly in home-assistant/home-assistant.io#31426 / see the lower 5-6 comments.

What version of Home Assistant Core has the issue?

core-2024.3.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

enphase_envoy

Link to integration documentation on our website

https://www.home-assistant.io/integrations/enphase_envoy

Diagnostics information

config_entry-enphase_envoy-4b3c49f0662e447bc88cf868faa8900c.json

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant]

Hey there @bdraco, @cgarwood, @dgomes, @joostlek, @catsmanac, mind taking a look at this issue as it has been labeled with an integration (enphase_envoy) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of enphase_envoy can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign enphase_envoy Removes the current integration label and assignees on the issue, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

enphase_envoy documentation
enphase_envoy source
_{^{(message by IssueLinks)}}

[catsmanac]

Thanks for opening this as separate item @joergbattermann.

As far as I can see in the Enphase documentation it seems that the combiner 5 comes with 3 CT's

EDIT: Thanks for sharing the diagnostics file

[joergbattermann]

I had added the diagnostic .json already above @catsmanac .. see the linked file under 'Diagnostic information' / this link to it & I think you mentioned to look for this section in the other issue:

If there's anything you need from the Local Envoy side of things, please also let me know.

[gyordanov]

i seem to be in the same situation as @joergbattermann, the system setup seems identical to mine ...

not sure if this helps, but i did add the Stream mqtt from Enphase Envoy (https://github.com/vk2him/Enphase-Envoy-mqtt-json) addon and with this configuration i can read all the CTs

mqtt:
  sensor:
    - name: envoy mqtt pv production
      state_topic: "envoy/json"
      value_template: '{{ value_json[0]["activePower"] | round(0) | int(0)}}'
      unique_id: envoy_mqtt_pv_production
      qos: 0
      unit_of_measurement: "W"
      state_class: measurement
      device_class: power
    - name: envoy mqtt net consumption
      state_topic: "envoy/json"
      value_template: '{{ value_json[1]["activePower"] | round(0) | int(0)}}'
      unique_id: envoy_mqtt_net_consumption
      qos: 0
      unit_of_measurement: "W"
      state_class: measurement
      device_class: power
    - name: envoy mqtt batteries
      state_topic: "envoy/json"
      value_template: '{{ value_json[2]["activePower"] | round(0) | int(0)}}'
      unique_id: envoy_mqtt_batteries
      qos: 0
      unit_of_measurement: "W"
      state_class: measurement
      device_class: power

with this one to split the grid power (it can be positive or negative) into 2 differnet sensors

    - name: envoy mqtt grid import
      state_topic: "envoy/json"
      value_template: '{% set val = value_json[1]["activePower"] | round(0) | int(0)%} {% if val > 0 %} {{val}}  {% else %} 0 {% endif %}'
      unique_id: envoy_mqtt_grid_import
      qos: 0
      unit_of_measurement: "W"
      state_class: measurement
      device_class: power
    - name: envoy mqtt grid export
      state_topic: "envoy/json"
      value_template: '{% set val = value_json[1]["activePower"] | round(0) | int(0)%} {% if val < 0 %} {{val|abs}}  {% else %} 0 {% endif %}'
      unique_id: envoy_mqtt_grid_export
      qos: 0
      unit_of_measurement: "W"
      state_class: measurement
      device_class: power

and same for the batteries .. just easier to show in cards and the energy dashboard

    - name: envoy mqtt batteries discharge
      state_topic: "envoy/json"
      value_template: '{% set val = value_json[2]["activePower"] | round(0) | int(0)%} {% if val > 0 %} {{val}}  {% else %} 0 {% endif %}'
      unique_id: envoy_mqtt_batteries_discharge
      qos: 0
      unit_of_measurement: "W"
      state_class: measurement
      device_class: power
    - name: envoy mqtt batteries_charge
      state_topic: "envoy/json"
      value_template: '{% set val = value_json[2]["activePower"] | round(0) | int(0)%} {% if val < 0 %} {{val|abs}}  {% else %} 0 {% endif %}'
      unique_id: envoy_mqtt_batteries_charge
      qos: 0
      unit_of_measurement: "W"
      state_class: measurement
      device_class: power

from what i can see my ivp/meters key is as @joergbattermann pasted above.

[catsmanac]

Yes, sorry I overlooked. The good news is data for all 3 is in there. I'm off to do some code changes to get grid numbers in correctly and add battery numbers as well.

[catsmanac]

Is there a generator in your systems as well, or just batteries?

[joergbattermann]

No generator in our system @catsmanac - that would also require a different System Controller model, the 3G which (I think) forgoes the possibility to have more than 4 batteries (but no more than 8) in favor of the generator input.

Anyway, our system does not have a generator.

[gyordanov]

@catsmanac no generator - just batteries, but my controller is indeed Controller 3G (with a generator support) but the generator is not installed or configured (maybe way down the road).

[catsmanac]

Thanks for the confirmation @joergbattermann and @gyordanov.

[catsmanac]

To add the whole home backup situation to our tests cases I could use an HA Envoy debug log over a reload of the Envoy

• Enable debug on the Envoy Integration
• Reload the Envoy integration
• Let it run for 1-2 minutes
• Disable Debug again on the Envoy, debug log file will download to your device
• Edit/remove as needed and upload the file here.

Thanks

[joergbattermann]

@catsmanac I have a file in which I replaced our specific serial numbers' first/last numbers a bit but hope that helps:

home-assistant_enphase_envoy_2024-03-07T20-32-29.329Z.log

[catsmanac]

Thanks @joergbattermann

[gyordanov]

@catsmanac here is my file, replaced the serials but i kept them unique so it should be ok .... i hope

the full log was 22mbs so i filtered for enphase_envoy|pyenphase hoping that is what you needed ....

envoy_whole_home_backup_debug.log.gz

[catsmanac]

Thanks @gyordanov, yes I only need the ones you filtered so thats fine, as well as replacing the serials.

Found the issue why it shows storage instead of net-consumption, it's an easy fix. Adding the storage data will take more time and will be in a separate update after the initial fix for the net-consumption.

[gyordanov]

@catsmanac thank you for all this work!! enphase with homeassistant is way way better than just with their dashboard !!!!

[joergbattermann]

Thanks @gyordanov, yes I only need the ones you filtered so thats fine, as well as replacing the serials.

Found the issue why it shows storage instead of net-consumption, it's an easy fix. Adding the storage data will take more time and will be in a separate update after the initial fix for the net-consumption.

Great, thank you @catsmanac ! Should we/I create a separate topic/'issue' to track the storage data aspect more like a feature request than a bug?

[catsmanac]

I'm not sure where feature request should go, I think they reject feature requests in the issues section, not 100% sure. This one was spin-off from the issue reported in HA docs, but it looks as if feature requests are done in the community. I'll implement the storage CT anyway, now I know it exists. It's kind of the missing link I still had around battery as without the CT there's no real way to get energy in/out the battery reported.

[joergbattermann]

Got it, thanks a lot @catsmanac

[catsmanac]

Fix has been approved and should show up in upcoming release.

As for the 'storage CT' I need 1 more bit of information. CT have Energy Delivered and Energy Received values. The production CT, in my mental picture facing the switchboard, reports solar energy produced in Energy Delivered. The Net-Consumption CT, facing the switchboard, report Energy from grid in Energy Delivered and Energy to Grid in Energy Received.

So I would like to understand how the storage CT reports. Is it switchboard facing or battery facing.

In your diagnostic reports there's the ivp/meters/readings section. It has 3 segments, one for production, one for consumption and the 3rd for storage ct. And each has 3 subsections for the phases.

Can you find the 3rd main section, you can identify it with "eid": 704643840, and look for the lifetime delivered and received values actEnergyDlvd and actEnergyRcvd. The values are probably Energy loaded or supplied by battery or something similar. If you can map these that would be great. Also see the activePower value, does positive/negative imply charge/discharge, or
other way around?

          "eid": 704643840,

          "actEnergyDlvd": 2274358.602,
          "actEnergyRcvd": 2971414.344,

          "activePower": -2579.823,
 

Let me know if you have questions on this, Thanks

[joergbattermann]

Trying to identify the info you're looking for @catsmanac but not quite sure what I am looking exactly at (numbers don't match up). What I do see however is that activePower is negative for the storage meter / 704643840 one, while we charge.

Right now that meter's data as per diagnostic HA log looks like this.. while Enlighten/Envoy looks circa like this (circa meaning.. a few seconds apart from each other):

If it helps & you're interested, we can gladly do some sort of live debugging / call with screen- & data sharing with you if it helps.

I also noticed that local envoy seemingly does not display (not sure if it / its APIs report) the "split" in consumption between what the household consumption of the overall consumption is vs what's used to charge the batteries, see:

.. with the currently produced 8.4xx kW we werecharging / putting 7.6-ish kW into the batteries and 0.8kW covered the house's consumption at the time of taking that last screenshot. Enlighten breaks that 'overall consumption' up, local envoy visually does not.

[catsmanac]

The question boils down to: while charging, is actEnergyDlvd increasing or is actEnergyRcvd increasing for the storage ct between 2 reports like 10-15 min apart. 7.6kw would be something like ~1800 Wh change if charge would be stable in that 15 min.

          "eid": 704643840,

          "actEnergyDlvd": 2274358.602,
          "actEnergyRcvd": 2971414.344,

          "activePower": -2579.823,
 

[joergbattermann]

actEnergyRcvd is increasing - I had downloaded the diagnostics 32 minutes apart from each other earlier today - with the charge rate continuously having been in the 7-8kW range (if I remember correctly) - and the data from 11:32am local time was this:

.. while the one from 12:04pm was this:

[catsmanac]

Ah excellent, those are the numbers I was looking for. CT is facing the switchboard and receiving when charging. Thanks.

[joergbattermann]

Sure thing, let me know whenever you need more info / data

[catsmanac]

2024.3.1 is available including the fix for the grid import export.

Work on the storage ct is started. That will require 2 PR for pyenphase, the underlying module used, and a subsequent PR for HA itself. Will take some time for reviews and tests.