mqtt stops cloud upload #52

townlema · 2023-12-03T09:36:34Z

hi guys, this is a great project and I have it working against my new inverter and sending mqtt data. However, I see that it seems to stop the upload to the s-miles cloud. Is this the case that you have experienced?

I'm attached to the inverter via my normal wlan and not the inverter's internal wifi.

Hope someone can shed some light on this one,

Thanks

dominikandreas · 2023-12-03T09:44:10Z

I think the reason might be the update frequency. Requesting data from the inverter too often might prevent it from sending data to the cloud. We should add an option to adjust the update frequency to see if we can find a setting that works well

townlema · 2023-12-03T09:50:20Z

Yes I think it is related to that because if you quit the docker image then it starts uploading to the cloud again. We need to find a good balance so both work together. Either that or I dump the cloud site but the cloud app is graphically nice (especially for the wife who doesn't care about all the techie stuff)

DennisOSRM · 2023-12-03T10:04:25Z

That's an interesting thing, thanks for the report. I had experimented a bit and landed on 30.5 second as a cooperative time interval that shouldn't block the update to the cloud. I also checked and it seems to work on my side. One feature that I have planned though is to implement the five second update mode, and then an option to also forward that data into the cloud.

townlema · 2023-12-03T11:12:10Z

I'll continue to play and investigate. Great work though - very happy someone is pushing this forward. Thanks

DennisOSRM · 2023-12-03T11:18:42Z

One thing that just crossed my mind is that the 30.5s threshold is tight, and the inverter is running internal countdowns when to send the next update to the cloud. It is just a hunch that in case the tool is restarted shortly after it queried the inverter it might be that the internal countdowns never reaches zero. Hard to verify this hypothesis, tho.

townlema · 2023-12-03T13:50:06Z

QQ - currently as mentioned I connect to the inverter off my main wlan. Can you connect to the inverter in the same way off the inverter's internal wifi network - maybe through this, if you can retrieve the data still, it would not interfer with uploading data to the s-miles cloud? Just a thought.

DennisOSRM · 2023-12-09T13:30:38Z

QQ - currently as mentioned I connect to the inverter off my main wlan. Can you connect to the inverter in the same way off the inverter's internal wifi network - maybe through this, if you can retrieve the data still, it would not interfer with uploading data to the s-miles cloud? Just a thought.

Interesting idea. Will try.

joemat · 2023-12-21T13:07:06Z

I changed the request delay to 60 seconds, with that the data is again also pushed to the cloud. mqtt/home assistant is still working of course.

The delay of 30.5 seconds did not work for me

henkwiedig · 2023-12-22T09:23:02Z

I can confirm this. Switched to 60.5 seconds and the s-miles cloud is updated again. Maybe make this a config switch.

DennisOSRM · 2023-12-22T09:24:42Z

Yeah, a mode to co-exist with the cloud makes sense.

Phil-Rei · 2024-01-23T09:25:35Z

How do I get the coop mode to work?

I've waited for the fix and created a new test vm (Ubuntu 22.04.3 LTS)

This is my config.toml, which is passed to the docker container:

inverter_host = "192.168.100.74"
coop_mode = true
update_interval = 60500

[simple_mqtt]
host = "192.168.222.1"
username = "MQTTuser"
password = "******"
port = 1994

So coop_mode is set to true and update_interval is increased as well.
This is how I've created the container:
sudo docker run -it -d -v ./config.toml:/config.toml --name hoymiles dennisosrm/hms-mqtt-publisher

But still the updates are every 30 sec, preventing s-miles cloud from updating.

DennisOSRM · 2024-01-24T07:11:56Z

Did you try the most recent version. Do you see any output on the console when the tool starts?

Phil-Rei · 2024-01-24T08:33:24Z

Did you try the most recent version. Do you see any output on the console when the tool starts?

I've created the test virtual machine on Monday 2024-01-22 from scratch. The fixed version was released earlier, so I thought I'll automatically get the most recent version. But maybe the docker repository is not up to date?

Anyway, here's the output when I start the container without (-d [detached]):

[2024-01-24T08:05:15Z INFO  hms_mqtt_publish] Running revision: 189413f95db615e1fad27c2e2538d1500ff446af
[2024-01-24T08:05:15Z INFO  hms_mqtt_publish] inverter host: 192.168.100.74
[2024-01-24T08:05:15Z INFO  hms_mqtt_publish] Publishing to simple MQTT broker
[2024-01-24T08:05:16Z INFO  hms2mqtt::inverter] Inverter is Online

Docker inspect:

[
    {
        "Id": "41a0751a6abdfe5e48a325f14affa57d598b11e2e4a975382ca9e6181ccf8fbe",
        "Created": "2024-01-24T08:05:15.694649308Z",
        "Path": "/bin/sh",
        "Args": [
            "-c",
            "hms-mqtt-publish"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 2366,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2024-01-24T08:05:15.976530208Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:c8673061969aa162065609be88d82738ab997f2359dec60acd49b7bb805e3bc0",
        "ResolvConfPath": "/var/lib/docker/containers/41a0751a6abdfe5e48a325f14affa57d598b11e2e4a975382ca9e6181ccf8fbe/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/41a0751a6abdfe5e48a325f14affa57d598b11e2e4a975382ca9e6181ccf8fbe/hostname",
        "HostsPath": "/var/lib/docker/containers/41a0751a6abdfe5e48a325f14affa57d598b11e2e4a975382ca9e6181ccf8fbe/hosts",
        "LogPath": "/var/lib/docker/containers/41a0751a6abdfe5e48a325f14affa57d598b11e2e4a975382ca9e6181ccf8fbe/41a0751a6abdfe5e48a325f14affa57d598b11e2e4a975382ca9e6181ccf8fbe-json.log",
        "Name": "/hoymiles",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "docker-default",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/home/user/Desktop/config.toml:/config.toml"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "ConsoleSize": [
                24,
                80
            ],
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "private",
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": [],
            "BlkioDeviceWriteBps": [],
            "BlkioDeviceReadIOps": [],
            "BlkioDeviceWriteIOps": [],
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": null,
            "PidsLimit": null,
            "Ulimits": [],
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware",
                "/sys/devices/virtual/powercap"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/6bda0bb16c780a6d8fffb1b7d829f8383108d649d5fb455a853fb20dca06995d-init/diff:/var/lib/docker/overlay2/2f5e3d5111da9b358b74f5e3668111b3b004c3faf449264373755a6af7e34aba/diff:/var/lib/docker/overlay2/ebcce50afb881fb2be3f9fc769425736577e56dd76a65fecb919360bd61bf5b3/diff:/var/lib/docker/overlay2/8f7a75b7fee99fe05ef9a38e5d619af91a39d3f0b6eb97ae9f163f2b441fad88/diff",
                "MergedDir": "/var/lib/docker/overlay2/6bda0bb16c780a6d8fffb1b7d829f8383108d649d5fb455a853fb20dca06995d/merged",
                "UpperDir": "/var/lib/docker/overlay2/6bda0bb16c780a6d8fffb1b7d829f8383108d649d5fb455a853fb20dca06995d/diff",
                "WorkDir": "/var/lib/docker/overlay2/6bda0bb16c780a6d8fffb1b7d829f8383108d649d5fb455a853fb20dca06995d/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/home/user/Desktop/config.toml",
                "Destination": "/config.toml",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "41a0751a6abd",
            "Domainname": "",
            "User": "",
            "AttachStdin": true,
            "AttachStdout": true,
            "AttachStderr": true,
            "Tty": true,
            "OpenStdin": true,
            "StdinOnce": true,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "hms-mqtt-publish"
            ],
            "Image": "dennisosrm/hms-mqtt-publisher",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "org.opencontainers.image.created": "2023-12-10T13:08:06.733Z",
                "org.opencontainers.image.description": "HMS-XXXXW-2T publisher and Home Assistant addon",
                "org.opencontainers.image.licenses": "BSD-2-Clause",
                "org.opencontainers.image.revision": "189413f95db615e1fad27c2e2538d1500ff446af",
                "org.opencontainers.image.source": "https://github.com/DennisOSRM/hms-mqtt-publisher",
                "org.opencontainers.image.title": "hms-mqtt-publisher",
                "org.opencontainers.image.url": "https://github.com/DennisOSRM/hms-mqtt-publisher",
                "org.opencontainers.image.version": "v0.3"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "82593e7226ef09d309799eafec63bbb6d414f1d7f431abdf8dd2f50974ffc675",
            "SandboxKey": "/var/run/docker/netns/82593e7226ef",
            "Ports": {},
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "d62ace01cd3c423d7eb9255f3c8bc7bad9047fe522fe1e7264f771eaca260ace",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.2",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "41a0751a6abd"
                    ],
                    "MacAddress": "02:42:ac:11:00:02",
                    "NetworkID": "0918a08c065e27bd2f38a5230e167813984d485bdf50c47eccc00d84ccae1561",
                    "EndpointID": "d62ace01cd3c423d7eb9255f3c8bc7bad9047fe522fe1e7264f771eaca260ace",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "DriverOpts": null,
                    "DNSNames": null
                }
            }
        }
    }
]

Also did a pull:

Using default tag: latest
latest: Pulling from dennisosrm/hms-mqtt-publisher
Digest: sha256:be42a63cb890712b6f3457d1425f0aa8a4474892d440f2d6d995f2f3ae3f5f4b
Status: Image is up to date for dennisosrm/hms-mqtt-publisher:latest
docker.io/dennisosrm/hms-mqtt-publisher:latest

Phil-Rei · 2024-01-24T08:36:36Z

I think the Docker repository still has the non fixed version:

DennisOSRM · 2024-01-24T08:50:15Z

Yep. A new version is to be released some time this week to docker.

Phil-Rei · 2024-01-24T09:03:22Z

Thank you for the support! 👍

I've now created the image from the Dockerfile. Here is what I did (maybe someone has the same problem):

git clone https://github.com/DennisOSRM/hms-mqtt-publisher.git
cd hms-mqtt-publisher
*** EDIT config.toml ! ***
sudo docker build -t hms-mqtt-publisher .
sudo docker run -it -d -v ./config.toml:/config.toml --name hoymiles hms-mqtt-publisher
sudo docker update --restart unless-stopped hoymiles

gendelo3 · 2024-03-15T11:09:05Z

Thanks for the software!
For me, even a 60 s interval wasn't working. 120 s is currently working.

I investigated a bit and I have a thought on this:

I noticed that the S-Miles cloud updates are sent at specific times (x:00, x:15, x:30, x:45), plus-minus a few seconds. Is this the same for every HMS-800W-2T?
Now, even with the update-interval set to 60.5 s, it happened that I got a mqtt message e.g. at 11:14:36, less than 30 seconds before the expected cloud update at 11:15.

May it be, that this is the reason?
So even with using more than 30 / 60 etc seconds as the interval, if we start the program at the wrong time, we might sometimes or even always "block" the cloud update?

If I am correct, a workaround to keep the cloud upload working always would be to use a multiple of 60s (minus the time needed to execute the code) as the interval and making sure to start at in a "fitting" part of a minute on the clock, e.g. in the range betwen 15-20s "after the full minute". A fix in software could be to allow checking the current time and e.g. skipping an update if it clashes with the update of the cloud.

gendelo3 · 2024-03-16T10:31:10Z

I got it to update reliably by skipping the publishing in the S-Miles cloud update interval:
gendelo3@5bb36d5

Maybe something like this or any other working approach could be merged?

DennisOSRM · 2024-03-16T10:54:16Z

Looks super interesting! Do I read this right that every 15 minutes an update is sent to the cloud? Perhaps this feature could be opt-in behind a config flag?

gendelo3 · 2024-03-16T11:01:51Z

From my testing it seems to be that way, although I just figured it out by trial and error, so there are no guarantees that I am right.
By anecdotal evidence, the earliest I saw in my 1 week of using the S-Miles cloud was 15 seconds before the "full" 15 minutes. And maybe 9 seconds after.

Putting this behind a config flag sounds like a good idea!

DennisOSRM · 2024-03-16T11:13:08Z

Looking forward to a pull request then 😉

DennisOSRM added bug Something isn't working question Further information is requested labels Dec 9, 2023

DaGo1120 mentioned this issue Dec 9, 2023

Feature Request: Hoymiles HMS 800W-2T Support for built-in DTUBI WiFi-Models lumapu/ahoy#1189

Open

DennisOSRM mentioned this issue Jan 11, 2024

Implement an optional cooperative mode that will also update S-Miles Cloud #78

Merged

2 tasks

DennisOSRM closed this as completed Jan 12, 2024

gendelo3 mentioned this issue Mar 17, 2024

Cooperative Mode for S-Miles cloud with parameter to enable / disable it. Will pause polling the inverter at appropriate times #110

Open

2 tasks

Matthies mentioned this issue May 19, 2024

update_interval is ignored #123

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mqtt stops cloud upload #52

mqtt stops cloud upload #52

townlema commented Dec 3, 2023

dominikandreas commented Dec 3, 2023

townlema commented Dec 3, 2023

DennisOSRM commented Dec 3, 2023

townlema commented Dec 3, 2023

DennisOSRM commented Dec 3, 2023

townlema commented Dec 3, 2023

DennisOSRM commented Dec 9, 2023

joemat commented Dec 21, 2023

henkwiedig commented Dec 22, 2023

DennisOSRM commented Dec 22, 2023

Phil-Rei commented Jan 23, 2024

DennisOSRM commented Jan 24, 2024

Phil-Rei commented Jan 24, 2024

Phil-Rei commented Jan 24, 2024

DennisOSRM commented Jan 24, 2024

Phil-Rei commented Jan 24, 2024

gendelo3 commented Mar 15, 2024 •

edited

Loading

gendelo3 commented Mar 16, 2024

DennisOSRM commented Mar 16, 2024

gendelo3 commented Mar 16, 2024

DennisOSRM commented Mar 16, 2024

mqtt stops cloud upload #52

mqtt stops cloud upload #52

Comments

townlema commented Dec 3, 2023

dominikandreas commented Dec 3, 2023

townlema commented Dec 3, 2023

DennisOSRM commented Dec 3, 2023

townlema commented Dec 3, 2023

DennisOSRM commented Dec 3, 2023

townlema commented Dec 3, 2023

DennisOSRM commented Dec 9, 2023

joemat commented Dec 21, 2023

henkwiedig commented Dec 22, 2023

DennisOSRM commented Dec 22, 2023

Phil-Rei commented Jan 23, 2024

DennisOSRM commented Jan 24, 2024

Phil-Rei commented Jan 24, 2024

Phil-Rei commented Jan 24, 2024

DennisOSRM commented Jan 24, 2024

Phil-Rei commented Jan 24, 2024

gendelo3 commented Mar 15, 2024 • edited Loading

gendelo3 commented Mar 16, 2024

DennisOSRM commented Mar 16, 2024

gendelo3 commented Mar 16, 2024

DennisOSRM commented Mar 16, 2024

gendelo3 commented Mar 15, 2024 •

edited

Loading