weather_level_adj : wrong history and calculation

[ctosae]

It seems that the plugin shortly after the rain loses count of the rain that has fallen:

• Mar 20 04:21 'Irrigation needed:\n0.9mm'
  • Mar 20 04:21 'Total rainfall:\n2.66mm'
• Mar 20 05:21 'Irrigation needed:\n0.0mm' <-- ok !
  • Mar 20 05:21 'Total rainfall:\n4.1mm'
• Mar 20 06:21 'Irrigation needed:\n1.3mm' <-- ??
  • Mar 20 06:21 'Total rainfall:\n2.3mm'
• Mar 20 07:21 'Irrigation needed:\n3.4mm' <-- ??
  • Mar 20 07:21 'Total rainfall:\n0.92mm'
• Mar 20 08:22 'Irrigation needed:\n4.2mm' <-- ??
  • Mar 20 08:22 'Total rainfall:\n0.2mm'

In thix example if I have to start at 08 or 09 o'clock it does not consider a few hours before.

Full log:

Mar 20 03:21 'Current temperature:\n11.0deg.C'
Mar 20 03:21 '________________________________'
Mar 20 03:21 'Daily irrigation:\n4.0mm'
Mar 20 03:21 'Total rainfall:\n2.0mm'
Mar 20 03:21 'Water needed (4days):\n3.6mm'
Mar 20 03:21 '________________________________'
Mar 20 03:21 'Irrigation needed:\n1.6mm'
Mar 20 03:21 'Weather Adjustment:\n12.3%'

Mar 20 04:21 'Checking weather status...'
Mar 20 04:21 'Current temperature:\n11.1deg.C'
Mar 20 04:21 '________________________________'
Mar 20 04:21 'Daily irrigation:\n4.0mm'
Mar 20 04:21 'Total rainfall:\n2.66mm'
Mar 20 04:21 'Water needed (4days):\n3.6mm'
Mar 20 04:21 '________________________________'
Mar 20 04:21 'Irrigation needed:\n0.9mm'
Mar 20 04:21 'Weather Adjustment:\n6.9%'

Mar 20 05:21 'Checking weather status...'
Mar 20 05:21 'Current temperature:\n11.1deg.C'
Mar 20 05:21 '________________________________'
Mar 20 05:21 'Daily irrigation:\n4.0mm'
Mar 20 05:21 'Total rainfall:\n4.1mm'
Mar 20 05:21 'Water needed (4days):\n3.6mm'
Mar 20 05:21 '________________________________'
Mar 20 05:21 'Irrigation needed:\n0.0mm'
Mar 20 05:21 'Weather Adjustment:\n0.0%'

Mar 20 06:21 'Checking weather status...'
Mar 20 06:21 'Current temperature:\n11.0deg.C'
Mar 20 06:21 '________________________________'
Mar 20 06:21 'Daily irrigation:\n4.0mm'
Mar 20 06:21 'Total rainfall:\n2.3mm'
Mar 20 06:21 'Water needed (4days):\n3.6mm'
Mar 20 06:21 '________________________________'
Mar 20 06:21 'Irrigation needed:\n1.3mm'
Mar 20 06:21 'Weather Adjustment:\n10.0%'

Mar 20 07:21 'Checking weather status...'
Mar 20 07:21 'Current temperature:\n11.1deg.C'
Mar 20 07:21 '________________________________'
Mar 20 07:21 'Daily irrigation:\n4.0mm'
Mar 20 07:21 'Total rainfall:\n0.92mm'
Mar 20 07:21 'Water needed (4days):\n4.3mm'
Mar 20 07:21 '________________________________'
Mar 20 07:21 'Irrigation needed:\n3.4mm'
Mar 20 07:21 'Weather Adjustment:\n26.2%'

Mar 20 08:22 'Checking weather status...'
Mar 20 08:22 'Current temperature:\n11.3deg.C'
Mar 20 08:22 '________________________________'
Mar 20 08:22 'Daily irrigation:\n4.0mm'
Mar 20 08:22 'Total rainfall:\n0.2mm'
Mar 20 08:22 'Water needed (4days):\n4.4mm'
Mar 20 08:22 '________________________________'
Mar 20 08:22 'Irrigation needed:\n4.2mm'
Mar 20 08:22 'Weather Adjustment:\n32.3%'

I also don't understand where the data for 3 days before and 3 days of forecasts are saved (as per setting)

Current Mar 28 15:00:

34015 Mar 28 12:38 'forecast5day_lat=43.9575&lon=12.7376-20230328_12.json'
34100 Mar 28 13:38 'forecast5day_lat=43.9575&lon=12.7376-20230328_13.json'
34159 Mar 28 14:38 'forecast5day_lat=43.9575&lon=12.7376-20230328_14.json'
  475 Mar 27 16:36  history_2023_03_27-16_36_37.json
  480 Mar 27 17:36  history_2023_03_27-17_36_42.json
  479 Mar 27 18:36  history_2023_03_27-18_36_47.json

Why only Mar 27 at 16.36, 17.36 and 18.36?

Here another example at Mar 19 23:45:

'forecast5day_lat=43.9575&lon=12.7376-20230319_21.json'
'forecast5day_lat=43.9575&lon=12.7376-20230319_22.json'
'forecast5day_lat=43.9575&lon=12.7376-20230319_23.json'
 history_2023_03_18-16_18_36.json
 history_2023_03_18-17_18_41.json
 history_2023_03_18-18_18_46.json

Always only 3 history in one day at same time (16.xx, 17.xx and 18.xx)

[ctosae]

@Dan-in-CA I'm testing your project but I can contribute with a donation if I can use it :)

[ctosae]

@Dan-in-CA sorry, can you help to understand what is happening in this pluging?

[Dan-in-CA]

This plugin has been contributed and edited by several users over time.
The logic is not well defined. I think it needs a complete re-write.

What do you think such a plugin should do?

Dan

[ctosae]

Hi Dan, thanks for reply!

What do you think such a plugin should do?

The idea of this plugin is good, but if not fixable I think it could be simplified (without temperature and more).
A plus of SIP project would be to avoid wasting water and not to irrigate too much, so:

• calculate how much water rained in 24h/48h (*) to calculate the difference required
• calculate how much it will rain in the next 24h/48h (**) to avoid early irrigation

Based on the parameters of this plugin you could take a cue and keep:

• Minimum run time field can be used to prevent a program from running if the plugin calculates a very short irrigation time. A setting of 0 minutes and 0 seconds will allow the program to run (no minimum). [DEFAULT 0:00]
• Min percentage field can be used to ensure that some irrigation will always be applied. [DEFAULT 0]
• Max percentage field has a maximum possible value of 200. [DEFAULT 100]
• History days used and Forecast days used fields which are the (*) and (**) above [DEFAULT 24 or 48h]
• Daily irrigation field should hold an estimated amount of water your system applies on a per-day basis. (used to calculate % with rain fallen/will fall) [CUSTOM]
• Status box is used by the plugin to display information about it's operation (to verify that it is working)

[ctosae]

Still on debug... @Dan-in-CA
Instead of building a new one, the plugin needs just a "little" fix and it's perfect...

Like the first example https://github.com/Dan-in-CA/sip_plugins/issues/72#issue-1644061589 there are only 3 files per day:

# ls -l history*
-rw-r--r-- 1 root root 595 May 12 16:54 history_2023_05_12-16_54_45.json
-rw-r--r-- 1 root root 492 May 12 17:54 history_2023_05_12-17_54_50.json
-rw-r--r-- 1 root root 496 May 12 18:54 history_2023_05_12-18_54_55.json

In this case:

# cat history* | jq -r .rain
{
  "1h": 0.24
}
{
  "1h": 0.32
}
{
  "1h": 1.78
}

Where: 0.24+0.32+1.78 is....

cat /var/log/syslog | grep rainfall | tail -n10
#033[36msip-irrigation    |#033[0m b'Total rainfall:\n2.34mm'
#033[36msip-irrigation    |#033[0m b'Total rainfall:\n2.34mm'
#033[36msip-irrigation    |#033[0m b'Total rainfall:\n2.34mm'
#033[36msip-irrigation    |#033[0m b'Total rainfall:\n2.34mm'
#033[36msip-irrigation    |#033[0m b'Total rainfall:\n2.34mm'
#033[36msip-irrigation    |#033[0m b'Total rainfall:\n2.34mm'
#033[36msip-irrigation    |#033[0m b'Total rainfall:\n2.34mm'
#033[36msip-irrigation    |#033[0m b'Total rainfall:\n2.34mm'
#033[36msip-irrigation    |#033[0m b'Total rainfall:\n2.34mm'
#033[36msip-irrigation    |#033[0m b'Total rainfall:\n2.34mm'

So the problem is that it saves only 3 times per day instead of every 1h (in this example the rain was for about 10h with a total of 10mm)

# cat weather_level_adj.json | grep da
    "daily_irrigation": "4",
    "days_forecast": "3",
    "days_history": "3",

Not sure if the bug is about days_history = 3 (used as "hours_history" ?)

[ctosae]

I concluded that it is better to improve the code, also using a new openweathermap feature.
We've been up here a long time, so I hired a developer to do it
You can ignore the case, thanks

[Dan-in-CA]

Wow that's a first!
I do know that one of the early contributors to that plugin was a professional software engineer.

One thing that has bothered me about the weather plugin in its present form is the amount of data it stores on the SD card with every check of weather conditions. Instead of saving just the date it uses it stores the entire response from Openweathermap including a lot of redundant data that is not needed. I have always tried to minimize the wear on the SD card as described in the SIP documentation:
https://github.com/Dan-in-CA/SIP/wiki/SD-card-life

[ctosae]

One thing that has bothered me about the weather plugin in its present form is the amount of data it stores on the SD card with every check of weather conditions. Instead of saving just the date it uses it stores the entire response from Openweathermap including a lot of redundant data that is not needed. I have always tried to minimize the wear on the SD card as described in the SIP documentation: https://github.com/Dan-in-CA/SIP/wiki/SD-card-life

Great point, I noticed this too and got concerned about it, although the new SDs are robust in terms of rewrites.

So initially I thought about minimizing the save date (an array saved with pickle or even pickle+gzip)

But after some analysis my new idea is great and now possible, to have the history... without saving any data and also without any data saved in the cloud! hehe ;)