Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Out of memory using weather with Gadgetbridge, causing font corruption #1788

Closed
1 task done
Hegz opened this issue Jun 30, 2023 · 27 comments
Closed
1 task done

Out of memory using weather with Gadgetbridge, causing font corruption #1788

Hegz opened this issue Jun 30, 2023 · 27 comments
Labels
bug Something isn't working

Comments

@Hegz
Copy link

Hegz commented Jun 30, 2023

Verification

  • I searched for similar bug reports and found none was relevant.

What happened?

Clock text becomes corrupt

What should happen instead?

I was expecting to see readable fonts

Reproduction steps

I've updated to 1.13, and enabled weather with gadget bridge and quickweather. After about 9 hours being on on my wrist, I glanced at my watch and the clock font's were corrupt. This has been happening for the past several days after updating to 1.13 and enabling weather support. I don't know the time periods for the other times.

The following are corrupted:

  • Pinetimestyle -- Clock text unreadable. Date, weather, steps are OK.
  • Infineat face -- Clock text unreadable. Day of week, date, and steps are OK.
  • Casio G7710 -- Clock text unreadable. Date/week/steps battery text are OK.

Unaffected faces

  • Digital
  • Analog
  • Terminal

More details?

I can switch to an unaffected face, and the watch will continue to work as normal. Attempting to flip back to Pinetimestyle causes the screen to lock before displaying the Pinetimestyle face, and then it reboots.

The previous reboot was at about 6:00am, which was a spontaneous reboot. I didn't see if there was corruption prior to the reboot. The corruption kicked in at around 3:00pm the same day.

I've attempted to reinstall the assets pack, as I thought it may have changed since I first installed it. This didn't change the behaviour.

Version

1.13

Companion app

Gadget Bridge

@Hegz Hegz added the bug Something isn't working label Jun 30, 2023
@Hegz
Copy link
Author

Hegz commented Jun 30, 2023

The text corruption appears the same as shown on #1376

@FintasticMan
Copy link
Member

When this happens, can you check the values shown under "Memory heap" on the 3rd page of the About page (you can get there quickly by holding the button for 2-3 seconds)? I have a feeling that that is caused by running out of memory.

@kieranc
Copy link
Contributor

kieranc commented Jul 3, 2023

The text corruption appears the same as shown on #1376

I disagree, this is a new and different corruption, sadly. It's now more transparent, with vertical banding. The extra strange bit is that PTS fonts are internal, previously only fonts loaded from ext flash were affected but now it seems to be affecting internal fonts as well. I saw this behaviour a couple of days ago but it only affected the temperature label on PTS, while I was provoking it, so I thought I was to blame... but apparently not.

@JF002
Copy link
Collaborator

JF002 commented Jul 3, 2023

Yep, I agree with @kieranc, I don't think this issue has the same cause than #1376 since PTS does not load anything from the external storage.

But it might still be memory related : the weather service does use dynamic memory allocation. If the memory is not correctly released, or if the weather service receives too many notifications, the memory could be too fragmented, or completely filled, which would prevent the watchface from loading correctly... Well this is just an hypothesis since I've never encountered this issue on my PineTime.

@Hegz would you like to post one or two of pictures of the display when the text is corrupted?

Also, would you like to disable the weather functionality from your companion app and see if this issue still occurs? I would like to know if this issue is linked to the weather service or to something else.

And @FintasticMan 's suggestion is also interesting:

When this happens, can you check the values shown under "Memory heap" on the 3rd page of the About page (you can get there quickly by holding the button for 2-3 seconds)? I have a feeling that that is caused by running out of memory.

@hubortje
Copy link

hubortje commented Jul 3, 2023

I experience a similar issue, yet my runtime is way longer than 9 hours. I'm not quite sure, but my watch is running since the 1.13.0 update was released.

The corruption is cleared by swiping into one of the adjacent menus, and swiping back to the watchface. Pressing the power button repeatedly to turn the watchface off and on again, does not solve the issue. Did not try to reboot the device tho.

Find a picture of the corrupted clock attached.

Corrupted watchface!

@Sokoloft
Copy link

Sokoloft commented Jul 3, 2023

I am also having the same issue. Below are a couple pictures of both the PTS watch face and Casio watch face being affected.

IMG_20230702_210042453_HDR

IMG_20230625_204016905_HDR (1)

I will try to look at the memory information as @FintasticMan suggested. However the watch likes to crash whenever this happens.

@JF002
Copy link
Collaborator

JF002 commented Jul 3, 2023

I will try to look at the memory information as @FintasticMan suggested. However the watch likes to crash whenever this happens.

Yeah, this is likely to happen in case of memory allocation error, unfortunately. Anyway, you could still have a look at that screen from time to time, and especially have a look at the "min free" value (which shows the lowest amount of memory available since InfiniTime was started on your watch) and "alloc err" (allocation error) and "overfl err" (overflow error). Ideally, these two last value should always be 0.

@hubortje
Copy link

hubortje commented Jul 3, 2023

random shower-hypothesis:
I had no trouble regarding the corrupted clockface, until I switched the settings in TWFG from a six-hours-interval, to an hourly-interval – maybe this has to do with the issue?

@jwest23
Copy link

jwest23 commented Jul 4, 2023

Possibly related, I've seen a couple of random reboots of my watch. I don't know if it's happening when I'm not looking, too. I have physically seen two reboots, though

@Sokoloft
Copy link

Sokoloft commented Jul 4, 2023

So far I'm 8h 31m uptime and min free value dropped a tiny bit. It started at 4744 and now its at 4736. Idk if that helps any. I'll monitor it closely tomorrow and see if I can get more uptime where I can monitor it.

@Hegz
Copy link
Author

Hegz commented Jul 4, 2023

PXL_20230704_022437529
PXL_20230704_022431169
PXL_20230704_022424973
PXL_20230704_022410995

I don't recall seeing the corrupt face since I reported this, but I also didn't reboot this at 4:30.

Here are images of the corrupt watch face, as well as the Diagnostic menus that I was able to access. I attempted to change to the Infinineat watch face, and it rebooted.

I'll attempt turning off weather on the watch-face & companion app (Gadgetbridge, quickweather) and report back in 24h, or when the device crashes again.

@Sokoloft
Copy link

Sokoloft commented Jul 4, 2023

With one day and 5h uptime. The min free value dropped to 4560. Worth mentioning I have my weather app not sending data to GadgetBridge. It seems more stable since doing that.

I also deleted the fonts and images folder on the watch and re-install the resource file. However @FintasticMan mentioned that shouldn't make a difference. Its likely the fact I have no weather data going to GB.

@Hegz
Copy link
Author

Hegz commented Jul 5, 2023

Almost 24 hours without weather data, and I'm not having any stability problems. "Min free" hasn't changed from 8080 all day. Some different values in the 4th screen of About. Not sure if it matters if it dose I already took a picture and can post.

I'll try turning on the quickweather gadget bridge Integration but leaving off the watch face display.

@Hegz
Copy link
Author

Hegz commented Jul 6, 2023

With the Integration on, and watch display off, it still crashed. it took 27h this time.

  • free: 9720
  • Min free: 128
  • Alloc err: 88

@Hegz
Copy link
Author

Hegz commented Jul 7, 2023

Last update to this, tried watch display weather off, quickweather on, and using the Casio watch face. Crashed at 7 hours.

Memory heap
Free 12560
Min free 32
Alloc err 275
Ovrfl err 0

@Sokoloft
Copy link

Sokoloft commented Jul 7, 2023

Just to add to the information. My watch has 3d 8h uptime still with 4560 min free. Using the Casio watch face and with the weather turned off in my weather application (fork of forecastie).

So anyone observing this can sum up that something with the weather integration is the issue. However even prior to this release I have had the watch face corrupt in the past. Opening an application on the watch and going back to the watch face would fix it. I think that is a known issue though.

@kieranc
Copy link
Contributor

kieranc commented Jul 7, 2023

So anyone observing this can sum up that something with the weather integration is the issue. However even prior to this release I have had the watch face corrupt in the past. Opening an application on the watch and going back to the watch face would fix it. I think that is a known issue though.

Agreed, the previous corruption was different, and seemed to be resolved by the heap unification, this one is new and probably caused by weather data being stored on the watch. I think that removing the 24h forecast from the data GB sends will help but I don't know how much, but more than that, I think we need to set valid expiry times for events on the timeline so they get cleaned up as they expire - currently, even ignoring the 24h forecast, if GB sends data every hour, the events have a fixed 6h expiry time so we could potentially have 5 expired but not cleaned up events in the timeline, even more if we count the 24h forecast. I don't know how much memory this consumes but evidence suggests it's enough to be a problem.

edit: I just prepared a PR for gadgetbridge to remove the 24h forecast and I'm wondering if it would be better to reduce the weather data sent even further to just what we need/use right now? We're sending wind speed and humidity and possibly a condition code, none of which are used.

@FintasticMan
Copy link
Member

What I've personally noticed, is that the amount of used memory stays pretty consistent, and then it suddenly gets used up. I haven't had time to test it with a debugger connected, but it seems to me that it might not be entirely due to just there being too many events on the timeline.

@Hegz
Copy link
Author

Hegz commented Jul 9, 2023

There was a GadgetBridge update with the note "InfiniTime: Fix weather expiry time"

It's still happening after that update. It may have taken a little longer, but was still less then 24h to crash.

@segaretroboy
Copy link

segaretroboy commented Jul 17, 2023

Had this happen to me as well, rebooted the watch before I thought to get debug info, but in my instance the weather and other icons on the pinetime style got corrupted too.
IMG_20230716_215128

@hectorzg
Copy link

I have experienced the same watch face corruption but in addition to that my pinetime screen only showed a black screen and there was nothing I could do to fix it, sometimes the issue fixed itself after a few hours and others I had to reinstall the latest firmware via gadgetbridge. This issue happened with any watchface.

I have downgraded to firmware 1.12 and haven't had any issues in the last couple of days.

@zonorti
Copy link

zonorti commented Jul 23, 2023

I have experienced the same behavior a few hours after enabling weather push (via Gadgetbridge).
Without weather I had stable 10 days of uptime.
image

@segaretroboy
Copy link

Wanting to add as this has continued on my device, after setting up Quick Weather and turning on Gadgetbridge integration, the watch has begun doing this much more frequently. Confirmed I had Allocation errors each time it happened.

@FintasticMan FintasticMan changed the title Watch face font corruption Out of memory using weather with Gadgetbridge, causing font corruption Aug 20, 2023
@StevenAlexander44
Copy link

StevenAlexander44 commented Aug 26, 2023

If you use the debug menu in gadgetbridge and spam click "Set weather", then you can see the bug in about 10-20 clicks from a fresh restart, and after about 20-30, the watch completely dies and reboots. After a fresh reboot, for me, at first it shows a blocked symbol "🛇", then after clicking "Set weather" once, it shows the correct weather, then after clicking "Set weather" a second time and after, it starts to show correct temperature shortly, and immediately rewrite with the incorrect number.

Versions of software in testing are:
InfiniTime: 1.13 with infineat and casio installed
Gadgetbridge: Nightly 0.75.1-20d0d61ed
breezy-weather: v4.6.4-beta

@FintasticMan FintasticMan linked a pull request Oct 14, 2023 that will close this issue
@vkareh
Copy link
Contributor

vkareh commented Jan 31, 2024

I have not experienced this since the SimpleWeatherService became available in 1.14

@AgentConDier
Copy link

Agreed, 13d uptime right now, with weather enabled and working 👍

@FintasticMan
Copy link
Member

Oh yeah, this issue only exists with the old weather implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.