[VU+ Ultimo4K] Possible memory leak, need help on how to figure it out

[phunkyfish]

Hi,

I recently set up my Ultimo4K from scratch as I was getting crashing issues every few days.

• System OE: OE-Alliance 4.3
• Firmware version: OpenViX 5.3.039 (2020-07-19)
• Kernel / Drivers: 3.14.28 / 20190424
• OWIF: 1.4.0

Unfortunately this did not help, I can still observe the memory reaching capacity via the top command every few days. It will then eventually crash. Prior to a fresh install I replaced the HDD.

The device is also used for development as I maintain the pvr.vuplus (Enigma2) add-on for kodi so there will be a lot of use via OpenWebIf, both for the XML APIs and streaming server.

I'm not sure exactly when this started but I could use some guidance on how to track down what causes this. Outside of occasionally manually updating OpenWebIf with fixes I would not know too much about the enigma2 internals. For all I know the memory usage pattern is expected and the issue lies somewhere else!

Thanks in advance for any help.

P.S.

I should add that the Ultimo4K has 2xDVB-S2 FBC tuners and a DUAL DVB-T2 tuner. It is configured via ABM for both Freesat and Saorview in Ireland. This is the only configuration applied since the fresh install. The saorview channels appear as part of the freesat bouquets using a custom mix file. All EPG data is embedded in the stream.

 

[ccs]

Crash logs will be a good start, usually found in /media/hdd/logs

 

[phunkyfish]

Cool, I will grab those on the next crash. But I don't have a logs folder in /media/hdd. Do I need to create it or would they be located elsewhere?

 

[abu baniaz]

Default location is /home/root/logs/

You should try installing htop and using that instead of top, data is better presented.

I fear you won't get a solution as there was an issue about memory leak on OWIF GitHub.

EDIT:
https://github.com/E2OpenPlugins/e2openplugin-OpenWebif/issues/751

 

[phunkyfish]

The log was actually in /home/root/logs. Here it is:

View attachment Enigma2_crash_2020-09-15_21-55-00.log

EDIT: I have htop installed now too. Not sure if it's relevant or not to this but it used to take 4/5 seconds to play a channel from a remote kodi host. Now it takes about 20 seconds to play a channel.

 

[abu baniaz]

If it is causing a problem, please uninstall htop.


Some other threads with issue

https://forums.openpli.org/topic/25860-recordings-failed-to-allocate-filepush-buffer/
https://www.world-of-satellite.com/showthread.php?58563-Repeated-crashes-quot-Failed-to-allocate-filepush-buffer-contact-MiLo-quot

EDIT:
I can't see any fixes in ATV 's movielist for memory leak

https://github.com/openatv/enigma2/commits/6.4/lib/python/Components/MovieList.py

Issue was opened here

https://github.com/openatv/enigma2/issues/1026

 

[phunkyfish]

@"abu baniaz"

Thanks for the guidance. Do you know of anyway I can prove this is a leak in Movies.py? Its a leak coming from somewhere.

The kodi addon updates timers and recordings every few minutes. Let me try disabling the recordings update and see does that slow down the rate of the leak.

 

[abu baniaz]

Sorry, I don't know.

Would be nice to track it down and fix.

 

[ccs]

Rebooting the box once a day should avoid any more crashes.

 

[abu baniaz]

And maybe using cacheflush plugin.

Best to avoid the crash until issue is fixed.

 

[phunkyfish]

Well I have to prove it's Movies.py first. But even if I do I need to figure out how to find a memory leak in a python file. That will be the hard part. As there has been no movement on the issue open on OpenATV I guess the only this will get fixed is if I do it myself.

 

[phunkyfish]

After only a few hours of testing memory consumption appears to stay consistently below 200MB used. I'll leave it run throughout the day but it pretty much confirms the leak is in MovieList.py.

Now just need to figure out how to fix it.

 

[abu baniaz]

I hope you saw jbleyel's response.

Hi @AbuBaniaz , i have made a copy of MovieList.py from enigma2 and then i have removed all the GUI parts.
I think the leak is in the GUI part of MovieList.py.

 

[phunkyfish]

Yes, I saw it. Now I just need to figure out which parts are the GUI parts!

 

[phunkyfish]

Any hints or guidance?

 

[abu baniaz]

The issue should be solved with this commit:
https://github.com/OpenViX/enigma2/commit/ab302a73911ae00230a5ab47854a523736848544

Thanks to Betacentauri