AB-Com PULSe 4K SE. Crazy offer only £99! FREE UK DELIVERY!
Superb quality and spec AB-Com PULSe 4K SE. Crazy offer! Only £99! FREE UK DELIVERY! 4K UHD, Enigma 2, Multiboot 4 images & more!...
AB-Com PULSe 4k Rev II Twin tuner only £149! FREE UK DELIVERY!
Superb quality and spec AB-Com PULSe 4K Rev II Twin Satellite tuner only £149! FREE UK DELIVERY! 4K UHD, Enigma 2, SATA HDD facility, Multiboot 4 images & more!...

[VU+ Duo2] AutoTimer error when it runs.

M

Migu

New member
Joined
Dec 27, 2021
Messages
10
Reaction score
0
Points
1
Couple of weeks after firmware updating I started to get Autotimer error every time when it runs.
One specific autotimer item is causing the error. If I disable that item, no more errors.
AutoTimer_error_6.0.006_2.webp
Autotimer error log:
Code: 
<160600.3889> 15:39:56.3095 [eEPGCache] lookup events with 'Simpsonit' in title (ignore case)
<160600.5162> 15:39:56.4369 [Task] >>> Error: [<Components.Task.FailedPostcondition object at 0x69a99340>]
<160600.5169> 15:39:56.4375 [Task] job Components.Task.Job name=Automaattiajastus #tasks=23 completed with [<Components.Task.FailedPostcondition object at 0x69a99340>] in Components.Task.Task name=Simpsonit_14
<160600.5184> 15:39:56.4390 [Task] unrecoverable task failure
<160600.5186> 15:39:56.4392  Automaattiajastus
Virhe: [Failure instance: Traceback: <class 'SystemError'>: <built-in function eEPGCache_search> returned a result with an error set
/usr/lib/python3.9/threading.py:973:_bootstrap_inner
/usr/lib/python3.9/threading.py:910:run
/usr/lib/python3.9/site-packages/twisted/_threads/_threadworker.py:47:work
/usr/lib/python3.9/site-packages/twisted/_threads/_team.py:181:doWork
--- <exception caught here> ---
/usr/lib/python3.9/site-packages/twisted/python/threadpool.py:238:inContext
/usr/lib/python3.9/site-packages/twisted/python/threadpool.py:254:<lambda>
/usr/lib/python3.9/site-packages/twisted/python/context.py:118:callWithContext
/usr/lib/python3.9/site-packages/twisted/python/context.py:83:callWithContext
/usr/lib/enigma2/python/Plugins/Extensions/AutoTimer/AutoTimer.py:344:JobStart
/usr/lib/enigma2/python/Plugins/Extensions/AutoTimer/AutoTimer.py:441:parseTimer
]<160600.5188> 15:39:56.4394

I have done couple of experiments, and here is the summary:
  • If there are lot of search results (~50 or more) for a keyword in autotimer, it fails.
  • Easy to reproduce by using OpenWebIf AutoTimer->Simulate.
  • The issue is not in the AutoTimer itself. It seems to be an EPG search issue. I reproduced the same kind of issue without autotimer, if searching the same keyword from EPG by WWW-browser:
    http://<IP-address>/web/epgsearch?search=<keyword>
  • I tried to limit the number of search results by modifying the python file: /usr/lib/enigma2/python/Plugins/Extensions/AutoTimer/AutoTimer.py line 441
    Original (maxmatches=3000):
    epgmatches = epgcache.search(('RITBDSE', 3000, EPG_searchType, match, caseMap[timer.searchCase])) or []
    Modified (maxmatches=44):
    epgmatches = epgcache.search(('RITBDSE', 44, EPG_searchType, match, caseMap[timer.searchCase])) or []
    No AutoTimer error after that modification. (bigger maxmatches >= 45 gives the error)
  • Same here: /usr/lib/enigma2/python/Plugins/Extensions/OpenWebif/controllers/models/services.py line 1001:
    Original:
    events = epgcache.search(('IBDTSENRW', 128, search_type, sstr, 1))
    Modified:
    events = epgcache.search(('IBDTSENRW', 43, search_type, sstr, 1))
    No Search error after that modification.

Some kind of resource limitation issue?
 
Is there a specific term that causes the crash?

Code: 
http://192.168.1.28/web/epgsearch?search=dog

On my box that gives 128 results an no crashs.
 
http://<IP>/web/epgsearch?search=Simpsonit
Gives "Processing Failed".
Code: 
<167255.8441> 17:30:48.9629 [eEPGCache] lookup events with 'Simpsonit' in title (ignore case)
<167255.9666> 17:30:49.0854
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/twisted/protocols/basic.py", line 548, in dataReceived
why = self.lineReceived(line)
File "/usr/lib/python3.9/site-packages/twisted/web/http.py", line 2243, in lineReceived
self.allContentReceived()
File "/usr/lib/python3.9/site-packages/twisted/web/http.py", line 2369, in allContentReceived
req.requestReceived(command, path, version)
File "/usr/lib/python3.9/site-packages/twisted/web/http.py", line 1003, in requestReceived
self.process()
--- <exception caught here> ---
File "/usr/lib/python3.9/site-packages/twisted/web/server.py", line 229, in process
self.render(resrc)
File "/usr/lib/python3.9/site-packages/twisted/web/server.py", line 294, in render
body = resrc.render(self)
File "/usr/lib/enigma2/python/Plugins/Extensions/OpenWebif/controllers/base.py", line 206, in render
data = func(request)
File "/usr/lib/enigma2/python/Plugins/Extensions/OpenWebif/controllers/web.py", line 1557, in P_epgsearch
return getSearchEpg(search, endtime, fulldesc, False, self.isJson)
File "/usr/lib/enigma2/python/Plugins/Extensions/OpenWebif/controllers/models/services.py", line 1001, in getSearchEpg
events = epgcache.search(('IBDTSENRW', 128, search_type, sstr, 1))
builtins.SystemError: <built-in function eEPGCache_search> returned a result with an error set
http://digibox/web/epgsearch?search=Simpsonit (7) gives less results, and returns 23 elements as expected.

Yes indeed, some other common keyword can return 128 results! such as:
http://digibox/web/epgsearch?search=Uuti
Very strange!
 
So it's probably limited to a maximum of 128?
 
According to the (comment in the) code the maximum number of matches to return is set in the second member of the incoming tuple.

And the AutoTimer.py code (#1) sends in a value of 3000. So something else could be happening there.
 
Last edited:
birdman said:
According to the (comment in the) code the maximum number of matches to return is set in the second member of the incoming tuple.

And the AutoTimer.py code (#1) sends in a value of 3000. So something else could be happening there.
Click to expand...
In webIF the limit is 128.
Code: 
events = epgcache.search(('IBDTSENRW', 128, search_type, sstr, 1))
 
Yes, probably there is something else than the count limit.

If I try to read all programs from FOX-channel, operation fails:
Code: 
http://digibox/api/epgservice?sRef=1:0:1:211:3001:20F6:EEEE0000:0:0:0:
Processing Failed

<329565.9072> 14:35:45.9966
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/twisted/protocols/basic.py", line 548, in dataReceived
why = self.lineReceived(line)
File "/usr/lib/python3.9/site-packages/twisted/web/http.py", line 2243, in lineReceived
self.allContentReceived()
File "/usr/lib/python3.9/site-packages/twisted/web/http.py", line 2369, in allContentReceived
req.requestReceived(command, path, version)
File "/usr/lib/python3.9/site-packages/twisted/web/http.py", line 1003, in requestReceived
self.process()
--- <exception caught here> ---
File "/usr/lib/python3.9/site-packages/twisted/web/server.py", line 229, in process
self.render(resrc)
File "/usr/lib/python3.9/site-packages/twisted/web/server.py", line 294, in render
body = resrc.render(self)
File "/usr/lib/enigma2/python/Plugins/Extensions/OpenWebif/controllers/base.py", line 206, in render
data = func(request)
File "/usr/lib/enigma2/python/Plugins/Extensions/OpenWebif/controllers/web.py", line 1607, in P_epgservice
return getChannelEpg(getUrlArg(request, "sRef"), begintime, endtime, self.isJson)
File "/usr/lib/enigma2/python/Plugins/Extensions/OpenWebif/controllers/models/services.py", line 782, in getChannelEpg
events = epgcache.lookupEvent(['IBDTSENCW', (_ref, 0, begintime, endtime)])
builtins.SystemError: <built-in function eEPGCache_lookupEvent> returned a result with an error set
There seems to be a problematic <e2event> in the middle of EPG-data.
I can read <e2event> before that:
Code: 
http://digibox/api/epgservice?sRef=1:0:1:211:3001:20F6:EEEE0000:0:0:0:&time=1644501900&endTime=20

events	
0	
picon	"/picon/fox.png"
id	44023
date	"To 10.02.2022"
begin	"16:05"
begin_timestamp	1644501900
duration	30
duration_sec	1800
end	"16:35"
title	"Simpsonit"
shortdesc	"Kausi 16. Jakso 3/21. Vihollinen vuoteessani. Kiusanhenki piinaa Bartia, ja Lisa pelkää, ettei hän ole koskaan kyllin laiha."
longdesc	""
sref	"1:0:1:211:3001:20F6:EEEE0000:0:0:0:"
sname	"FOX"
tleft	1557
progress	-20367
now_timestamp	1644410246
genre	""
genreid	0
result	true
And after that:
Code: 
http://digibox/api/epgservice?sRef=1:0:1:211:3001:20F6:EEEE0000:0:0:0:&time=1644505500&endTime=20

events	
0	
picon	"/picon/fox.png"
id	44025
date	"To 10.02.2022"
begin	"17:05"
begin_timestamp	1644505500
duration	30
duration_sec	1800
end	"17:35"
title	"Simpsonit"
shortdesc	"Kausi 2. Jakso 21/22. Radioaktiivinen mies. Bart, Milhouse ja Martin ostavat harvinaisen sarjakuvanumeron jaetulla omistuksella, millä on karmivat seuraukset heidän ystävyydelleen."
longdesc	""
sref	"1:0:1:211:3001:20F6:EEEE0000:0:0:0:"
sname	"FOX"
tleft	1616
progress	-21155
now_timestamp	1644410300
genre	""
genreid	0
result	true
But that specific <e2event> (16:35-17:05) fails::
Code: 
http://digibox/api/epgservice?sRef=1:0:1:211:3001:20F6:EEEE0000:0:0:0:&time=1644503700&endTime=20
Processing Failed
In the Vu+Duo2 screen I can see it in the EPG, but description is empty.

Couple of days earlier, I tried 5.4.016 firmware, and I took the whole XML-log from FOX channel successfully.
The second <e2event> below is the problematic one. It's having a long <e2eventdescription> with couple of scandinavian characters (äö):

Code: 
        <e2event>
		<e2eventid>44023</e2eventid>
		<e2eventstart>1644501600</e2eventstart>
		<e2eventduration>1800</e2eventduration>
		<e2eventcurrenttime>1644078455</e2eventcurrenttime>
		<e2eventtitle>Simpsonit</e2eventtitle>
		<e2eventdescription>Kausi 16. Jakso 3/21. Vihollinen vuoteessani. Kiusanhenki piinaa Bartia, ja Lisa pelkää, ettei hän ole koskaan kyllin laiha.</e2eventdescription>
		<e2eventdescriptionextended/>
		<e2eventservicereference>1:0:1:211:3001:20F6:EEEE0000:0:0:0:</e2eventservicereference>
		<e2eventservicename>FOX</e2eventservicename>
		<e2eventgenre id="0"/>
	</e2event>
	<e2event>
		<e2eventid>44024</e2eventid>
		<e2eventstart>1644503400</e2eventstart>
		<e2eventduration>1800</e2eventduration>
		<e2eventcurrenttime>1644078455</e2eventcurrenttime>
		<e2eventtitle>Simpsonit</e2eventtitle>
		<e2eventdescription>Kausi 2. Jakso 20/22. Aviokriisi. Homerin on valittava kahdesta vaihtoehdosta: osallistua avioliittonsa pelastavaan terapiaan tai livistää järvelle kalastamaan ennätyssaalista. Kotona Bart ja Lisa käyttävät tilannetta hyväkseen säännöist</e2eventdescription>
		<e2eventdescriptionextended/>
		<e2eventservicereference>1:0:1:211:3001:20F6:EEEE0000:0:0:0:</e2eventservicereference>
		<e2eventservicename>FOX</e2eventservicename>
		<e2eventgenre id="0"/>
	</e2event>
	<e2event>
		<e2eventid>44025</e2eventid>
		<e2eventstart>1644505200</e2eventstart>
		<e2eventduration>1800</e2eventduration>
		<e2eventcurrenttime>1644078455</e2eventcurrenttime>
		<e2eventtitle>Simpsonit</e2eventtitle>
		<e2eventdescription>Kausi 2. Jakso 21/22. Radioaktiivinen mies. Bart, Milhouse ja Martin ostavat harvinaisen sarjakuvanumeron jaetulla omistuksella, millä on karmivat seuraukset heidän ystävyydelleen.</e2eventdescription>
		<e2eventdescriptionextended/>
		<e2eventservicereference>1:0:1:211:3001:20F6:EEEE0000:0:0:0:</e2eventservicereference>
		<e2eventservicename>FOX</e2eventservicename>
		<e2eventgenre id="0"/>
	</e2event>

Could this issue be related to long description text with UTF-8 characters?
 
Is this being populated with EpgImport? Possible some chars are not being handled correctly. Or the source file contains chars that are not UTF8 encoded.
 
Huevos said:
Is this being populated with EpgImport? Possible some chars are not being handled correctly. Or the source file contains chars that are not UTF8 encoded.
Click to expand...
Surely that should be a matter for any importer to handle.
It should ensure that everything in the EPG cache is utf-8.
 
Huevos said:
Is this being populated with EpgImport? Possible some chars are not being handled correctly. Or the source file contains chars that are not UTF8 encoded.
Click to expand...

No, I am not using EpgImport. EPG-data comes from the broadcasted stream (DVB-T2 receiver).
 
birdman said:
Surely that should be a matter for any importer to handle.
It should ensure that everything in the EPG cache is utf-8.
Click to expand...
I'm not sure how this is helpful to find the route of the problem.
 
Huevos said:
Can you go back to a 5.4 image and retry.
Click to expand...
Currently there is only one problematic event in all channels when using 6.0.006 image.
http://digibox/api/epgservice... fails for it. Description for it is empty in the screen.

I went back to a 5.4.016 image, retried and collected data.
I can see the description of the problematic event, but the last word is truncated, and the last character is wrong. It's À although it should be ä.
screen_snapshot_fw_5_4_016_mod.webp
XML and json data for it:
Code: 
<e2event>
	<e2eventid>44889</e2eventid>
	<e2eventstart>1644774480</e2eventstart>
	<e2eventduration>725</e2eventduration>
	<e2eventcurrenttime>1644768940</e2eventcurrenttime>
	<e2eventtitle>Sportliv</e2eventtitle>
	<e2eventdescription>Naisleijonien kesäleiri 2020 oli Anni Keisalalle isku vasten kasvoja - ja tarpeellinen sellainen. Nuorena hän pärjäsi lahjoillaan, mutta enää se ei riittänyt. Hänen oli otettava itseään niskasta kiinni. Keisala teki käänteentekevän pä</e2eventdescription>
	<e2eventdescriptionextended></e2eventdescriptionextended>
	<e2eventservicereference>1:0:19:5DF:2001:20F6:EEEE0000:0:0:0:</e2eventservicereference>
	<e2eventservicename>Yle Teema & Fem HD</e2eventservicename>
	<e2eventgenre id="64">Urheilu: urheilu (yleinen)</e2eventgenre>
</e2event>

  {
   "begin": "19:48", 
   "sname": "Yle Teema & Fem HD", 
   "end": "20:00", 
   "title": "Sportliv", 
   "genreid": 64, 
   "now_timestamp": 1644768971, 
   "shortdesc": "Naisleijonien kes\u00e4leiri 2020 oli Anni Keisalalle isku vasten kasvoja - ja tarpeellinen sellainen. Nuorena h\u00e4n p\u00e4rj\u00e4si lahjoillaan, mutta en\u00e4\u00e4 se ei riitt\u00e4nyt. H\u00e4nen oli otettava itse\u00e4\u00e4n niskasta kiinni. Keisala teki k\u00e4\u00e4nteentekev\u00e4n p\u00e4", 
   "picon": "/images/default_picon.png", 
   "begin_timestamp": 1644774480, 
   "duration": 12, 
   "duration_sec": 725, 
   "sref": "1:0:19:5DF:2001:20F6:EEEE0000:0:0:0:", 
   "longdesc": "", 
   "date": "Su 13.02.2022", 
   "progress": -3040, 
   "tleft": 103, 
   "genre": "Urheilu: urheilu (yleinen)", 
   "id": 44889
  },
The description is long, but not the longest one. The problematic event description, which does not work:
Code: 
Naisleijonien kesäleiri 2020 oli Anni Keisalalle isku vasten kasvoja - ja tarpeellinen sellainen. Nuorena hän pärjäsi lahjoillaan, mutta enää se ei riittänyt. Hänen oli otettava itseään niskasta kiinni. Keisala teki käänteentekevän pä
Char count: 234
Byte count: 248
But this works, although it's one byte longer:
Code: 
12/13. Ruotsin suosituimman tietovisailun toinen semifinaali. Musiikista vastaa Vargas & Lagola, ja tänään ovat mukana myös vierailevat tähdet Agnes Carlsson, Roger Pontare, Nassim Al Fakir, Sami Al Fakir, Fares Al Fakir sekä Ayman Al Faki
Char count: 243
Byte count: 249
The only difference I can see is such that the first one is having scandinavian character (2 bytes UTF-8) in the end.
 
I don't know if this applies, but there is a problem with uk freeview dvb-t2 sd broadcasts and some accents.

I have added an extra line to encoding.conf (https://github.com/OpenViX/enigma2/commits/future-release/data/encoding.conf)

to change the fallback encoding table (which doesn't appear to be set in sd broadcasts). It works for me because I only use dvb-t2.

Code: 
.
.
.
chn GB18030
fas UTF-8
per UTF-8
fa UTF-8
pes UTF-8

#Fallback encoding when in dvb-text no encoding table is given
#and no Countrycode  and no transponders configs in this config file
[COLOR="#FF0000"]*   ISO6937[/COLOR]

#Sorry for that.. in DVB Spec this is the default behavior 
#when no other encoding is given in dvb-texts..
#but this breaks too much providers yet..
.
.
.

I've put my copy of encoding.conf in /etc/enigma2, it then overrides the system version.
 
Last edited:
I agree with ccs. Something is wrong with the encoding. It is a pity you didn't show us a screenshot from the 6.0 image of the same problem.
 
Huevos said:
I agree with ccs. Something is wrong with the encoding.
Click to expand...
So, going back to my previous point, it would make sense(?) if everything putting text into the epg cache put it in as utf-8 (ignoring errors on encoding?).
That way everything retrieving data would always get utf-8 and always work.
 
Huevos said:
I agree with ccs. Something is wrong with the encoding. It is a pity you didn't show us a screenshot from the 6.0 image of the same problem.
Click to expand...

Unfortunately I cannot take a screenshot from the 6.0 image of the same problem right now, because currently there is no problematic events in any of channels.

The description for it was just empty in the screen. It was like in the below screenshot, which I have edited manually:
screen_snapshot_fw_6_0_006_edited.webp
Could it be so that truncating UTF-8 encoded Unicode text in the end may cut in the middle of a UTF-8 encoded code-point and cause corruption?
 
birdman said:
So, going back to my previous point, it would make sense(?) if everything putting text into the epg cache put it in as utf-8 (ignoring errors on encoding?).
That way everything retrieving data would always get utf-8 and always work.
Click to expand...
If you start with an incorrectly encoded string and convert it to UTF8 the output will be wrong. That is what encoding.conf is for. And even then it still relies on the input maintaining the encoding correctly.
 
I was seeing, for example, Benalmádena until I changed the encoding table to ISO6937, when the correct text Benalmádena was stored.

@Joe_90 worked out what the encoding should have been, although I don't know how.
 
ccs said:
I was seeing, for example, Benalmádena until I changed the encoding table to ISO6937, when the correct text Benalmádena was stored.

@Joe_90 worked out what the encoding should have been, although I don't know how.
Click to expand...

IIRC - because ISO6937 was specifically mentioned in the Freeview/Freesat D Book as one of the acceptable encodings.


EDIT - here's the reference in "D" book:

Code: 
F.1 Introduction
Table F-1 lists the displayable characters that shall be implemented by all
decoders conforming to this specification.
Note that the UK MHEG-5 profile specifies several characters that are not
defined within ISO/IEC 6937 [42] on which the DVB SI specifications are
based. These characters may be broadcast in, and shall be displayed
correctly within MHEG-5 applications; there is no guarantee that they will be
displayed correctly within SI delivered text. The characters in the shaded
cells are shown only to emphasise gaps.

It goes on to list UCS-2, UTF-8 and ISO 6937 encodings and the name for each character.
 
Last edited:
You must log in or register to reply here.

Similar threads

S
[VU+ Duo4K] EPG Failing to Download - Out of Memory Error
2
Replies
25
Views
1K
twol
twol
D
XMLTV EPG Import error
Replies
4
Views
3K
dsayers
dsayers
enf70
[ViX_Misc] OpenWebif issue
Replies
11
Views
4K
pembo
P
Y
[Mut@nt HD2400] AutoTimer Error [Failure instance: Traceback:
Replies
5
Views
2K
Sicilian
Sicilian
L
Mut@ntHD51 - reboot himself problem.
Replies
5
Views
3K
Joe_90
Joe_90

OpenViX Feeds Status

Back
Top