[Zgemma H7] Weird EPG errors. EPG Refresh. Recent versions of OpenViX e.g. 5.4.008.

[BrokenUnusableAccount]

I'm trying to get the most over the air EPG I can by using EPG Refresh (as much as possible for both Freeview and 28.2°E satellite both radio and television).

I have EPG Refresh set to spend 150 seconds on each of 11426H, 11778V and each of the Freeview multiplexes in the small hours every morning.

I keep getting weird extra programs in the satellite TV part of the EPG from the previous week but not at the same time.

For instance on 5th March 2021 on satellite BBC 2 both SD and HD I got program "Chris Packham: Asperger's and Me" 11:54-12:54 (actually shown on, as far as I can see, 3rd March 23:30-00:30, before EPG Refresh ran).
It's kind of showing over the middle of "Athletics..." 09:00-13:00.

There are numerous similar examples all through the EPG for around this time on 5th March.

At the moment I'm thinking it's something to do with OpenTV EPG going over the top of existing Freesat EPG information but it seems like it must be some weird kind of bug that makes it manifest itself in this way.

Not really expecting an quick fix, just thought it's so weird I must document it.

 

[ronand]

What sources have you enabled? With multiple sources in use the data is different from each source. Its recommended not to enable the freesat epg when using opentv data.

 

[adm]

What sources have you enabled? With multiple sources in use the data is different from each source. Its recommended not to enable the freesat epg when using opentv data.

I'm starting to believe that there is another bug in the EPG collection from 28.2E satellite and opentv irrespective of how the IEPG is accessed/obtained. A few weeks back I thought the problem with two programs being displayed or overwritten in a EPG time slot may just be that the 7 day EPG obtained with the OpenTV EPG downloader being overwritten by the now/next data EIT when program times changed.

I now believe that I'm seeing something similar to BefuddledBrian in that the overwriting of the current/future EPG is with a program broadcast in the past – and with data that may/will have been in the EPG previously. My last experience of this the overwrite the programs in the EPG was from the previous evening at the time when I switched from watching live TV to watching a delayed version of a program. This was programs from, say, 8pm on a Tuesday being overwritten into the EPG data for 10am on a following Wednesday. The highlighting in the (grid) EPG goes wrong at this point as though both program names are present.. It’s not every channel – usually more than one but less than six. Unfortunately although I regularly see this overwriting/double EPG entries I cannot establish a firm pattern for what is triggering it.

Other things that may be relevant:

I have both opentv EPG downloader and EIT enabled because I want the over the air EPG for both FTA Sky and UK terrestrial Freeview. Freesat EPG enable = no

I put my box into deep standby every night by first switching off with the remote off button which puts the box into standby. A power timer than kicks in which then puts the box into deep standby 10 minutes later. This means that the EPG data in RAM must be transferred to/from the hard disk on shutdown and subsequent power-up.

In 5.4.007 and previous versions I often see a missing 4 hours of EPG on satellite and for all channels. The 7 day EPG data is available prior AND after to the missing 4 hours. A manual forced download using OpenTV EPG downloader does not correct the problem. The only way seems to use the menu to delete the EPG cache and then force a opentv download. One thing to note that during this "blank" 4 hours in the EPG the program description on all channels is identical and it is a partial description of a program previously broadcast - in my experience it only consists of a few words but can be identified as a real program.

I have a habit of watching a recording and than switching back to live TV without pressing the recording stop button which means the recording may carry on for perhaps 10 minutes in the background before automatically reaching the end. I have menu → setup → epg → grid EPG → channel selection = selecet current service which means that when I go back to the (grid) epg the program that I’m watching on live TV is highlighted UNLESS I have previously watched a recording and not terminated it with a record stop button press. Afterwards the highlighted channel in the (grid) EPG is always the channel I was watching prior to watching the recording – and it never changes irrespective of changing channels for live TV.

 

[ronand]

The missing 4 hours may be down to not letting it download data long enough. It has been reported a few times to use 180 seconds instead of 150 (I have mine set to 180s). There is no harm trying a longer download duration. I haven't noticed anything myself on my H7 and I do have EIT epg enabled (for saorview) which does download UK satellite data too as well as using opentvzapper. I also use epgimport for 30W. Also its so harm to delete the epg cache and refresh the data as the database may be corrupted.

 

[adm]

The missing 4 hours may be down to not letting it download data long enough. It has been reported a few times to use 180 seconds instead of 150

Its nothing to do with time. 150/180 seconds is EPG refresh and not OpenTV EPG download which I'm currently using. In addition, in the past I've actually selected IEPG channel and left it for 10+ minutes and it still hasn't filled the 4 hour hole in the epg.

I haven't noticed anything myself on my H7 and I do have EIT epg enabled (for saorview) which does download UK satellite data too as well as using opentvzapper. I also use epgimport for 30W. Also its so harm to delete the epg cache and refresh the data as the database may be corrupted.

The problem is that the EPG cache may get corrupted but may not be noticed. Autotimers etc. are trying to set timers based on this data. The 4 hour hole in EPG data may only be seen if you scroll through 3 or 4 days of data and then you may find it in a random place, something that I don't often do. A reboot doesn't clear this 4 hour hole nor does a manual force to refetch EPG data. It is if the EPG cache has a section protected by a read only flag prevting an overwrite. When the problem does occur it is always a 4 (ish) hour hole.

It has been hinted that in the EPG settings menu that the periodic Automatic save and Automatic reload functions are buggy. I have both options set to "no" but maybe the same mechanism is used to save and reload the EPG data when the box is placed into deep standby and possibly the same problems may not be seen by those who only put the box into standby.

BTW, OpenTV EPG download has no option to first delete the cache

 

[ronand]

I regularly look ahead through the epg for programs of interest and have not noticed this behaviour. The only times I have noticed gaps are when the data download has been interrupted. It was Brian that said he was using 150 secs to download his data. By the looks of his description it seems his data is corrupted (my data for 5th March on BBC2 is correct as I have been looking at it all week for the athletics). I don't clear my cache unless needed but it does need it now and again.

 

[Joe_90]

I have had similar issues with EPG "corruption" recently. I'm using OpenTVzapper (with Freesat EPG disabled, but with EIT reading enabled for my terrestrial channels). It's set on the default options (run every 6 hours).
On Channel 4 I noticed that the EPG entry for 09.30 contained the details for "Naked Attraction" which had been broadcast the previous evening. No number of forced re-runs of OpenTVzapper would clear the entry. I had to delete the EPG cache and then re-run. Like @adm, I have PowerTimers which shut the box down if more than 20 minutes in standby. I would also manually put the box in Deep Standby when I'm finished using it. I can't figure out what is triggering the issue.

 

[Huevos]

Can you guys please provide screengrabs.

 

[adm]

Can you guys please provide screengrabs.

Blank epg example ! posted a month ago

https://www.world-of-satellite.com/showthread.php?63961-OpenTV-EPG-downloader-oddity

Double entry/overwrite example I posted 4 months ago

https://www.world-of-satellite.com/showthread.php?63384-Grid-EPG-highlight-bug

I initially posted that it may be a highlighting bug but have since come to the conclusion that this occurs when the EPG has been overwritten with data from the past and/or data from the past has been overwriten with new epg data but the previous data not deleted.

I've seen the same problems on 28.2E sky FTA channels when using EPG Refresh, OpenTV EPG download and a bespoke method where I set a timer to tune into IEPG every day at 6:30am and record 15 minutes (the recording was routed to trash but it populated the EPG). From this I don't believe it is necessarily related to the mechanism that sets the tuning into IEPG. With reference to double entries/overwritting of the EPG I've seen examples for a long time so not just 5.4.xxx. In the past week I've seen the blank 4 hours once and double/overwritten EPG a couple of times.

The double entries/overwritting of the EPG has been seen in the past when using Freesat epg and OpwenTv disabled.

 

[BrokenUnusableAccount]

Thank you @ronand, @adm, and @fat-tony.

When I wrote my original message I had these EPG settings:
EPG button action = Grid EPG
INFO button action = Event Info
EPG location = /media/hdd
EPG filename = epg
Automatic save = Yes
Save every (in hours) = 1
Automatic reload = No
Show EIT now/next in infobar = No
Enable EIT EPG = Yes
Enable MHW EPG = No
Enable FreeSat EPG = Yes
Enable ViaSat EPG = No
Enable Netmed EPG = No
Enable Virgin EPG = No
Enable OpenTV EPG = Yes
Maintain old EPG data for = 45
Include EIT in http streams = Yes
Include AIT in http streams = Yes
Country for EPG event rating information = Auto Detect
Country for EPG event genre information = Auto Detect

and these EPG Refresh settings:
Refresh EPG automatically = Yes
Duration to stay on service (seconds) = 150
EPG refresh auto-start earliest (hh:mm) = 03:37
EPG refresh auto-start latest (hh:mm) = 08:27
Delay if not in standby (minutes) = 3
Refresh EPG using = Fake recording
Show Advanced Options = Yes
Skip protected Services = Background only
Show Setup in plugins = Yes
Show Setup in extension menu = Yes
Show 'EPGRefresh Start now' in extension menu. = Yes
Show popup when refresh starts and ends. = Yes
Wake up from standby for EPG refresh. = Yes
Force scan even if receiver is in use. = Yes
Shutdown after EPG refresh. = No
Flush EPG before refresh = Yes
Inherit Services from AutoTimer = No
Run AutoTimer after refresh = Yes

and EPG Refresh was zapping to 11426H (Freesat), 11778V (OpenTV) and each of the Freeview multiplexes in that order.

In particular notice that I have EPG Refresh set to flush the EPG cache before it begins zapping.
The spurious old information still appeared on the EPG.
In fact yesterday I'd also tried manually flushing the EPG, rebooting and manually flushing it again before starting EPG Refresh.
No difference. The spurious old EPG stuff still reappeared from somewhere!

With multiple sources in use the data is different from each source. Its recommended not to enable the freesat epg when using opentv data.

I'm trying to get some EPG for satellite radio. But yes - point taken, I may yet have to give up and accept that I can't have EPG for satellite radio.


Tonight I've fiddled with EPG Refresh and I now have it doing OpenTV first, then Freesat and then Freeview, and I'm going to see if that makes any difference.
So far I think it does seem better.

It's not that straightforward getting EPG Refresh to zap in a particular order. It tends to sort things into an order of it's own preference but since it's only the transponder (or multiplex) that matters you can try choosing different things on the same transponders and checking what order they end up being in.

 

[Huevos]

Blank epg example ! posted a month ago

https://www.world-of-satellite.com/showthread.php?63961-OpenTV-EPG-downloader-oddity

Double entry/overwrite example I posted 4 months ago

https://www.world-of-satellite.com/showthread.php?63384-Grid-EPG-highlight-bug

I initially posted that it may be a highlighting bug but have since come to the conclusion that this occurs when the EPG has been overwritten with data from the past and/or data from the past has been overwriten with new epg data but the previous data not deleted.

I've seen the same problems on 28.2E sky FTA channels when using EPG Refresh, OpenTV EPG download and a bespoke method where I set a timer to tune into IEPG every day at 6:30am and record 15 minutes (the recording was routed to trash but it populated the EPG). From this I don't believe it is necessarily related to the mechanism that sets the tuning into IEPG. With reference to double entries/overwritting of the EPG I've seen examples for a long time so not just 5.4.xxx. In the past week I've seen the blank 4 hours once and double/overwritten EPG a couple of times.

The double entries/overwritting of the EPG has been seen in the past when using Freesat epg and OpwenTv disabled.

The most recent commit was 19 days ago so we need screengrabs from a build more recent than that. I can't find any holes in my EPG data at the moment.

Also, although it is possible to discard all epg data before running OpentvZapper, it is a hack. epgcache.cpp should be in charge of deleting items as and when necessary. So if anything needs fixing it is in there.

 

[BrokenUnusableAccount]

Can you guys please provide screengrabs.

I'll do screenshots next time I get examples of this problem.

 

[BrokenUnusableAccount]

Can you guys please provide screengrabs.

I put EPG Refresh back to zapping to 11426H (Freesat), 11778V (OpenTV) and each of the Freeview multiplexes in that order and manually started it at about 00:15 on 5th March 2021.
(At this stage I don't know if that "helped" it go wrong or not)

I think, but I'm not sure, that it seemed like it wouldn't go wrong if I ran EPG Refresh before midnight.

Here is screenshot showing
Match of the Day 12:09-12:44 over the middle of Football Focus 12:00-13:00 on BBC One HD and BBC One South.
the end of Being Bridget Jones 11:54-12:54 over the end of Athletics... 08:45-12:20
(then 60s Icons:... 12:20-13:00 over then end of that) on BBC Two HD.
Made in Britain 12:09-12:59 over the end of New: Simply Raymond Blac 11:40-12:40
(then ITV Lunchtime News 12:40-12:55 over the end of that) on ITV Meridian HD.
Mock the Week 12:04-12:44 over the top of American Picker 12:00-13:00 on Dave.
Around the World in 80 Treasures 12:09-13:09 over the middle of ..programmes start at 7:00pm 11:30-17:30 on BBC Four HD.

 

[something_fishy]

I think that I am seeing the same thing. It seems most visible as a random program appearing late morning on top of BBC2's long rolling news coverage. Or in the following example sports coverage:



This is using the built in openTV downloader with all other epg sources turned off:




This is with the most recent version of Openvix:


To confirm this effect I cleared the epg cache and did a OpenTV forced download from the extension menu. This populated the epg without this problem. This is the day after following a few automatic updates.

Before this I tried the freesat epg which I also don't think demonstrated this issue.

Thanks

 

[ronand]

Yes Bridget Jones has crept into my BBCHD epg listing for tomorrow

 

[ccs]

Being Bridget Jones actually broadcast on bbc2 hd on Thursday (4th) at 23:30 for 1 hour.

Also to be broadcast next Friday (12th) at 02:00.

 

[abu baniaz]

Maybe it was in the Diary?

 

[Joe_90]

Being Bridget Jones actually broadcast on bbc2 hd on Thursday (4th) at 23:30 for 1 hour.

Also to be broadcast next Friday (12th) at 02:00.

I was actually watching some of that programme last night. I checked my own EPG just now and BBC2HD does not have that entry overwriting the athletics tomorrow. The fact that it has happened in the same place for two users would indicate an issue with the actual processing of the EPG data, rather than any setting problem, particularly as EIT and Freesat data is turned off in one configuration. It's very similar to what I've seen myself, where programmes scheduled for late evening (23:00ish) will overwrite events prior to 12:00 on a subsequent day.

 

[ronand]

I think we can now rule out corrupt local epg files. I have EIT epg enabled (for saorview) and opentvzapper for 28e. But of course the EIT epg on 28e get absorbed too.

 

[Joe_90]

I have the same config settings as I use Saorview also. This is not a recent phenomenon, though. I've seen it in the past, but it was usually dismissed as a settings issue/clash between OpenTV and Freesat sources, which doesn't seem to be the case now.