Vix 5.1.006. Time wrong

[twol]

@SpaceRat
“In addition, I got a new job lately, so I don't have 12h a day left for E2 anymore ...“
Congratulations

“Image size“ - thought that was why you introduced upx and “flashsize“. ??

 

[SpaceRat]

@SpaceRat
“In addition, I got a new job lately, so I don't have 12h a day left for E2 anymore ...“
Congratulations

Thanks

“Image size“ - thought that was why you introduced upx and “flashsize“. ??

Actually, FLASHSIZE was introduced to create "bigger, better, Burger King" images, e.g. with translations on the console too, more sophisticated recovery (e.g. a "dialog" based emergency recovery tool on the box, in most cases, a box can still be reached through ssh and/or telnet, even if everything else is f*cked up, even networking).
I also wanted to get rid of certain Turbo-DM800 restrictions, that means tools with limited functionality, just because we needed to keep the image as small as possible to still be able to build images for 64MB junk boxes, while almost no really usable box has less than 256 MB of flash.

And the idea for the dialog based recovery was, that any IPv6 capable machine inside a LAN can locate and connect any IPv6 capable machine, even if the other side is entirely borked, because the link-local address remains valid, even if you otherwise don't use IPv6 at all!
So a small tool on any PC could locate a borked E2 box, initiate an ssh or telnet session and invoke the recovery routine.

Almost everything would be recoverable through such a system:
- Store the borked settings
- Gather debug/crash logs from boxes that Joe Average could never access due to networking being borked
- Reset to default values
- Restore a different set of backup settings
- Flash a backup image through ofgwrite
- Flash a newer/older original image through ofgwrite
- ...

"dialog" based means: A user interface like "make menuconfig", with menus and submenus ... thus very user-friendly, especially compared to a bunch of shell commands.

It ended up with dozens of boxes being unflashable due to the CFE flash not even being able to flash images with as few as 100 MB, although the boxes had 256, 512 or even 1024 MB of flash size (This btw. is also the reason why image backups are often worthless, because as soon as you create a backup of original image + some plugins, you also often exceed these ~96 MB).

That's why we added the virtual FLASHSIZE 96 MB:
No real box has 96MB, it's just an indicator that the box doesn't suffer from "smallflash" (= 64MB), but also has very limited image sizes for USB flash.

 

[Joe_90]

However, ntpdate itself can be run with a -p1 option to only get one sample, and then takes 0.7 seconds on my et8000.

root@et8000:~# time ntpdate -q -p 1 ntp.ubuntu.com 
server 91.189.91.157, stratum 2, offset 0.000598, delay 0.11975
server 91.189.94.4, stratum 2, offset 0.000928, delay 0.04651
server 91.189.89.199, stratum 2, offset 0.001112, delay 0.04568
server 91.189.89.198, stratum 2, offset 0.001159, delay 0.04585
23 Jan 18:41:11 ntpdate[1397]: adjust time server 91.189.89.199 offset 0.001112 sec

real    0m0.740s
user    0m0.010s
sys     0m0.007s

That should be good enough.

Thanks @birdman - interesting that the "-p1" option is available in ntpdate. As I have a time server (stratum 1) on my LAN I could take advantage of that. Maybe "ntpclient" might be more appropriate for a quick initial time setting on our boxes, though? Good find!

 

[birdman]

Maybe "ntpclient" might be more appropriate for a quick initial time setting on our boxes, though? Good find!

Sticking with ntpdate, but speeding up its first (boot-time) usage should be fine.

 

[SpaceRat]

.... ET10K

root@et10000:~# cat /proc/stb/fp/rtc
0

(I use DVB time.)

It has been reported for that box.
The affected boxes have different stickers on the front, but behind them all are the same driver devs.

Gesendet von meinem SM-N910F mit Tapatalk

 

[birdman]

As I have a time server (stratum 1) on my LAN I could take advantage of that.

Yes. But the way this setting is used it a little odd.
In particular, as far as I can tell if you ever change it to something other than "pool.ntp.org" you can never change it back to "pool.ntp.org". Something I need to look at.

 

[SpaceRat]

The images can learn the NTP server via DHCP if the DHCP server supports that option.
No need for any manual adjustment in that case.

Gesendet von meinem SM-N910F mit Tapatalk

 

[Joe_90]

The images can learn the NTP server via DHCP if the DHCP server supports that option.
No need for any manual adjustment in that case.

Gesendet von meinem SM-N910F mit Tapatalk

I've tested that option myself - DHCP server pointing to NTP server on my LAN. I disabled the option subsequently because of some comment in one of the threads saying that enigma doesn't use DHCP-supplied NTP addresses. I can't remember exactly what the comment said, but I disabled in case there was some impact on my setup.

 

[SpaceRat]

I've tested that option myself - DHCP server pointing to NTP server on my LAN. I disabled the option subsequently because of some comment in one of the threads saying that enigma doesn't use DHCP-supplied NTP addresses.

It does (Since 25 days):
https://github.com/oe-alliance/oe-a...08e6007#diff-1033ba7550a019b1efc5adcb2f00ad7c
[ntp] Various enhancements

- Only wait as long as necessary (and only on ifup)
- Add support for (and prefer) local NTP servers (e.g. own router) learned through DHCPv4 (odhcp6c does not support the ntp-servers option, so no IPv6 support yet)
- Always step (rather than slew) time on ifup

and

https://github.com/oe-alliance/oe-alliance-core/commit/24966d4b1e1e2c7d67181a8d7a7a9fdc31ec08f0
[busybox-udhcpc] Add support for learning NTP servers through DHCPv4

In action:

Jan 24 20:15:22 solo2se daemon.notice ntpdate[15230]: adjust time server [b]192.168.[/b]XXX.1 offset -0.000973 sec
Jan 24 20:15:22 solo2se daemon.notice stb-hwclock: Current system time has been written into FP pseudo RTC.

 

[birdman]

The images can learn the NTP server via DHCP if the DHCP server supports that option.
No need for any manual adjustment in that case.

It can - if DHCP is setting it. But that may not fit in with how enigma2 is setting it. Not sure yet - I'm not sure where a DHCP setting ends up, or how it ends up there - I seem to recall a patch a few days ago to handle that.
EDIT: only now have I read the previous post....

 

[birdman]

Yes. But the way this setting is used it a little odd.
In particular, as far as I can tell if you ever change it to something other than "pool.ntp.org" you can never change it back to "pool.ntp.org". Something I need to look at.

Not only that, but of you do a re-flash and restore your settings they will imply you are using what you have configured (as it is in the settings file), but in fact you will be using "pool.ntp.org", as [FONT=courier\ new]/etc/default/ntpdate[/FONT] (the file which is actually used) isn't backed-up by default so you get the system default, and this is only changed when you change the setting in enigma2 (unless you change it to "pool.ntp.org", when it won't be changed at all).
All a bit of a mess, really.

 

[Joe_90]

Whole thing is a mess at this point. I've gone back to "time by transponder" on my HD51...

 

[birdman]

Hopefully I'll write some code [STRIKE]tomorrow[/STRIKE] today and sort some of this out, or at least have some scripts with comments about what is going on....

 

[birdman]

- Add support for (and prefer) local NTP servers (e.g. own router) learned through DHCPv4 (odhcp6c does not support the ntp-servers option, so no IPv6 support yet)

There is also a DHCP option (option 152 - see RFC6926) to get the time directly from the DHCP server. Could/does the DHCP server get that?

 

[birdman]

Then there is a coding issue inside E2 itself
There is a variable m_time_ready somewhere and no time related checks should be done before it has been set <full stop>.

That issue was introduced by fake-hwclock setting an approximate time. Before that, if your system was starting from zero then [FONT=courier\ new]m_time_ready[/FONT] wouldn't have become set until a "real" time was set.

I think I now have a sysvinit script (to replace ifup links), an ntpdate-sync and a [FONT=courier\ new]NetworkTime.py[/FONT] the init script will try to set an NTP time ASAP, background the setting if that fails and the enigma2 code won't set m_time_read until a time is set (by NTP or transponder).

Untested - I may post them here later for comments while testing...

 

[birdman]

Untested - I may post them here later for comments while testing...

The first test seems to work. It uses a quick time set (-p 1) and the sysvinit script run straight after the network script succeeds with no backgrounding:

ntpdate-sync: called with
ntpdate-sync: ifup call ignored...
ntpdate-sync: called with
ntpdate-sync: ifup call ignored...
time-setter: start called
ntpdate-sync: called with -fg -q -abs
ntpdate-sync: calling ntpdate
ntpdate-sync: ntpdate succeeded
ntpdate-sync: ntpdate exit code 0

 

[twol]

The first test seems to work. It uses a quick time set (-p 1) and the sysvinit script run straight after the network script succeeds with no backgrounding:

ntpdate-sync: called with
ntpdate-sync: ifup call ignored...
ntpdate-sync: called with
ntpdate-sync: ifup call ignored...
time-setter: start called
ntpdate-sync: called with -fg -q -abs
ntpdate-sync: calling ntpdate
ntpdate-sync: ntpdate succeeded
ntpdate-sync: ntpdate exit code 0

Sounds great, just need fat-tony to test in his wifi environment

 

[Joe_90]

That issue was introduced by fake-hwclock setting an approximate time. Before that, if your system was starting from zero then [FONT=courier\ new]m_time_ready[/FONT] wouldn't have become set until a "real" time was set.

I think I now have a sysvinit script (to replace ifup links), an ntpdate-sync and a [FONT=courier\ new]NetworkTime.py[/FONT] the init script will try to set an NTP time ASAP, background the setting if that fails and the enigma2 code won't set m_time_read until a time is set (by NTP or transponder).

Untested - I may post them here later for comments while testing...

I think that (m_time_ready) was what was tripping up the time-related checks (PowerTimers, ABM timers, CrossEPG timers, AutoStandby etc. etc.) because it had a plausible time from the fake-hwclock. I knew that disabling the fake-hwclock resolved the odd behaviour of the timers, but I didn't know why at the time. I figured that the system was behaving differently when it had a zero time at startup, but didn't know why. Now I do, so thanks for that, @birdman :thumbsup:

Once you post a set of test fixes I will try them out!

 

[birdman]

OK, well here are the three files.

View attachment timeset.zip

• etc/rc3.d/S11-time-setter
  The sysvinit file that gets run after the network has been started up.
  First it deletes any /tmp/ntp_time_set (there should be none at boot up, but...)
  Tries one quick ntpdate setting. If it succeeds, OK; otherwise it submits a background task to try again after 2, 4, 8, 16, 16 and 16 seconds. If it is still failing then it gives up.
  On a successful ntpdate setting it creates [FONT=courier\ new]/tmp/ntp_time_set[/FONT].
• usr/bin/ntpdate-sync
  The front-end to ntpdate. This now takes various arguments to get it to run in the back/foreground, do a quick/default time setting and let you select servers. The defaults when run by hand from a terminal are different to those when run at boot time or by enigma2 (only because if you run it by hand you probably want it to set the absolute time).
  This does nothign if MODE
• usr/lib/enigma2/python/Components/NetworkTime.py
  This checks for the presence of [FONT=courier\ new]/tmp/ntp_time_set[/FONT], and remembers once it has seen it, before making the call that sets m_time_ready. But only if you are using NTP rather then transponder time.

The first two scripts currently log some info to [FONT=courier\ new]/var/tmp/timeset.log[/FONT] (which starts afresh on each reboot). Not much, but should allow for tracking any oddities.

To install it cd to / and unzip the file. BUT FIRST I suggest you take a copy of the current /usr/bin/ntpdate-sync and /usr/lib/enigma2/python/NetworkTime.pyo file (note the .pyo) so you can revert should you need to.

This does nothing if METHOD is set in the environment, which means you don't have to remove the ifup symlinks to it.

EDIT: Updated (including the zip attachment) to put the NetworkTime.py file under Components.

 

[Joe_90]

Thanks @birdman. I may only get to test this this afternoon as I'm out of the house most of the morning.