Guide listings evaporating after a few hours
Added by Phil McKerracher about 4 years ago
I've recently installed Tvheadend (HTS 4.3) on a Synology NAS (DS213j running DSM 6.2.3-25426 Update 2). It's getting its signal from an HDHomeRun Duo tuned to UK Freeview channels (from Crystal Palace, London). I'm accessing live channels mainly from Kodi installations from various devices. I can also access all the various components from a web browser on the LAN. Generally I'm really pleased with the features and design of the whole setup and I've successfully recorded and played back a few programs. But I'm tearing my remaining hair out over some niggling problems, and the loss of guide data is the worst. I don't see much discussion about this so I think I must be doing something wrong - I wonder if anyone can help please?
If I go to Tvheadend > Channel/EPG > EPG Grabber and click "Trigger OTA EPG Grabber" and then go to "Electronic Programme Guide" I gradually see the guide fill up with about 21,000 entries. Almost every channel has a "Title" and "Extra Text" for every timeslot for the next week. In the HDHomeRun interface I see both tuners connect (to different frequencies) while this is happening. In the Kodi interface if I go to TV > Guide I see the expected coloured grid of programmes with titles and descriptions.
For some reason I don't see listings for any of the HD channels but that's not a showstopper - I worked around it by mapping the corresponding SD channels to their HD counterparts.
I've had no success using the "UK Freeview" EPG source but that's a different issue.
But here's the problem - if I come back a few hours later, most (but not all) of the programme information has disappeared! The grid is still there showing programme durations, but it's just grey with hardly any titles or descriptions. Something is deleting the guide information. I have been unable to work out what, or find any setting that might control it (by trial and error or by googling). I've spent hours on this.
As far as I can tell, there's nothing wrong with the TV signal or the HDHomeRun device. A TV set connected to the same aerial happily shows all programme information including HD channels. The HDHomeRun app always shows the current programme information correctly on any channel.
Also as far as I can tell, the Kodi display always accurately mimics the guide information from Tvheadend (I've tried viewing from Kodi on a Windows PC, an Android phone and an Amazon Firestick).
So it seems to be something to do with the Synology environment. The NAS is also storing backups and video from a surveillance camera, but playback of a recorded HD TV program was pretty flawless, which was a pleasant surprise. Does anyone have any idea what might be causing this "bit rot" and what to do about it? I'm happy to post any requested logs if necessary.
Replies (9)
RE: Guide listings evaporating after a few hours - Added by saen acro about 4 years ago
Do you enable "save on disk" for epg?
RE: Guide listings evaporating after a few hours - Added by Phil McKerracher about 4 years ago
Yes, I've tried with and without, it makes no difference.
RE: Guide listings evaporating after a few hours - Added by Dave H about 4 years ago
Is there anything shown in the log (can be seen at the bottom of the TVH window or on disk) when the EPG is being scanned and/or when it disappears?
What EPG Grabbers are you running? (Config-Channel/EPG-EPG Grabber Modules?
FWIW, I have Over-the-air EIT DVB Grabber (priority 1) and Over-the-air UK Freeview (priority 5) and it all works for me.
Also, what is shown in Config-DVB Inputs-Muxes as regards scanning?
RE: Guide listings evaporating after a few hours - Added by Phil McKerracher about 4 years ago
Thanks for your help, it's greatly appreciated. I've just spent another day on this and my patience is wearing thin, but I think I've made some progress.
I've only just noticed the log window at the bottom, thanks for pointing that out. I will keep an eye on it, it will be very helpful.
Yesterday it occurred to me that the problem was most probably due to a second EPG source (from a different mux) overwriting the existing entries with blanks. So I spent a lot of time clearing everything out and manually enabling them one by one to see what was going on, but I hit so many bugs that it was difficult to conclude anything. Nevertheless I left just one EIT source on one mux enabled overnight with a complete guide populated, but this morning most of the entries had vanished as usual.
The EPG Grabbers screen is interesting because previously I haven't seen a list of alternative grabbers in this screen. At some stage today I noticed that a bunch of them had appeared, including the "UK Freeview" one I was looking for earlier. So I've enabled just that one, on a single tuner for now. So far it seems to have successfully found all the channels I'm interested in - including the HD ones. At first I thought it was missing lots of channels but after taking a break for a meal they all suddenly appeared. So maybe I'm being impatient. Also, every time I scan for services or map them to channels I see a different number of results - sometimes as low as 70 or as high as 170. It's not consistent.
As I type, a "stream" is still running in Status > Streams and a "subscription" titled "epggrab" is active. It's been over an hour since I triggered the latest scan - is this normal? When I previously used NextPVR with a TV card in my PC it would find all the muxes and EPG listings in about 10 minutes. With Tvheadend I can't get it to do a complete scan for muxes at all. I'd like to, because according to ukfree.tv there are more muxes available here than in the Tvheadend "pre-defined muxes" list for Crystal Palace - although I know the ukfree listing is itself out of date because I can get frequencies that aren't listed and some of the listed ones are dead.
RE: Guide listings evaporating after a few hours - Added by Dave H about 4 years ago
You mean the EPG Grabber Modules screen? I don't see anything like what you describe on the EPG Grabber screen.
I don't think it takes too long to complete a grab; I don't actually know. Something like w_scan should be able to tell you what's there if TVH isn't. But I have no knowledge or experience of NAS, so who knows!
RE: Guide listings evaporating after a few hours - Added by Phil McKerracher about 4 years ago
I did indeed mean the EPG Grabber Modules screen, sorry for the confusion.
To answer your question about what is shown in Config-DVB Inputs-Muxes - it currently shows Scan status "IDLE" and Scan result "OK" on half a dozen muxes. Earlier there were failures on two additional muxes (which I have removed).
The good news is this morning the guide information was still there, so the problem may be solved. I'm not sure exactly which configuration change was responsible, but the big difference is I'm now using the "UK: Freeview" grabber with all other grabber modules disabled.
Twelve additional services appeared this morning as well. It's possible that's because they simply weren't transmitting when I last initiated a scan late at night.
I suspect most of the problems I've experienced are a result of running TVH on a Synology NAS. The default timeouts are probably too short and the operating system has limitations. There are some obvious bugs like TVH immediately crashing if I try to view a channel using the "Watch TV" window, and the Start wizard never completing, so it's work in progress. Also I suspect I'm probably initiating multiple simultaneous scans sometimes (for networks, muxes, services, programmes) because often nothing at all seems to happen when I trigger them (no tuner activity, no change after refreshing screens). That can cause all sorts of confusing symptoms.
My next step is to gradually re-enable all the stuff I've disabled to try to isolate the problem.
RE: Guide listings evaporating after a few hours - Added by Phil McKerracher about 4 years ago
The problem's back! Grrr.
An update appeared for the tvheadend module in Synology and after I applied it I couldn't connect to TVH any more and a process "bzip2" consumed 100% CPU for hours. This appears to be a known problem reported 4 years ago and never fixed. https://tvheadend.org/boards/5/topics/20532
I had to uninstall and reinstall the whole module, which fixed the problem but wiped all my settings (and recordings). This is unfortunate!
I eventually discovered how to enable the UK Freeview EPG module. You have to go to Configuration > General > Base > Web Interface Settings > Default view level and set it to Advanced and click "Save". Then if you go to Configuration > Channel/EPG a new menu appears called "EPG Grabber Modules" (simply selecting View Level: Advanced here isn't enough, which is a bug). You can then enable Over the air: UK Freeview. Until you do all that, the option "UK Freeview" can be selected in Configuration > DVB Inputs > Muxes but will have no effect. Another bug, I think.
Having done all that again, the program listings are back and have survived the night EXCEPT for all the HD channels (and a few others). So I'm still trying to work out what I did last time to get those to appear.
RE: Guide listings evaporating after a few hours - Added by Phil McKerracher about 4 years ago
Update: I found that the button at Configuration > Channel/EPG > EPG Grabber > Trigger OTA EPG Grabber wasn't doing anything (no activity in Status > Stream). But if I go to Configuratio > DVB Inputs > Muxes and manually change each one from IDLE to ACTIVE and click Save then some more channels are found. (I did change some other settings as well so not 100% sure this is the culprit.)
Now I have 118 channels including HD channels and all the ones I really need.
The HDHomeRun device finds 159 channels with a single button click, so I'm still missing quite a few. Why does it have to be so hard to do the same in TVH?
RE: Guide listings evaporating after a few hours - Added by Phil McKerracher about 4 years ago
Another update: I found some more channels - I'm now up to 130. In case it helps anyone else in the UK, the "Pre-defined MUXes" for London in Tvheadend seem to be out of date (and I couldn't get the "--Generic--" scans to work, they just hang). Also the list at https://ukfree.tv/transmitters/tv/Crystal_Palace seems to be out of date as well (not TVH's fault, obviously).
However I did find a list for manual tuning that seems more accurate here https://www.freeview.co.uk/help/coverage-checker/detailed-view. It says the Crystal Palace channel numbers (both now and after 2025) are:
23 26 30- 25 22 28- 55
Unfortunately Tvheaded only knows about frequencies but there's a conversion table from channel numbers to frequencies here for the UK that seems to work (albeit with no explanation of the minus signs)
https://www.digitalspy.com/tech/terrestrial/a12613/uhf-channel-and-frequency-guide/