HD Recordings - buffer underun? [Solved]
Added by Paraic McDonagh over 5 years ago
Hi,
I just wanted to ask the community for some help. In the last few months, I've noticed that HD TV recordings have been pausing to buffer very frequently.
This is usually to the extent that the program becomes unwatchable. Sometimes when it happens the program even stops completely.
I've spent quite a bit of time investigating the root cause, but have no solution so far.
My setup is an Ubuntu ver.18.04.2 box running fully current TVHeadend Server (4.2.6-84).
The box contains a mobo with integrated AMD APU, HDMI out, USB 3 etc. The system drive is a 128 GB Samsung SSD, TV recordings and movies are stored on a 3 TB Seagate HD.
I have a LAN connected Nvidia Shield running Kodi 18.1 and the current TVHeadend client as provided with that Kodi version.
I also have a Beelink GT1 Ultimate box running CoreElec, again running the current Kodi 18.1 version. I was using the Beelink box as my main Kodi box but switched it out for the Shield to try and resolve the issue. They are obviously different platforms (LibreElec v Android), so I was surprised that this didn't help.
Things I've verified not to be the cause of the issue:
LAN speeds.
CPU Overheads.
Disk access speeds.
I've tested, tuned and verified the above individually and am sure that they are not the issue.
I have 1000 Mbps n/w speeds end to end, server cpu averages about 40% when in use, CPU on the Shield rarely goes over 10% per core etc.
But in summary, I can put a 4K movie on the Seagate drive and will play flawlessly on either Kodi box via my LAN. Which is confirmation.
Also Live TV HD channels play flawlessly, as do SD live and recorded TV, as well as 4K/HD/SD movies and all internet streaming content.
When I use system monitor on my Linux box, to view resource use at the time the issue happens, the CPU activity never exceeds 50% (On either of 2 cores). Data transmission rises just after the playback pausing event corresponding to the buffering on my client box pulling data as expected, but it's not enough to reach any kind of upper threshold.
On the Kodi box, I can view stream info (Ctrl + Shift + O) for buffer information during playback. When a HD TV recording is playing, it goes along skipping between 100%/99%. When the event happens, with no warning and apparently at random, I can see the buffer suddenly drop; E.g.: 100%...99%...76%...54%...12%...7%. Then playback will pause and a buffer progress ring will display until the buffer refills to 100%. Sometimes, as I mentioned, playback will stop completely.
Things I've tried:
I've updated TVHeadend server to the latest nightly build just yesterday (ver 4.3)
Same for Kodi clients (Now on 18.2 RC1).
I've tried a whole range of "Chunk Size" settings in TVH client settings.
Increased priority of "pass" stream to "Important" in TVH server.
Reduced other overheads at the time of playback (No active recordings, Netflix streaming, Live TV playback, Movie streaming etc.).
Nothing above has any effect. I am at a loss to identify any other TVH setting that might help.
My suspicion is that this issue has crept in with updates, probably around the time of upgrading to Kodi 18.1. But I can't be sure.
Has anyone else experienced this issue? Any help or suggestions would be appreciated. I will have minimal further time available to spend, so please understand that I might not be able to post logs in a hurry (I'm a log noob), but any and all insight would be most welcome.
Thank you!
Replies (5)
RE: HD Recordings - buffer underun? - Added by Em Smith over 5 years ago
Only other thing I can think of to try is to check on Kodi the decoder (ctrl-shift-O, or maybe just O). Is it hardware?
Perhaps also temporarily test playback with deinterlace disabled to see if it makes any difference.
Other things I can think of is for a file that had problems, you can try running "file defrag" on it and see if it makes any difference.
Final thing to try is to mount the recordings directly in Kodi (smb/nfs) and see if playback is ok through there.
RE: HD Recordings - buffer underun? - Added by Paraic McDonagh over 5 years ago
Thank you for your reply.
In player settings -> I had Allow hardware acceleration mediacodec(surface) and mediacodec selected. So hardware acceleration was on I presume.
I disabled both and checked playback with them off. A recording played smoothly for the few minutes I viewed it, but just right now I just don't have enough time available to watch a full episode to the end. I would need to watch a few hours to confirm that the issue is resolved, so I will revert back later today or tomorrow, after I have viewed some programs. I will let you know if this makes a difference. I'm curious though, which should I be using; hardware or software acceleration??
With regards interlacing, the same applies. I will revert back after some longer testing.
I made my TV Recordings folder into a samba share and can verify that all recordings play perfectly.
One major difference is that the advancedsettings.xml cache settings come into effect when doing this, so there is a generous buffer allocation.
It isn't practical for normal use though because the file ordering is not logical for determining most recent recordings etc.
I don't fully understand why cache settings have no effect on normal tvheadend recording playback. I suspect it's because tvh sets up its own transport stream server/client mechanism? But surely having an option of using standard smb file access for TV Recordings as an option, would solve a lot of these kinds of issues, since buffering on flaky networks could be managed better. Can anyone shed some light on this?
Thanks you,
Paraic.
Thanks for your assistance. I will revert back shortly with testing results.
RE: HD Recordings - buffer underun? - Added by Em Smith over 5 years ago
Yes, Kodi doesn't use the same settings for PVR, but ksoo added the "read in chunks" option to pvr.hts and it can now be set to 512KB in Kodi's pvr.hts settings. Though, it sounds like that's the one you've tried changing.
https://github.com/kodi-pvr/pvr.hts/issues/364
For decoding, I'd expect "hardware" to be better, but on my machine it was all reset to software for some reason in the Kodi playback expert settings.
The other problem I had was that although my disk was fast, it was almost full and heavily fragmented. "filefrag my-recording.ts" took a while and showed tens of thousands of fragments. If I copied the file to a new file (which defragments), and then moved it back over the top of the original file, then it would play fine. Might be worth a try.
RE: HD Recordings - buffer underun? - Added by Paraic McDonagh over 5 years ago
Thank you for your reply. Sorry for the delay getting back.
I decided that I would do a clean install of Ubuntu and then setup tvheadend from scratch.
The logic for this was that I had in the back of my mind that there were lots of added software and drivers installed over the last year or so, which might have been influencing the issue.
Anyway, it took quite a while to set up channel logos, epg, ordering of channels etc. So I have been busy!
This time, I wiped the system drive and did a very clean and minimum install. I fully expected the issue to be solved, since I had already swapped the client Kodi box out (CoreElec -> Nvidia Shield Android) and verified network throughput.
However, the issue is still there! Recorded TV programs still buffer frequently and often freeze and crash out. I thought it was HD playback only but I have also witnessed SD recordings doing this also.
I finally got around to monitoring the tvheadend server logs during playback of TV recordings. The issue seems to be related to HTSP streaming of the recordings. The error message when playback crashes is: "write error - broken pipe".
The issue I'm experiencing, looks exactly like what is being described in this post: [https://tvheadend.org/boards/5/topics/30644?r=30677]
At this stage, I have concluded that the issue must be a bug in either the tvh server or client software. But it does not seem to be something that I can resolve by adjusting settings. Any help resolving this would be appreciated. Current version of TVH Server: 4.2.8-23~g74cdf3f16
Can anyone answer this: If I rollback to an earlier version (4.2.5), can I preserve or backup all of my channel/EPG configuration so I don't have to go through all the pain of hours of channel ordering and epg linking?
RE: HD Recordings - buffer underun? - Added by Paraic McDonagh over 5 years ago
A final update on this issue. I have identified the root cause. My Seagate Drive was the culprit.
I had wiped it clean, formatted it as ext4 and de-fragmented it after accumulating a few new recordings. I also verified that playback of movies on the same drive, were smooth and did not show the symptoms that I was seeing with TVH. This was evidence enough for me and I moved on to other possible causes. However, I came back full circle, having exhausted every other avenue and decided to run some diagnostics on the drive.
I used a tool called GSmartControl as suggested here: https://help.ubuntu.com/community/Smartmontools
The drive's SMART statistics suggested that the drive performance had degraded from new. Probably surface wear and tear and mechanical inefficiency of the head.
This was not bad enough to cause data loss, but it caused the overall read performance of the drive to be reduced enough to cause the TVH stream to choke. Movies played fine because of cache settings. Probably the drive would be still okay for some other task but on the other hand, could fail imminently.
I have now replaced the drive with an identical Seagate Barracuda 3Tb one. Everything works perfectly now. No more buffering and crashing out during playback!
I just thought I'd share, in the hope that it helps someone else.