Tvheadend

Manuel,
The continuity errors on Movistar (Imagenio) are likely the bandwidth you have available for the channel either your LAN or WAN.
Question. If you record a channel for a few seconds does it still give you the pixelation issues ? (Open the recorded file locally from VLC and let me know)
How is your local network configured. Wired gigabit ? Any switches in-between the tvheadend machine and the router ?
Is 'p72p1' a virtual interface or a physical one ?
I'll help where I can, it's been a few years since I lived in Spain and had Imagenio but I did get it working very well.
Phill

thanks phill,
this interface is physical, a gigabit interface, wired, it is connected directly to the router, which is a gigabit router, no switch in-between!
vlc has no such problem at all, and for that matter no other app i have tested does, nor does vdr, which i have tested as well!
the wan connection is 100 megabit fibre optic!
as for the test you asked, when i record it with tvheadend and then watch it with vlc, then same pixelation issues are there!
hope this take us further, thanks again for your help!

Noticed something. Not sure who maintains the unstable Ubuntu repository for 14.04, but I've had build issues with some of the lastest gits. Build #4837 - v3.9.570 was the last one the bot was able to build. I'd revert to that and use the file from the builds section to install it.

Have you tried using the transcoding (it'll use libav) and a different container?

Really there's nothing obvious that sticks out. You could take a 10sec wireshark dump of one of the streams and post it to a bug report. It would give the devs something concrete to look at.

Phill

You could also have a look here:
https://tvheadend.org/issues/1986

I'm not sure if the code for iptv allows 'PMT monitoring' but you could defiantly try to check it's off.
Looks like there are some know issues though that adam is aware of.

Adam's point in the thread is spot on:-

-------------
Updated by Adam Sutton about 1 month ago

Comment
The problem is that there are CC errors in the stream, a CC error in simple terms means "something went missing". Basically each packet carries a 4 bit counter (unique to the PID), if you get a packet with a CC which is not exactly +1 from the last, then you've missed something and got a continuity error.

Now this can happen for several reasons, but the most common are:

1. Intentional errors. This is rare, but not unheard of for some DVB broadcasters to insert intentional errors to try and trip up unofficial receivers (though it should at worst cause unnecessary error reports, not real damage).
2. Lost in transit, due to errors etc... (i.e. receiver hardware never received the packet)

3. Lost in transfer, i.e. packet arrived, but was lost from buffers before it was passed to application (usually implies application running slowly, and my original thoughts on this issue).
4. Lost in processing, it got to the application but then mysteriously disappeared (i.e. a bug in the processing logic).

I'm ignoring 1 assuming its a non-issue. 1 is difficult to disprove, however your dump file which was generated with a completely different application shows CC errors (so I can only surmise that some genuine, transit, errors are occurring, though odd no transport errors are reported).

3 is really difficult to test without real systems on which the problems occur, however I've done quite a bit to improve this so I really hope this is gone. But you could check very latest master and look at the new CC counter field on the stream status page.

4 is probably the most likely source of errors. At the moment I'm not sure packets are going missing as such, but possibly the way in which CC errors are treated is less than ideal.

Adam
-------

Post your Debug logs.

Phill

thanks again phill,
i have tested this build in the past Build #4837 - v3.9.570, and i remember that i had to revert back to it from later bad builds. but it was all the same with iptv. i am dealing with this problem for quite some time now, longer than a month.
i also tried a transcoding build 3 weeks ago, actually not related to this bug, but in connection to a plex plugin that uses transcoding, it worked fine, but the iptv error was still there. it has been there all this time, no matter what i have tried. like for example compiling other libraries that tvheadend relies on, but the results were all the same.
i also do not know who is responsible for these ubuntu builds, but except for this bug, everything else seems to be working as expected.
here i am attaching a current debug log (started like this tvheadend -f -u hts -g video -s), hope this helps, thanks!
and a wireshark dump.

i just compiled manually latest version 3.9.722~g65a3c45, all went perfect with the compilation, but the error with iptv is still there, although i get the impression that it gets better with latest builds, but i cannot be sure because the cc errors are still there, but the picture seems to be less affected.

when the picture takes the full screen it is not ok, but the problem is visible only in the last lines of the picture, so it is mostly acceptable, the problem gets bad when the picture takes only part of the screen and the screens shows black bars, here the error also happens in the the last lines, but then as a side effect it expands over the bottom black bar and it becomes annoying, it produces sometimes a sort of reflection of the picture in the black bar, which makes this effect worst, as it sort of distorts the whole picture.

I think this is may be issue based on the log.

May 26 14:12:13 Nest tvheadend¹⁰⁶⁵: cat: invalid table length 5 min 7
May 26 14:12:13 Nest tvheadend¹⁰⁶⁵: message repeated 6 times: [ cat: invalid table length 5 min 7]

Phill

thanks phill, yes, this is the only message visible, but now what is it and how do i fix this?

it appears here https://github.com/tvheadend/tvheadend/blob/master/src/input/mpegts/dvb_psi.c (google does not show it elsewhere :-( ), but this finding is not helping me much...

When Tvheadend parses the DVB tables it complaining because the length of the CAT table isn't as it expects.

Does Movistar encrypt the iptv feed now ? Do you need a CI and Card to view it ?

Movistar never used to encrypt the stream, so I suspect that something else in the stream is making tvheadend bump the positions on the stream and decode the tables incorrectly. Either Movistar injected something into the stream or the parser isn't quite accurate. (work to investigate either way !)

Can you run a dvbsnoop and dump a cat table pid 0x01, I think you should be able pipe ncat to dvbsnoop.

Phill

gonna do that asap, as for your question, no movistar is not encrypting it, no CI in play here.
i gonna suppose something is being injected and tvheadend is reacting accordingly to it, but vlc and vdr for example do not care and the picture in this other apps is ok, i hope tvheadend can be recoded to do the same...

You can try this also:

mplayer rtp://ip:port -dumpstream -dumpfile stream.ts

dvbsnoop -s ts -if stream.ts > snoop.txt

Phill

BTW. Did you ever try tvheadend 2.13 that was the version I was using when I was in Madrid. I didn't have fibre, but it worked fine over 20mb adsl for std def channels. I think v3 was the beginning of Adam's major rewrite, so it would be good test to see if the issue was there back then.
Phill

no, never tried this old version, it crossed my mind actually, but everything else works as expected, and i am looking for a long term solution, this older version is or will be deprecated, so i never did...
here it goes, the snoop file as you suggested in the previous post using mplayer (this is fox hd channel rtp://@239.0.9.134:8208)

(oops, too big, gonna hit ctrl-c sooner in my next try)

and here it goes, these are rather big text files that dvbsnoop produces...

I've raised a defect report for the cat: Invalid Length with a suggested fix (reduce the minlength from 7 to 5). Looking at the code though it's unlikely the cause of the other continuity issues.

Did you try 2.13 ? I just want to now if something was introduced in the rewrite that could cause this.

Thx

Phill

I've been looking at this a little more this evening and reading up on udp size etc, I don't claim to be an expert but here's what I noticed. The old 2.13 code use to set the setsockopt SO_RCVBUF to 262142 bytes, for the udp receive buffer, which was basically the default linux OS default . You can check you system setting using:

/proc/sys/net/core/rmem_default
/proc/sys/net/core/rmem_max

The current code use 300*188 as the setsockopt SO_RCVBUF which is much smaller.

Now honestly I have no clue if this makes a difference, but if your amenable to compiling your own code, you could try adjusting the receive buffer and see if that makes a difference with the dropped packets.

You'll need to change iptv_udp.c IPTV_PKT_SIZE below and replace it with 262142

conn = udp_bind("iptv", name, url->host, url->port,
                  im->mm_iptv_interface, IPTV_PKT_SIZE);

Phill

thanks phill,
i am sorry, i did not try 2.13, to my knowledge this version is very different, so i did not try, i also expect that this will take me several hours and i am waiting for the right moment to do so.
i have recompiled as you suggested changing IPTV_PKT_SIZE but all remained the same, this change did not make any visible impact, the error was the same, no new error, no less errors, and the picture presented the same problem, no more, no less.
i have no idea how to play with this number, so i reverted it..

Can you run a full trace and post the log. I'm not sure what all the trace options are but something like this should be sufficient.

--trace mpegts,linuxdvb,iptv,udp,psi,pmt,cat -l tvh.log

Hopefully that works and give a full breakdown of code

Thanks

Phill

Hi Manuel,
Did you fix the issue? I have the same problem and almost same scenario. Movistar Imagenio TV + Fiber 100MB/s + Raspberry Pi with XBMC+Tvheadend. I tried with the latest Raspbmc (repository "deb http://apt.tvheadend.org/stable wheezy main") and Openelec versions. Network path is all wired like this: 100/10MB WAN <-> 100MB ROUTER <-> 1GB SWITCH <-> 100MB Raspberry Pi.

Channels play smoothly from VLC from a Macbook air and from an old Intel PC, but nothing from tvheadend. On the other hand, I have a DVB-T usb capture connected to the Raspberry Pi which works fine in all the tvheadend versions I tested.
I read something that I could be related with RTP handling in the lastest versions of tvheadend.

Any clue will be highly appreciated and some beers and good tapas are guaranteed as reward :-)

Thanks,
Miguel

In short no.
I found an issue with the cat table which I've submitted a basic pull request to try to fix. No sure that it will fix the overall problem though. You're welcome to try to compile and try.
I also asked if Manuel could take a trace log and post it so we can see more clearly where the issue might be. You could do the same. It would certainly give people more to work off, since it can't be replicated unless you have iptv (which i no longer do) its a difficult one to fix without being able to see what's going on.
Thanks
Phill

hi guys, sorry i cannot be more involved in my own issue... :-(
phill, one thing i did was to pick your patch and apply, i tested this new version for 10 minutes (sorry again i am out of time right now) but the issue was still there, the continuity counter error as well as the pixelation.
(i have to say that my problem with the pixelation at the bottom of the picture is not really that bad, only annoying sometimes)
i will follow your latest instructions as soon as i can...

Hi! Thanks for the answers. I am temporally fixing some issues with the NAT configuration since I got the IPTV vlan and routes, but it is my intention to try that new version but also prepare a spare Raspberry Pi with ssh open for the developers, so you can actually test it with IPTV. I will send the credentials to Phill by email when I get it ready.
BR

That would be great.
I remember when I had IPTV over ADSL that subscribing to more then one channel would just blow my WAN bandwidth and cause packets to drop, which obviously manifest as continuity errors. I have no idea how may channels a fibre connection can support simultaneously (HD. I'd guess is around 8Mbps so at least a couple should be possible without noticing packet issues ;-) )
Just need to be careful that other factors, like heavy network usage don't influence any testing at the time.
Phill

perexg has made some changes to the iptv code to use the udp multipacket interface.
You could compile and see if it has made any improvement to the continuity errors.
commit here: https://github.com/tvheadend/tvheadend/commit/7cc9ce7d728985fa8b168fb3f1c570098c868570
Phill