Tvheadend

You probably mean RTP/UDP protocol, for the standard UDP packets, we cannot do much. And yes, this layer is missing in src/input/mpegts/iptv/iptv_udp.c for the IPTV input. You may try to look to src/input/mpegts/satip/satip_frontend.c - satip_frontend_rtp_decode() - sf_udp_packets for the simple implementation. Ideally, this code should be moved to a common place and reused.

This is MPEG-TS over UDP multicast, not RTP (common for IPTV over fiber around here).

If it's just a plain UDP packet, then there's no sequence/index information, so you cannot recover the original TS packet sequence. See: https://cs.wikipedia.org/wiki/User_Datagram_Protocol

The RTP has an extra sequence number, so you can recover the order using this information.

I don't think that's true. The TS packets have a continuity counter (which TVH also complains about when packet reordering occurs). Wireshark is also able to detect reordering directly from the stream. We could use the countinuity counter to implement a reorder buffer.

I agree that this should go straight into the input code. For packet loss (which can and will happen as you noted): We need to define some reasonable timeframe during which we reorder packets and consider them lost afterwards. I'd propose 100ms as a default since there is pretty much no provider which runs networks with more than 100ms latency and the same multicast feeder and it's still not long enough to be a nuisance.

100ms is very low threshold - there might be some scheduling / buffering gaps in the kernel code and we don't know the exact receive time at the physical level. I think that the code might be designed without time constraints...

100ms should be fine, TVH should run with SCHED_RR on high-density systems regardless. But that doesn't matter yet anyways. I looked into the code a bit more, but I'm not sure if iptv_udp.c is a good place to put the buffer handling. I think I found you on IRC (perexg, right?), are you available to quickly discuss some of these things?

Jaroslav Kysela wrote:

Ok, I see your point now. I believe that this recovery should go to the input code (to work with whole UDP packets not separate TS packets) - src/input/mpegts/iptv/iptv_udp.c - iptv_udp_read() function. Also, we should keep in mind that the packets might be lost (unrecoverable CC error).

Hi Jaroslav,

The packet reordering it's a huge problem with UDP and RTP. I feel the person how has created this issue ask about a processing for reorder plain UDP streams. So here is my suggestion:

- When receiving a plain UDP TS, and a pid with audio or video exists (for the rest of pids, it's quite difficult to achieve), then you can save all the last 16 CC received counters of this pid. Then you buffer the entire input UDP packets for a short time (my preference is a range between 50-100ms). The list of 16 CC counters will point to the UDP packets in the input buffer. And then the algoritm is quite simple: for each new TS packet for this pid, update the list; then iterate for the list and search for "reorders". If a reoder is detected then reorder the entire UDP incoming packet. It's important that if a packet is missed, then the reordeing detection will fail, and no action is done in this case.

What you think about this proposal?

Mono Polimorph wrote:

Jaroslav Kysela wrote:

Ok, I see your point now. I believe that this recovery should go to the input code (to work with whole UDP packets not separate TS packets) - src/input/mpegts/iptv/iptv_udp.c - iptv_udp_read() function. Also, we should keep in mind that the packets might be lost (unrecoverable CC error).

Hi Jaroslav,

The packet reordering it's a huge problem with UDP and RTP. I feel the person how has created this issue ask about a processing for reorder plain UDP streams. So here is my suggestion:

- When receiving a plain UDP TS, and a pid with audio or video exists (for the rest of pids, it's quite difficult to achieve), then you can save all the last 16 CC received counters of this pid. Then you buffer the entire input UDP packets for a short time (my preference is a range between 50-100ms). The list of 16 CC counters will point to the UDP packets in the input buffer. And then the algoritm is quite simple: for each new TS packet for this pid, update the list; then iterate for the list and search for "reorders". If a reoder is detected then reorder the entire UDP incoming packet. It's important that if a packet is missed, then the reordeing detection will fail, and no action is done in this case.

What you think about this proposal?

Yeah, I think this is the way to go. I've looked into it some more and it should be fairly straightforward to do inside iptv_udp.c. I switched over to a capture card, so this is no longer a big problem for me, but I still have the original streams to test against and will probably eventually implement it. I'm currently very busy though.

I had the same problem with my TS streams over udp, when the network got congested i got a lot of TS corruption. I fixed the problem by using udpxy, i rarerly get stream errors now

I have the same problem with my operator's UDP transmissions a fix would be great.

On the other hand, I am also interested in reordening DVB transmissions packets eg. with a certain buffer? (in my case about 3 seconds fault caused by unordered video packets), I have some transmissions whose packages are mixed for 3 seconds (every 40 seconds or so), would it be possible to solve (reorder) this too?