Tvheadend

I updated my fork with upstream code + some fix to overcome some problems with ts recordings recordings when using libav (autodetected). The actual code is bogus. So depending on the installation of the machine you record, you may use the (globally same old code but modified to work with new architecture) or new code using (very badly) libav.

I just changed default to do not use libav AND some fixes for mkv. The new mkv works with my panasonic viera except it does no seems to correctly do A/V synchro (for me). Must still check to see why.

So you can try my new fork (the old code is still available as a branch).

EricV

Thanks. I'm using pre-packaged tvheadend in qnap nas and prefer to do so in the future as well, so installing other or newer versions from git is not an option for me at the moment. To be honest, this all seems a bit too complicated for my understanding and time available for this.

Can you tell me what the differences are in the actual resulting ts files or how I can investigate them further? The goal here would be to run some kind of post processor command to repair the files. The other way for me is to roll back to the old version.

timo enarvi wrote:

Can you tell me what the differences are in the actual resulting ts files or how I can investigate them further? The goal here would be to run some kind of post processor command to repair the files. The other way for me is to roll back to the old version.

if you can reinstall the old version, do two small recording of the same channel, same program (I insist it should be same content as audio/video characteristic may change otherwyse) and publish it somewhere so we could look at it.

Ok, here we go. About 2 minute samples of freestyle skiing recorded from the same program just minutes apart with both setups.

Sample 1 was recorded with v3.3.392~g2dcbc36-unknown and is broken/not working
https://dl.dropbox.com/u/7384033/sample1.ts

Sample 2 was recorded with EricV version 9b71601b06 works perfectly
https://dl.dropbox.com/u/7384033/sample2.ts

tsinfo sample2.ts.mpeg
Reading from sample2.ts.mpeg
Scanning 1000 TS packets

Packet 1 is PAT
Program list:
Program 1 -> PID 0fff (4095)

Packet 2 is PMT with PID 0fff (4095)
Program 1, version 0, PCR PID 0201 (513)
Program streams:
PID 0201 ( 513) -> Stream type 02 ( 2) H.262/13818-2 video (MPEG-2) or 11172-2 constrained video
PID 0294 ( 660) -> Stream type 04 ( 4) 13818-3 audio (MPEG-2)
ES info (6 bytes): 0a 04 66 69 6e 00
Languages: fin
PID 07eb (2027) -> Stream type 06 ( 6) H.222.0/13818-1 PES private data (maybe Dolby/AC-3 in DVB)
ES info (10 bytes): 59 08 66 69 6e 10 00 01 00 32
subtitling_descriptor:
language='fin', subtitling_type=16
composition_page_id=1, ancillary_page_id=50

Found 2 PAT packets and 1 PMT packet in 1000 TS packets
valette@tri-yann3:~$ tsinfo sample1.ts.mpeg
Reading from sample1.ts.mpeg
Scanning 1000 TS packets

Packet 2 is PAT
Program list:
Program 1 -> PID 0101 (257)

Found 2 PAT packets and 0 PMT packets in 1000 TS packets

tsinfo -max 5000 sample1.ts.mpeg
Reading from sample1.ts.mpeg
Scanning 5000 TS packets

Packet 2 is PAT
Program list:
Program 1 -> PID 0101 (257)

Packet 1003 is PMT with PID 0101 (257)
Program 1, version 0, PCR PID 0201 (513)
Program streams:
PID 0201 ( 513) -> Stream type 02 ( 2) H.262/13818-2 video (MPEG-2) or 11172-2 constrained video
PID 0294 ( 660) -> Stream type 04 ( 4) 13818-3 audio (MPEG-2)
ES info (6 bytes): 0a 04 66 69 6e 00
Languages: fin
PID 07eb (2027) -> Stream type 06 ( 6) H.222.0/13818-1 PES private data (maybe Dolby/AC-3 in DVB)
ES info (10 bytes): 59 08 66 69 6e 10 00 01 00 32
subtitling_descriptor:
language='fin', subtitling_type=16
composition_page_id=1, ancillary_page_id=50

Found 10 PAT packets and 4 PMT packets in 5000 TS packets.

So ts generation indeed differs but apart that, mediainfo and tsinfo show the same things. Should look at pts/dts now but that more complicated.

I think its better if packet 1 is PAT and packet2 is PMT like in the old EricV2 branch.

Eric Valette wrote:

I think its better if packet 1 is PAT and packet2 is PMT like in the old EricV2 branch.

Yes, definitely - it makes no sense to write the PMT before the PAT. I've just pushed a fix to master that swaps the order.

I'm sure it was just a mistake that they were written PMT first.

But I find it odd that players give up looking for a PMT so early. IIRC, the specifications just say that they should appear every X seconds, not that they must be at the start of the stream, or adjacent. So PAT, then 1000 A/V packets, then the PMT should have been within specs (unless that X is very small, or the stream has a very low bitrate).

I know hardware decoders usually drops anything before finding a PAT AND a PMT packet, so recording anything before is useless. Then code has changed to let PAT packet flow through as we should have exactly the same number of PAT/PMT packets (at least this was the case in V2. Originally we generated them only once at beginning of the file and this was causing pb to IP STB that missed the first packets. We had to read them manually hence the dumb rules of each 1000 packet insertion).

Eric Valette wrote:

Found 10 PAT packets and 4 PMT packets in 5000 TS packets.

This seems to be a bug in tsinfo.

I've taken the first 5000 packets (940000 bytes) from a tvheadend passthrough recording I made last December. tsinfo is giving this exact output on this file - "Found 10 PAT packets and 4 PMT packets in 5000 TS packets."

I've then used my own TS-processing tools to actually look at the number of PAT and PMT packets amongst those 5000, and there are 5 of each.

It's understandable that tsinfo doesn't see the first PMT packet (because it comes before the first PAT packet), but I can't explain why it thinks there are 10 PAT packets in the file, there are definitely only 5!

I've applied the patch and retested, now tsinfo find only too little packets

"Found 2 PAT packets and 1 PMT packet in 754002 TS packets" (2 min recording size /188). Going home!!!

Thanks for the support.

BTW: you should add this line to the PAT/PMT packet generation code : pm->pm_pc += 2;

Eric Valette wrote:

I've applied the patch and retested, now tsinfo find only too little packets

"Found 2 PAT packets and 1 PMT packet in 754002 TS packets" (2 min recording size /188). Going home!!!

Thanks for the support.

BTW: you should add this line to the PAT/PMT packet generation code : pm->pm_pc += 2;

tested with dvdbsnoop. Indeed tsinfo is bogus but check with todays git!!!

dvbsnoop s ts -n 5005 -if JUNGLE-D_EAU-DOUCE\(HD\).2012-11-16.ts | grep 'Program Association Table'
PID: 0 (0x0000) [= ISO 13818-1 Program Association Table (PAT)]

Section table: 0 (0x00) [= Program Association Table (PAT)]

PID: 0 (0x0000) [= ISO 13818-1 Program Association Table (PAT)]
> Section table: 0 (0x00) [= Program Association Table (PAT)]
PID: 0 (0x0000) [= ISO 13818-1 Program Association Table (PAT)]

Section table: 0 (0x00) [= Program Association Table (PAT)]

PID: 0 (0x0000) [= ISO 13818-1 Program Association Table (PAT)]
> Section table: 0 (0x00) [= Program Association Table (PAT)]
PID: 0 (0x0000) [= ISO 13818-1 Program Association Table (PAT)]

Section table: 0 (0x00) [= Program Association Table (PAT)]

root@STB-atom-ion-3:/home/hts# dvbsnoop s ts -n 5005 -if JUNGLE-D_EAU-DOUCE\(HD\).2012-11-16.ts | grep 'Program MAP Table'root@STB-atom-ion-3:/home/hts# dvbsnoop s ts -n 5005 -if JUNGLE-D_EAU-DOUCE\(HD\).2012-11-16.ts | grep 'Program Map Table'
> Section table: 2 (0x02) [= Program Map Table (PMT)]

Section table: 2 (0x02) [= Program Map Table (PMT)]

Section table: 2 (0x02) [= Program Map Table (PMT)]

Section table: 2 (0x02) [= Program Map Table (PMT)]

root@STB-atom-ion-3:/home/hts# dvbsnoop s ts -n 5005 -if test1.2013-03-07.ts | grep 'Program Association Table'
PID: 0 (0x0000) [= ISO 13818-1 Program Association Table (PAT)]
> Section table: 0 (0x00) [= Program Association Table (PAT)]
root@STB-atom-ion-3:/home/hts# tsinfo -max 5005 JUNGLE-D_EAU-DOUCE\(HD\).2012-11-16.ts
Reading from JUNGLE-D_EAU-DOUCE-(HD).2012-11-16.ts
Scanning 5005 TS packets

Packet 2 is PAT
Program list:
Program 1 -> PID 02c6 (710)

Packet 1003 is PMT with PID 02c6 (710)
Program 1, version 0, PCR PID 02d0 (720)
Program streams:
PID 02db ( 731) -> Stream type 04 ( 4) 13818-3 audio (MPEG-2)
ES info (6 bytes): 0a 04 71 61 61 00
Languages: qaa
PID 02dc ( 732) -> Stream type 04 ( 4) 13818-3 audio (MPEG-2)
ES info (6 bytes): 0a 04 67 65 72 00
Languages: ger
PID 02dd ( 733) -> Stream type 04 ( 4) 13818-3 audio (MPEG-2)
ES info (6 bytes): 0a 04 71 61 61 00
Languages: qaa
PID 02d0 ( 720) -> Stream type 1b ( 27) H.264/14496-10 video (MPEG-4/AVC)
PID 02da ( 730) -> Stream type 04 ( 4) 13818-3 audio (MPEG-2)
ES info (6 bytes): 0a 04 66 72 65 00
Languages: fre
PID 02e4 ( 740) -> Stream type 06 ( 6) H.222.0/13818-1 PES private data (maybe Dolby/AC-3 in DVB)
ES info (10 bytes): 59 08 66 72 65 10 00 03 00 01
subtitling_descriptor:
language='fre', subtitling_type=16
composition_page_id=3, ancillary_page_id=1
PID 02e5 ( 741) -> Stream type 06 ( 6) H.222.0/13818-1 PES private data (maybe Dolby/AC-3 in DVB)
ES info (10 bytes): 59 08 67 65 72 10 00 01 00 01
subtitling_descriptor:
language='ger', subtitling_type=16
composition_page_id=1, ancillary_page_id=1
PID 02e6 ( 742) -> Stream type 06 ( 6) H.222.0/13818-1 PES private data (maybe Dolby/AC-3 in DVB)
ES info (10 bytes): 59 08 66 72 65 10 00 02 00 01
subtitling_descriptor:
language='fre', subtitling_type=16
composition_page_id=2, ancillary_page_id=1

Found 10 PAT packets and 4 PMT packets in 5005 TS packets

Eric,

I'm not sure what you were trying to explain with your last post, especially as the formatting appears to be broken. Can you summarise?

Dave.

Dave Chapman wrote:

Eric,

I'm not sure what you were trying to explain with your last post, especially as the formatting appears to be broken. Can you summarise?

Dave.

That
1) inded tsinfo is broken and that dvbsnoop gives the right count for PAT/PMT packets on old files
2) that todays git does not seem to generate the right packet count

I'm home rebuilding to see if
1) my tv likes the TS better,
2) I get the same behavior on my branch (uncorrect generation)

But wifes call for dinner ;-)

--eric

Dave,

I tested the fix and the TV does play the TS and has the right number of PAT/PMT packets. However I'm not 100% sure yet it works in all case as I had TS that did work and other that did not (but not sure on which channels). Will need to record more than 2 mins and test other channels.

More later.

For counting the number of PAT packets with dvbsnoop, I would use:

dvbsnoop -s ts -if recording.ts  | grep '^PID: 0 '

And for the PMT packets, search for packets with the PMT PID. e.g. if the PMT PID is 257, do:

dvbsnoop -s ts -if recording.ts  | grep '^PID: 257 '

Hi again,

I just got to try the fix and it doesn't work for me. Sorry for the time it took me, there are currently no updates on qnap because they have not published the source code for their newest firmware..

I now have 3.5.97~g4670400 installed and the same problem continues.

Anything I can do to help diagnose this further?