Tvheadend

the ts file does have audio and video so the stream is understanded by tvheadend.
The main problem is when recording those streams, then it skips the complete audio stream.

below the log from tvheadend:
Logfile from tvheadend displays:
Oct 23 20:49:55 backend tvheadend16807: dvr: # type lang resolution samplerate channels
Oct 23 20:49:55 backend tvheadend16807: dvr: 1 H264 1920 x 1088
Oct 23 20:49:55 backend tvheadend16807: dvr: 2 AC3 dut 96000 0 <disabled, no valid input>
Oct 23 20:49:55 backend tvheadend16807: dvr: 3 TELETEXT
Oct 23 20:49:55 backend tvheadend16807: dvr: 4 CA

When watching this stream using mplayer i do have audio and video:
mplayer http://192.168.10.10:9981/playlist/channelid/44
VIDEO H264 AUDIO A52 NO SUBS (yet)! PROGRAM N. 1
Stream not seekable!
FPS seems to be: 25.000000 ==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264) ========================================================================== ==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
AUDIO: 48000 Hz, 2 ch, s16le, 384.0 kbit/25.00% (ratio: 48000->192000)
Selected audio codec: [ffac3] afm: ffmpeg (FFmpeg AC-3) ==========================================================================
AO: [pulse] 48000Hz 2ch s16le (2 bytes per sample)

William van de Velde

William Velde wrote:

Hello,

attached is a ts file which uses an audio codec which is unusual but mediainfo does recognize the audio stream.

(thank you eric)
tsinfo /tmp/test.ts
Reading from /tmp/test.ts
Scanning 1000 TS packets

Packet 1 is PAT
Program list:
Program 1 -> PID 0fff (4095)

Packet 2 is PMT with PID 0fff (4095)
Program 1, version 0, PCR PID 0201 (513)
Program streams:
PID 0201 ( 513) -> Stream type 1b ( 27) H.264/14496-10 video (MPEG-4/AVC)
PID 0065 ( 101) -> Stream type 81 (129) User private
ES info (9 bytes): 0a 04 64 75 74 00 6a 01 00
Languages: dut
DVB AC-3 (1 byte): 00

Found 2 PAT packets and 1 PMT packet in 1000 TS packets

mediainfo /tmp/test.ts
General
ID : 1 (0x1)
Complete name : /tmp/test.ts
Format : MPEG-TS
File size : 5.72 MiB
Duration : 4s 454ms
Overall bit rate : 10.7 Mbps

Video
ID : 513 (0x201)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.0
Format settings, CABAC : Yes
Format settings, ReFrames : 3 frames
Codec ID : 27
Duration : 4s 680ms
Bit rate : 9 811 Kbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.189
Stream size : 5.47 MiB (96%)
Color primaries : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
Transfer characteristics : BT.709-5, BT.1361
Matrix coefficients : BT.709-5, BT.1361, IEC 61966-2-4 709, SMPTE RP177

Audio
ID : 101 (0x65)
Menu ID : 1 (0x1)
Format : AC-3
Format/Info : Audio Coding 3
Mode extension : CM (complete main)
Codec ID : 129
Duration : 4s 416ms
Bit rate mode : Constant
Bit rate : 384 Kbps
Channel(s) : 6 channels
Channel positions : Front: L C R, Side: L R, LFE
Sampling rate : 48.0 KHz
Bit depth : 16 bits
Compression mode : Lossy
Delay relative to video : -463ms
Stream size : 207 KiB (4%)
Language : Dutch

So we have found two different kind of bugs leading to invalid/nonexisting audio. For bug 321, its the ts parsing (psi.c) that does not detect the EAC3 audio. I take the opportunity to repeat here that this bug can be fixed by applying part of the TS patch. See bug for the patch.

Then it can be removed while doing the mkv muxing, like for this example with the error : AC3 dut 96000 0 <disabled, no valid input>.

For ts scanning, tsinfo is well suited.

tsinfo -max 5000 /tmp/tmp.mpg
Reading from /tmp/tmp.mpg
Scanning 5000 TS packets

Packet 856 is PAT
Program list:
Program 70 -> PID 0028 (40)

Packet 1058 is PMT with PID 0028 (40)
Program 70, version 9, PCR PID 04b0 (1200)
Program streams:
PID 012d ( 301) -> Stream type 06 ( 6) H.222.0/13818-1 PES private data (maybe Dolby/AC-3 in DVB)
ES info (7 bytes): 56 05 65 6e 67 09 00
Teletext: language=eng, type=Initial, magazine 1, page 0
PID 04b0 (1200) -> Stream type 02 ( 2) H.262/13818-2 video (MPEG-2) or 11172-2 constrained video
PID 04b1 (1201) -> Stream type 04 ( 4) 13818-3 audio (MPEG-2)
ES info (6 bytes): 0a 04 64 61 6e 00
Languages: dan

Found 12 PAT packets and 6 PMT packets in 5000 TS packets

After submitting the comment above I realized that the difference between the two channels is the TELETEXT.

My fault was that I forgot that the version of tvheadend I use integrates a patch that delivers raw teletext to clients:
https://github.com/mikrohard/tvheadend/commit/3b407aa2053b1db3316873acd05c64319676eb34 (https://github.com/andoma/tvheadend/pull/44)

Looking at all differences between my fork and the master of andoma I reverted the patch mentioned above and tested the recording on the same channels. The result was positive: working audio in the recordings.

Just for reference, the only differences between the working version of mine and andomas master are upstart jobs instead of init based one and one additional changeset from Jernej Fijačko solving a possible memory leak in htsp:
https://github.com/joergschiller/tvheadend/compare/andoma:master...joergschiller:testing_on_sound_issue

I should mention that recording with the teletext patch almost all channels can be recorded with sound. I only found a few.

It looks like the order of the TELETEXT stream is a problem when the teletext patch is in use.

Witouth teletext patch

ARD

Nov 13 23:04:29 tv tvheadend[19096]: dvr:  # type                 lang resolution       samplerate channels  
Nov 13 23:04:29 tv tvheadend[19096]: dvr:  1 H264                      1280 x 720                             
Nov 13 23:04:29 tv tvheadend[19096]: dvr:  2 MPEG2AUDIO           deu                   48000      2          
Nov 13 23:04:29 tv tvheadend[19096]: dvr:  3 MPEG2AUDIO           mis                   48000      2          
Nov 13 23:04:29 tv tvheadend[19096]: dvr:  4 AC3                  deu                   48000      2          
Nov 13 23:04:29 tv tvheadend[19096]: dvr:  5 TELETEXT             ger                                         
Nov 13 23:04:29 tv tvheadend[19096]: dvr:  6 DVBSUB               deu                                         

QVC

Nov 13 23:05:21 tv tvheadend[19096]: dvr:  1 TELETEXT                                                         
Nov 13 23:05:21 tv tvheadend[19096]: dvr:  2 H264                      1920 x 1088                            
Nov 13 23:05:21 tv tvheadend[19096]: dvr:  3 AC3                  deu                   48000      2          

ANIXE

Nov 13 23:06:06 tv tvheadend[19096]: dvr:  # type                 lang resolution       samplerate channels  
Nov 13 23:06:06 tv tvheadend[19096]: dvr:  1 H264                      1280 x 1088                            
Nov 13 23:06:06 tv tvheadend[19096]: dvr:  2 AC3                  deu                   48000      2          

With teletext patch

ARD

Nov 13 23:07:29 tv tvheadend[7549]: dvr:  1 H264                      1280 x 720                             
Nov 13 23:07:29 tv tvheadend[7549]: dvr:  2 MPEG2AUDIO           deu                   48000      2          
Nov 13 23:07:29 tv tvheadend[7549]: dvr:  3 MPEG2AUDIO           mis                   48000      2          
Nov 13 23:07:29 tv tvheadend[7549]: dvr:  4 AC3                  deu                   48000      2          
Nov 13 23:07:29 tv tvheadend[7549]: dvr:  5 TELETEXT             ger                                         
Nov 13 23:07:29 tv tvheadend[7549]: dvr:  6 DVBSUB               deu                                         

QVC

Nov 13 23:08:02 tv tvheadend[7549]: dvr:  1 TELETEXT                                                         
Nov 13 23:08:02 tv tvheadend[7549]: dvr:  2 H264                      1920 x 1088                            
Nov 13 23:08:02 tv tvheadend[7549]: dvr:  3 AC3                  deu                   96000      0          <disabled, no valid input>

ANIXE

Nov 13 23:08:31 tv tvheadend[7549]: dvr:  # type                 lang resolution       samplerate channels  
Nov 13 23:08:31 tv tvheadend[7549]: dvr:  1 H264                      1280 x 1088                            
Nov 13 23:08:31 tv tvheadend[7549]: dvr:  2 AC3                  deu                   48000      2          

For me there is no bug in current master of tvheadend. I will comment on the pull request because this seems to be introduced by the teletext patch.

i can confirm i'm using the teletext patch as wel.
never thought about to test with the master version.

With kind regards
William

Could someone please make a raw TS dump of the stream (with VLC). It should be at least a minute of the stream so that I can look into the problem.

Thanks for your input,
Jernej

Hello Jernej,

i have created a ts dump with the write full stream to disk option enabled using mplayer -dumpstream.
i have uploaded the files (one recording with the no valid input and the mplayer-streamdump.ts file) to www.cobradevil.org/downloads/tvheadend/

The files are about 42 MB and is now uploading so it can take 15 minutes more before the files are all there, slow uplink.
Is that enough or do you need more information?

With kind regards,

William

I think it's enough... I'll look into it tomorrow.

Regards,
Jernej

Hello William,

I tried your stream dump with tvheadend teletext patch and the sound was recorded without a problem. But I noticed that there is no teletext elementary stream inside your ts dump. Does the problem also appear on channels without teletext?

If mplayer dump maybe ignored teletext you could try VLC (it dumps teletext for sure).

Regards,
Jernej

Hello Jernej,

arg....

i have tried with vlc but even then i don't see the teletext stream ?
but i am now uploading a raw mux dump which does contain the teletext info, using mediainfo.

Text #1
ID : 32 (0x20)-100
Menu ID : 2004 (0x7D4)
Format : Teletext
Language : Dutch

can you take a look and tell me if that is the right file?
still busy uploading now.

Could it be that the teletext stream needs identification in the mkmux.c file?
See commit for EAC3: https://github.com/andoma/tvheadend/commit/ce549c6a5488c923e7d3448ed07ccd8e625f138b

With kind regards,

william

The main problem (as I see it right now) is the audio stream being flagged as "not valid" which is done in "src/plumbing/globalheaders.c".

      if(threshold) {
    ssc->ssc_disabled = 1;
      } else {
    return 0;
      }

Maybe the teletext stream somehow interferes with the headers creation... but thats just a guess for now... I need a working sample to take a look.

i send you a pm on github.

i can provide access to my tvheadend box.
And i also uploaded the file to: cobradevil.org/downloads/tvheadend/

With kind regards

William

At the moment I only have two raw dumps taken with tvheadends builtin "Write full DVB MUX to disk" function. They are ~30s long.

No sound when recording:
http://dl.dropbox.com/u/15769255/_dev_dvb_adapter0_STB0899_Multistandard10773250_H.dump1_no_sound_when_recording.ts

Working sound when recording:
http://dl.dropbox.com/u/15769255/_dev_dvb_adapter0_STB0899_Multistandard10773250_H.dump1_working_sound_when_recording.ts

Hope this can be a starting point. I'll try to dump even longer streams later.

Greetings
Joerg

PS: This is not the channel I wan't to record but unfortunately the only one I can reproduce the error :)

Obviously I misunderstood the concept of mux dumps. The dump includes all channels on the transponder.

After starting tvheadend with the dump as input the problematic channel is presented as "rawts_526f". Trying to start a recording on this results in disabled audio channel. Without the teletext patch it works as expected.

./tvheadend -r ~/tmp/tv/_dev_dvb_adapter0_STB0899_Multistandard10773250_H.dump1_no_sound_when_recording.ts

Nov 15 13:56:39 subscription: "DVR: test" subscribing on "rawts_526f", weight: 300, adapter: "", network: "", mux: "", provider: "", service: "", quality: 100
Nov 15 13:56:40 dvr: test from adapter: "", network: "", mux: "", provider: "", service: "" 
Nov 15 13:56:40 dvr:  # type                 lang resolution       samplerate channels  
Nov 15 13:56:40 dvr:  1 PMT                                                              
Nov 15 13:56:40 dvr:  2 TELETEXT                                                         
Nov 15 13:56:40 dvr:  3 H264                      1920 x 1088                            
Nov 15 13:56:40 dvr:  4 AC3                  deu                   96000      0

For reference: in the same dump channel "rawts_526c" works but it doesn't have teletext or at least tvheadend doesn't recognize it.

Nov 15 14:02:37 subscription: "DVR: test" subscribing on "rawts_526c", weight: 300, adapter: "", network: "", mux: "", provider: "", service: "", quality: 100
Nov 15 14:02:39 dvr: test from adapter: "", network: "", mux: "", provider: "", service: "" 
Nov 15 14:02:39 dvr:  # type                 lang resolution       samplerate channels  
Nov 15 14:02:39 dvr:  1 PMT                                                              
Nov 15 14:02:39 dvr:  2 H264                      1280 x 1088                            
Nov 15 14:02:39 dvr:  3 AC3                  deu                   48000      2

I would like to add that I can successfully record audio on other channels with teletext as long the teletext comes after the video/audio in the list tvheadend presents (please see https://www.lonelycoder.com/redmine/issues/754#note-7).

Thanks Joerg! I'm able to reproduce the problem with your ts dump. I'll try to find and fix the problem now...

Well, some data of the teletext feed can be recorded into the mkv ... Tvheadend converts the teletext subtitles to an ascii feed in the mkv ...

So make sure this does not break. But good to know that at least it is starting to become more clear what is happening.

Hein

Don't worry... converted teletext subtitles are presented as "SCT_TEXTSUB"... this patch skips just "SCT_TELETEXT" which is raw teletext data.

Hello Jernej,

i can confirm that this patch fixes the recording.

but after applying the raw ts dump i also get an invalid mkv with no audio.
Will post that in the pull request from Marc.

Thank you for looking into this.

William

I don't mean to hijack this thread, but I have what appears to be a similar issue where tvheadend is incorrectly parsing the audio and as a result is not muxing any audio into the resulting mkv file. I originally logged this as bug #278 about a year ago, with no progress to date. I'd originally reported as a muxing issue with the mpeg2audio format, but I now believe that tvheadend is mis-identifying the audio - it's really AC-3 audio (this is an example of the stream as recorded using azap/cat and using "mediainfo" to show the formats):

mediainfo mpeg2audio.ts
General
Complete name : mpeg2audio.ts
Format : MPEG-TS
Format profile : No PAT/PMT
File size : 26.0 MiB
Duration : 1mn 3s
Overall bit rate mode : Variable
Overall bit rate : 3 448 Kbps

Video
ID : 81 (0x51)
Format : MPEG Video
Format version : Version 2
Format profile : Main@Main
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : M=3, N=45
Duration : 1mn 2s
Bit rate mode : Variable
Bit rate : 3 084 Kbps
Maximum bit rate : 15.0 Mbps
Width : 704 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate : 29.970 fps
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.305
Stream size : 22.9 MiB (88%)

Audio
ID : 84 (0x54)
Format : AC-3
Format/Info : Audio Coding 3
Mode extension : CM (complete main)
Duration : 1mn 3s
Bit rate mode : Constant
Bit rate : 192 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Bit depth : 16 bits
Compression mode : Lossy
Delay relative to video : -69ms
Stream size : 1.45 MiB (6%)

I'd be happy to supply whatever information, files, etc. are needed to help track this down - I just need someone to step up and agree to look at them.

Thanks!

Reading the TS patch, I think only adding eAC3 support can cause the problem. I find a bug in the EAC3 support patch. Could you try to apply this patch on top of the ps patch

https://github.com/EricV/tvheadend/commit/09014fe0a2896c45b8d4f1f3e26bf38eb120725e

Eric,

I'm not sure if you were addressing me with your latest comment or not, but I had no change in operation after applying the patch you referenced to the current trunk. I'd previously applied the patch from bug #321, I'm hoping that's what you meant by "the ps patch".

I made a commit to update my pull request. I also added two lines to remove a memory leak and completely disable recording of raw teletext data.

https://github.com/mikrohard/tvheadend/commit/a8026cd5b412c5096b0904f060ad5ffccdb883cf

For anyone interested i found a way to decrypt my raw muxdump.

http://cobradevil.org/index.php?title=Setup_tsdecrypt

Remember to do this a quick as possible after you have created to dump. This because some providers have a very short time period where valid codewords can be obtained from the smartcard.

I think i have this same error

Im not raport this before, because im wait for some fix but after half year nothing changed in this topic :(

On some channels is some problem with AC3 (a52) codec, over VLC stream i dont hear sound,
im record pass-throught but still even if i see audio i cant play :(

I check channels parametr, all pids are actual, all audio tract are visible

And is one more problem
On this 2 channels - when i try change audio track on XBMC - then all drope frames, sometime crash etc.

This is example: recorded file
https://dl.dropbox.com/u/49442039/Rio%20Ogl%C4%85daj%20w%20telewizji.2012-12-27.ts

For second audio i dont see size of track :( (checked by media info)
@
Audio #1
ID : 636 (0x27C)
Menu ID : 1 (0x1)
Format : AC-3
Format/Info : Audio Coding 3
Codec ID : 129
Duration : 2mn 41s
Compression mode : Lossy
Language : Polish

Audio #2
ID : 736 (0x2E0)
Menu ID : 1 (0x1)
Format : AC-3
Format/Info : Audio Coding 3
Mode extension : CM (complete main)
Format settings, Endianness : Big
Codec ID : 129
Duration : 2mn 41s
Bit rate mode : Constant
Bit rate : 384 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Bit depth : 16 bits
Compression mode : Lossy
Stream size : 7.41 MiB (7%)@