Tvheadend

The teletext stream is always skipped in all versions of TVH for mkv. The recent versions just ignore the teletext stream changes and does not restart muxers. You should specify / identify more the reason, why players does not like the resulting mkv. I suspect that there are audio changes (like AC3 2.0 to AC3 5.1 etc.).

Jaroslav Kysela wrote:

The teletext stream is always skipped in all versions of TVH for mkv. The recent versions just ignore the teletext stream changes and does not restart muxers. You should specify / identify more the reason, why players does not like the resulting mkv. I suspect that there are audio changes (like AC3 2.0 to AC3 5.1 etc.).

Well, I'm sorry (and embarrassed) to correct my original report, there are NO dvb_teletext streams involved, whatsoever.

I meant dvb_subtitle ! DVB teletext was just a typo.

The recent TVHeadend versions mark "Chapters" in the beginning of the program and after, I suppose there is some stream configuration changes that cause these.
And these chapter-shifts are the places where the video hangs.

How do you suggest I investigate the streams in different timeframes? I'm not that good with ffprobe, but I can get a .ts file from a problematic recording, for example?

Here's an example of a recording with the old TVheadend (to mkv), it produces three files, the first padding file, the large actual program, and the the padding file after the program.

First padding file (ffprobe output, just the streams):

Duration: 00:01:38.17, start: 0.000000, bitrate: 7556 kb/s
    Stream #0:0(eng): Video: h264 (Main), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
    Stream #0:1(fin): Audio: ac3, 48000 Hz, stereo, fltp, 448 kb/s (default)
    Stream #0:2(fin): Subtitle: dvb_subtitle (default)
    Stream #0:3(fin): Subtitle: dvb_subtitle (default)

And the actual program:

Duration: 00:55:00.73, start: 0.000000, bitrate: 7756 kb/s
    Stream #0:0(eng): Video: h264 (Main), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
    Stream #0:1(eng): Audio: ac3, 48000 Hz, stereo, fltp, 448 kb/s (default)
    Stream #0:2(fin): Subtitle: dvb_subtitle (default)

So there seems to be profile changes in both audio and dvb_subtitle streams. So yes, sure, the problem may be either audio or subtitle stream (or something else).

There is no problem is when you record, there is a when player plays stream with change it content aka added/removed streams
ex.
added/removed subtitles/audio.

Problem is with players
they read first situation where have situation "x" but after time situation is "y" player stay in situation "x"

example
recording start with latest seconds of commercials before movie
commercials have only video and audio
recording go and start event
with contain new list of streams
2 audio streams 3 subtitles streams
but player still thing that situation is on beginning when commercials finishing

funny example
Relay race
at finish is expecting first runner

other problem MKV container do not support stream start after starting with TS support

Solution is to CUT "bad beginning",
or some how if possible, developers to make recording to start in moment,
when stream pids change it counts of contents.

Yes I agree that this might be more or less the problem of player software, since they can't handle the alternating substreams.

For me the simplest solution would be if I could tick a box in Recording-properties of TVH that would cause the recording to restart in a new file every time the stream properties change ("Restart recording with mux changes", for example). I already have a script that chooses the right file and discards the padding files, and it has worked thus far. The way it records everything in a single mkv seems to be a bit complicated.

So maybe this bug report can be interpreted as a feature suggestion.

There seems to be a little issue with the changed/reconfigured streams in recent tvh. Could you try this patch?

iff --git a/src/dvr/dvr_rec.c b/src/dvr/dvr_rec.c
index 3a289615a..d98750fd0 100644
--- a/src/dvr/dvr_rec.c
+++ b/src/dvr/dvr_rec.c
@@ -1457,6 +1457,7 @@ dvr_thread(void *aux)
     case SMT_STOP:
        if (sm->sm_code == SM_CODE_SOURCE_RECONFIGURED) {
         // Subscription is restarting, wait for SMT_START
+        muxing = 0;

        } else if(sm->sm_code == 0) {
         // Recording is completed

Jaroslav Kysela wrote:

There seems to be a little issue with the changed/reconfigured streams in recent tvh. Could you try this patch?

[...]

Sorry for the reply taking so long. I'm using debian/ubuntu tvh-packages normally, so compiling the test build needed some preparation. I did a fresh Debian Stretch 9.1 installation, installed the needed packages (and firmware for my Hauppauge SoloHD adapters (x2))and got tvheadend's master build running, with my DVB-tuners working. Pretty much default settings, just mapped all free channels that my DVB-C can find.

Now I can't test if the suggested patch (above) works or not, since there seems to be an unrelated issue with the current master. I can record with "pass" container .ts-files normally, BUT, with matroska, tvh crashes instantly if I either start a recording manually from epg or a programmed recording tries to start. This is the info tvheadend bursts out before quitting:

2017-08-28 11:27:30.001 [ INFO] dvr: "Weather" on "France24" recorder starting
2017-08-28 11:27:30.002 [ ALERT] CRASH: Signal: 11 in PRG: ./build.linux/tvheadend (4.3-362~g2a5aab6b9) [08affb2b2eda86262ab4c3921b524b3b370c6dcd] CWD: /home/hosode/tvheadend
2017-08-28 11:27:30.003 [ ALERT] CRASH: Fault address (nil) (Address not mapped)
2017-08-28 11:27:30.004 [ ALERT] CRASH: Loaded libraries: linux-vdso.so.1 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 /lib/x86_64-linux-gnu/libz.so.1 /usr/lib/x86_64-linux-gnu/libavahi-common.so.3 /usr/lib/x86_64-linux-gnu/libavahi-client.so.3 /lib/x86_64-linux-gnu/libdbus-1.so.3 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/librt.so.1 /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /lib/x86_64-linux-gnu/libmvec.so.1 /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libsystemd.so.0 /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libgcc_s.so.1 /lib/x86_64-linux-gnu/libselinux.so.1 /lib/x86_64-linux-gnu/liblzma.so.5 /usr/lib/x86_64-linux-gnu/liblz4.so.1 /lib/x86_64-linux-gnu/libgcrypt.so.20 /lib/x86_64-linux-gnu/libpcre.so.3 /lib/x86_64-linux-gnu/libgpg-error.so.0
2017-08-28 11:27:30.004 [ ALERT] CRASH: Register dump [23]: 00007f55536d9d2800000000000001b000005649359baea000007f555349d244000000000000000100000000000000000000564938cc231000000000000000010000000000000000000056493664137e00007ffdc0317b100000564938cf9a20000000000000006900000000000000000000564938cf9bc000007ffdc0317ab800007f55533d882d0000000000010297002b0000000000330000000000000004000000000000000efffffffe7ffbba150000000000000000
2017-08-28 11:27:30.004 [ ALERT] CRASH: STACKTRACE
2017-08-28 11:27:30.027 [ ALERT] CRASH: /home/hosode/tvheadend/src/trap.c:148 0x56493598165a 0x564935784000
2017-08-28 11:27:30.044 [ ALERT] CRASH: ??:0 0x7f55541a80c0 0x7f5554197000
2017-08-28 11:27:30.062 [ ALERT] CRASH: ??:0 0x7f55533d882d 0x7f5553340000
2017-08-28 11:27:30.076 [ ALERT] CRASH: /home/hosode/tvheadend/src/muxer/muxer_mkv.c:1547 0x5649359dba9e 0x564935784000
2017-08-28 11:27:30.089 [ ALERT] CRASH: /home/hosode/tvheadend/src/muxer.c:324 0x5649359d73eb 0x564935784000
2017-08-28 11:27:30.102 [ ALERT] CRASH: /home/hosode/tvheadend/src/profile.c:1643 0x56493599a975 0x564935784000
2017-08-28 11:27:30.114 [ ALERT] CRASH: /home/hosode/tvheadend/src/dvr/dvr_rec.c:117 0x5649359c5cb1 0x564935784000
2017-08-28 11:27:30.128 [ ALERT] CRASH: /home/hosode/tvheadend/src/dvr/dvr_db.c:2174 0x5649359c0aff 0x564935784000
2017-08-28 11:27:30.140 [ ALERT] CRASH: /home/hosode/tvheadend/src/main.c:705 0x56493593b884 0x564935784000
2017-08-28 11:27:30.140 [ ALERT] CRASH: __libc_start_main+0xf1 (/lib/x86_64-linux-gnu/libc.so.6)
Segmentation fault
root@debian:/home/hosode/tvheadend#

With "pass", tvh outputs as expected:

2017-08-28 11:40:28.211 [ INFO] dvr: entry 8de85f0d06cca382e95f61c23962f095 "Formula 1: Belgian osakilpailu" on "MTV3" starting at 2017-08-28 10:58:30, scheduled for recording by "hosode"
2017-08-28 11:40:28.212 [ INFO] dvr: "Formula 1: Belgian osakilpailu" on "MTV3" recorder starting
2017-08-28 11:40:28.212 [ INFO] subscription: 0014: "DVR: Formula 1: Belgian osakilpailu" subscribing on channel "MTV3", weight: 500, adapter: "Silicon Labs Si2168 #0 : DVB-C #0", network: "tkudvbc", mux: "242MHz", provider: "MTV Oy", service: "MTV3", profile="pass"
2017-08-28 11:40:28.215 [ INFO] dvr: /home/hosode/Formula 1: Belgian osakilpailu.ts from adapter: "Silicon Labs Si2168 #0 : DVB-C #0", network: "tkudvbc", mux: "242MHz", provider: "MTV Oy", service: "MTV3"
2017-08-28 11:40:28.215 [ INFO] dvr: # type lang resolution aspect ratio sample rate channels
2017-08-28 11:40:28.215 [ INFO] dvr: 1 MPEG2VIDEO ? ?
2017-08-28 11:40:28.216 [ INFO] dvr: 2 MPEG2AUDIO fin ? ?
2017-08-28 11:40:28.216 [ INFO] dvr: 3 TELETEXT
2017-08-28 11:40:28.216 [ INFO] dvr: 4 DVBSUB fin
2017-08-28 11:40:28.216 [ INFO] dvr: 5 DVBSUB dut
2017-08-28 11:40:28.217 [ INFO] dvr: 6 MPEG2AUDIO dut ? ?
2017-08-28 11:40:28.217 [ INFO] dvr: 7 MPEG2AUDIO swe ? ?
2017-08-28 11:40:36.146 [ INFO] subscription: 0012: "epggrab" unsubscribing

Sorry for taking this off topic, I'm still very much concerned about the original issue with the mkv-recordings and I want to provide help to pinpoint the issue. But there seems to be something else going on, also.

The matroska issue should be fixed in v4.3-363-ge1ad2342d .

Yes I can confirm, mkv-recording seems to work now! Ok now I'll make some tests with the master -> mkv and then with the patch (above), some results will follow

Jaroslav Kysela wrote:

There seems to be a little issue with the changed/reconfigured streams in recent tvh. Could you try this patch?

[...]

After some testing, I can conclude that this patch (the one line of code ) :
-restores the behaviour: mkv recording restarts every time the stream configuration changes, and in the end there are several files named RecordedProgram.mkv, RecordedProgram-1.mkv etc.
-the resulting files seem to have no "conflicts", which would cause the video to hang
-TVHeadend shows these gaps in Failed recordings, with a comment "reconfigured stream", 4.0.9 did not do this.
-the resulting files still have 1-2 chapters (for example 15-90 seconds from the beginning), I assume they are based on some time stamps, but they do not represent any change in streams and do not seem to cause problems. Not sure where they come from, 4.0.9 did not create any time stamps in mkv files.
-the resulting mkv-files still have the metadata as they should, and it is easy to further parse to an .nfo file.

So yes the patch more or less does exactly what I'm talking about, thank you.

I can upload some example recordings, if they are of any use?

Here are links to some example files from yesterday. (Record to mkv).

1) Recording with TVheadend's master. None of the players can properly play past 44s timepoint (where the streams get reconfigured, just before the actual program), but for example Kodi can play rest of the file if you manually skip to the middle of the program. Size about 940Mb.

https://drive.google.com/open?id=0B8vb22q140X3bEFIdXhCM1VIMkk

2) Recording with the patch, first padding file. This can be played with (with dvb subtitles) with any of the compatible players (MPC-HC, Kodi, SMPlayer) (VLC does not support dvb subtitles in mkv-files, btw). Stops just where the streams get reconfigured. Size about 50Mb

https://drive.google.com/open?id=0B8vb22q140X3UnZ0LTljdXpYYWs

3) Recording with the patch, the actual program. There are two chapter marks, I suppose they are from EPG, the actual time when the program was supposted to be aired (the time between the stamps is 2:00, the time of the program). Starts just after the padding file. Can be played with any of the above mentioned players (with dvb subtitles). Size about 103Mb

https://drive.google.com/open?id=0B8vb22q140X3UWpJeHVQYWdIdDQ

While testing, I noticed that even MPC-HC struggles with these mkv-files, if they get large. The seek times get up to point where the program seems to hang forever.
SMPlayer and especially Kodi are the most compatible players, with dvb subtitle support and fast seeking. But this is off topic, again, not related to the problem.

...so the patch works, and I'm even able to copy-paste a new option in dvr_config.c, I was thinking something like (after Skip commercials):

{
.type = PT_BOOL,
.id = "reconfigure-restart",
.name = N_("Restart after reconfigure"),
.desc = N_("Restart the recording if the streams are "
"reconfigured during the recording. "
"Default is unchecked, but try checking this if "
"you have trouble playing the recordins, especially "
".mkv files and players may haveproblem."),
.off = offsetof(dvr_config_t, dvr_reconfigure_restart),
.opts = PO_ADVANCED,
.def.i = 1,
.group = 2,
},

...but since I'm not a coder, I'd rather let someone else implement it in the code. Unless the patch is more of a temporary fix, until the whole mkv-recording gets (somehow) fixed? Still, the tickbox would not be that intruding, and it might help someone.

I wrote this: https://github.com/TurboK234/tvheadend/commit/e5bc706b98540d20c7ee9780d05f04c69ff132cf
, to allow a subtle option to use the patch. This creates a checkbox on the Recording settings tab, just after padding length options (as it is mostly related to the recording that happens before or after the actual program).

Not sure if this is suitable in master, but anyone who wants to record mkv files in areas that constantly change their stream properties, this might help.

Henrik Söderström wrote:

I wrote this: https://github.com/TurboK234/tvheadend/commit/e5bc706b98540d20c7ee9780d05f04c69ff132cf
, to allow a subtle option to use the patch. This creates a checkbox on the Recording settings tab, just after padding length options (as it is mostly related to the recording that happens before or after the actual program).

Not sure if this is suitable in master, but anyone who wants to record mkv files in areas that constantly change their stream properties, this might help.

Please do submit this patch to master for review :).

Mark Clarkstone wrote:

Please do submit this patch to master for review :).

Encouraged by your response, I created a pull request ( https://github.com/tvheadend/tvheadend/pull/1001 ). My real concern is that my coding skills are next to 0, only replicating already present code

Thank you for your contribution, but the muxer may accept or reject the reconfiguration request. At the present, mkv muxer rejects it (because the header must be updated) or eventually, multi-segment mkv can be created (not implemented). The missing muxer reconfiguration request was an error.

Ok! Thanks for the explanation and for implementing the fix in master! No need for my PR anymore

Just to make this clear for others reading about this, it seems that fix to this issue was released in tvheadend-4.2.4 already, can be seen: https://github.com/tvheadend/tvheadend/compare/v4.2.3...v4.2.4 . I thought it was not it any 4.2.x release because the issue says "found in version: 4.3-334~g7db25fa" . So e.g. latest LibreElec stable build 8.2 should contain this fix.

Yes, I can confirm that, sorry for not updating the info thoroughly. The build mentioned in the issue header is from the nightly build at the time of the issue report. The fix was implemented in the master back then, and also in the stable builds shortly afterwards. I'm using the stable build myself now, and the .mkv recording works just fine, as the recording restarts every time the stream properties change.

This issue seems to be back. I created a bug #5576.

Seems to be that #5576 is not exactly same than this issue. But it is related, so good to link it here.