Tvheadend

tvh: --trace descrambler,capmt - https://tvheadend.org/projects/tvheadend/wiki/Traces
oscam: enable log level 128

Could you retest with v4.3-720-g3af771188 ? The commit https://github.com/tvheadend/tvheadend/commit/1db6a4c39fed19f3525ab97e77182797d23a8407 should fix this issue.

Not sure if it is the problem, but the caPMT does not match the real PMT and so oscam ends up sending index 6 and 7 whereas Tvheadend sent only 5 pids in the caPMT

11:10:53 60B87671 c   (dvbapi) capmt:
11:10:53 60B87671 c   (dvbapi)   03 00 05 01 00 19 01 82 02 00 00 81 08 00 00 00 
11:10:53 60B87671 c   (dvbapi)   00 00 01 00 09 84 02 13 8D 09 04 0E 00 17 75 02 
11:10:53 60B87671 c   (dvbapi)   06 18 00 00 04 05 F0 00 00 04 05 F2 00 00 04 05 
11:10:53 60B87671 c   (dvbapi)   FA 00 00 04 05 FC 00 00 
11:10:53 60B87671 c   (dvbapi) Receiver sends PMT command 3 for channel 0005
11:10:53 60B87671 c   (dvbapi) Receiver wants to demux srvid 0005 on adapter 0000 camask 0001 index 0000 pmtpid 0000
11:10:53 60B87671 c   (dvbapi) Demuxer 0 try to start new filter for caid: 0001, provid: 000001, pid: 0000
11:10:53 60B87671 c   (dvbapi) Sending packet to dvbapi client (fd=10):
11:10:53 60B87671 c   (dvbapi)   40 3C 6F 2B 00 00 00 00 00 00 00 00 00 00 00 00 
11:10:53 60B87671 c   (dvbapi)   00 00 00 00 00 00 00 00 00 FF 00 00 00 00 00 00 
11:10:53 60B87671 c   (dvbapi)   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
11:10:53 60B87671 c   (dvbapi)   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
11:10:53 60B87671 c   (dvbapi)   04 
11:10:53 60B87671 c   (dvbapi) Demuxer 0 Filter 1 started successfully (caid 0001 provid 000001 pid 0000)
11:10:53 60B87671 c   (dvbapi) Demuxer 0 found pmt type: 81 length: 8 (assuming enigma private descriptor: namespace 0000 tsid 01 onid 09)
11:10:53 60B87671 c   (dvbapi) Demuxer 0 ecmpid 0 CAID: 0E00 ECM_PID: 1775 PROVID: 000000 
11:10:53 60B87671 c   (dvbapi) Demuxer 0 stream Videostream (MPEG-2)(type: 02 pid: 0618 length: 0)
11:10:53 60B87671 c   (dvbapi) Demuxer 0 stream Audiostream (MPEG-2)(type: 04 pid: 05f0 length: 0)
11:10:53 60B87671 c   (dvbapi) Demuxer 0 stream Audiostream (MPEG-2)(type: 04 pid: 05f2 length: 0)
11:10:53 60B87671 c   (dvbapi) Demuxer 0 stream Audiostream (MPEG-2)(type: 04 pid: 05fa length: 0)
11:10:53 60B87671 c   (dvbapi) Demuxer 0 stream Audiostream (MPEG-2)(type: 04 pid: 05fc length: 0)
11:10:53 60B87671 c   (dvbapi) Demuxer 0 found 1 ECMpids and 5 STREAMpids in caPMT

11:10:54 60B87671 c   (dvbapi) pmt:
11:10:54 60B87671 c   (dvbapi)   02 B0 71 00 05 C3 00 00 E6 18 F0 0C 0F 04 53 41 
11:10:54 60B87671 c   (dvbapi)   50 53 09 04 0E 00 17 75 02 E6 18 F0 0A 86 08 1E 
11:10:54 60B87671 c   (dvbapi)   00 00 00 00 00 00 00 04 E5 F0 F0 00 04 E5 F2 F0 
11:10:54 60B87671 c   (dvbapi)   00 04 E5 FA F0 00 04 E5 FC F0 00 85 E4 28 F0 00 
11:10:54 60B87671 c   (dvbapi)   89 FB 6A F0 0B FE 09 12 02 3D 00 00 00 00 00 00 
11:10:54 60B87671 c   (dvbapi)   89 FB 71 F0 0B FE 09 19 02 14 00 00 00 00 00 00 
11:10:54 60B87671 c   (dvbapi)   89 FB 6F F0 0B FE 09 17 03 33 00 00 00 00 00 00 
11:10:54 60B87671 c   (dvbapi)   A5 EE 6E 4F 
11:10:54 60B87671 c   (dvbapi) Demuxer 0 stream Videostream (MPEG-2)(type: 02 pid: 0618 length: 10)
11:10:54 60B87671 c   (dvbapi) Demuxer 0 stream Audiostream (MPEG-2)(type: 04 pid: 05f0 length: 0)
11:10:54 60B87671 c   (dvbapi) Demuxer 0 stream Audiostream (MPEG-2)(type: 04 pid: 05f2 length: 0)
11:10:54 60B87671 c   (dvbapi) Demuxer 0 stream Audiostream (MPEG-2)(type: 04 pid: 05fa length: 0)
11:10:54 60B87671 c   (dvbapi) Demuxer 0 stream Audiostream (MPEG-2)(type: 04 pid: 05fc length: 0)
11:10:54 60B87671 c   (dvbapi) Demuxer 0 stream Audiostream (DTS 8)(type: 85 pid: 0428 length: 0)
11:10:54 60B87671 c   (dvbapi) Demuxer 0 stream Reserved(type: 89 pid: 1b6a length: 11)
11:10:54 60B87671 c   (dvbapi) Demuxer 0 stream Reserved(type: 89 pid: 1b71 length: 11)
11:10:54 60B87671 c   (dvbapi) Demuxer 0 found 1 ECMpids and 8 STREAMpids in PMT

I cheated when I wrote my code and stopped Tvheadend from sending the real PMT. This was especially useful when I use audio stream filters to only use the audio track I want (blocked the radio tracks...) For example for this channel I set the audio stream filter to use pid 1520 exclusively...

BTW - probably best to just do a "git pull" to get all the changes.

Also, older versions of oscam-emu did not parse all the PMT pids and would have matched the caPMT. But I am sorry I do not recall when it was changed.

Joe User wrote:

Also, older versions of oscam-emu did not parse all the PMT pids and would have matched the caPMT. But I am sorry I do not recall when it was changed.

Anything i can do to help?

If you are using the latest oscam-emu pathc, you can try setting "Max pids for extended CWs" to 5 (under config/DVB-api) and see if that helps.

Joe User wrote:

If you are using the latest oscam-emu pathc, you can try setting "Max pids for extended CWs" to 5 (under config/DVB-api) and see if that helps.

I have searched documentation, not too sure how to do that? What is the parameter name?

Eric Dec wrote:

Joe User wrote:

If you are using the latest oscam-emu pathc, you can try setting "Max pids for extended CWs" to 5 (under config/DVB-api) and see if that helps.

I have searched documentation, not too sure how to do that? What is the parameter name?

Ok, I found the parameter through the Oscam web Gui, I changed it to 5, restarted oscam, but same result, tvheadend is crashing.
Is there another way to get this working? Using the stream-relay? I found a post from yourself about that, but would I be able to use the latest tvheadend and latest oscam?

I am usually running my own version of Tvheadend because it has been quite stable and it includes some changes that I made for myself which are probably of no use to anyone else.

I tested the official version a few months ago and it was working with powervu, but today I tried the latest version form git and while it did not crash, it was not descrambling the channels. I ran it from valgrind and it did crash with the following errors:

==16655==
==16655== Process terminating with default action of signal 11 (SIGSEGV)
==16655==  Bad permissions for mapped region at address 0x0
==16655==    at 0x0: ???
==16655==    by 0x3C79B5: key_find_struct (descrambler.c:941)
==16655==    by 0x3CA3E6: descrambler_descramble (descrambler.c:1043)
==16655==    by 0x3D3D09: ts_recv_packet1 (tsdemux.c:340)
==16655==    by 0x3D1007: mpegts_input_process (mpegts_input.c:1419)
==16655==    by 0x3D1007: mpegts_input_thread (mpegts_input.c:1553)
==16655==    by 0x3239E3: thread_wrapper (wrappers.c:161)
==16655==    by 0x69D50A3: start_thread (pthread_create.c:309)
==16655==    by 0xBAE987C: clone (clone.S:111)
==16655==
==16655== HEAP SUMMARY:
==16655==     in use at exit: 20,723,906 bytes in 184,420 blocks
==16655==   total heap usage: 946,891 allocs, 762,471 frees, 248,752,049 bytes allocated
==16655==
==16655== LEAK SUMMARY:
==16655==    definitely lost: 6,479 bytes in 15 blocks
==16655==    indirectly lost: 549 bytes in 7 blocks
==16655==      possibly lost: 13,376 bytes in 38 blocks
==16655==    still reachable: 20,703,502 bytes in 184,360 blocks
==16655==         suppressed: 0 bytes in 0 blocks
==16655== Rerun with --leak-check=full to see details of leaked memory
==16655==
==16655== For counts of detected and suppressed errors, rerun with: -v
==16655== Use --track-origins=yes to see where uninitialised values come from
==16655== ERROR SUMMARY: 57 errors from 2 contexts (suppressed: 0 from 0)
==16655== could not unlink /tmp/vgdb-pipe-from-vgdb-to-16655-by-root-on-???
==16655== could not unlink /tmp/vgdb-pipe-to-vgdb-from-16655-by-root-on-???
==16655== could not unlink /tmp/vgdb-pipe-shared-mem-vgdb-16655-by-root-on-???

Sorry I do not have time to debug right now, but will try to look at it later.

Did you test v4.3-733-g295288821 ? There's another update for this issue.

BTW - I set an audio filter to exclusively use pid 1520, that is why only pids 1560 and 1520 appear.
But, it crashes with or without the filter.

So I had it return null if pid == 0, and it still crashes.
But if I just comment out the failing line, it runs ok...

     //   tk_old->key_csa.csa_flush(&tk_old->key_csa, (mpegts_service_t *)t);

Ok, last comment tonight (need to concentrate on other things...)
After changing channels a few times. it did crash on pid == 0 even with the failing line commented out.
Sorry don't have time to debug further.

Another little fix is in v4.3-734-g4433c27d8 . The tk_old should be zero for PIDs which are not descrambled. And the point is, if (tsb³ & 0x80) == 0) (scrambling bit is not set) the key_find_struct() should not be called. The function ts_recv_packet0/ts_recv_packet2 should be called instead.

@Eric, if you comment out line 941 of src/descrambler/descrambler.c

//        tk_old->key_csa.csa_flush(&tk_old->key_csa, (mpegts_service_t *)t);

then it will work (for at least awhile), but not necessarily fixed. ;)

Joe User wrote:

@Eric, if you comment out line 941 of src/descrambler/descrambler.c
[...]

then it will work (for at least awhile), but not necessarily fixed. ;)

I commented the line as above, indeed it does not crash, but it does not decode.
(I tried on few releases I had downloaded)
What else could it be? My oscam emu (I also tried the very old one where your change request was implemented)

@Joe Miller User : you should look why tk_old->key_csa.csa_flush is NULL. This code should be never reached. The key descramblers including callbacks should be set in descrambler_keys() - tvhcsa_set_type() calls.

Eric Dec wrote:

Joe User wrote:

@Eric, if you comment out line 941 of src/descrambler/descrambler.c
[...]

then it will work (for at least awhile), but not necessarily fixed. ;)

I commented the line as above, indeed it does not crash, but it does not decode.
(I tried on few releases I had downloaded)
What else could it be? My oscam emu (I also tried the very old one where your change request was implemented)

Sorry, I had added brackets after the if statement before to add the printf. It also crashed sometimes for pid == 0 so I ignore that case.
Try this change:

diff --git a/src/descrambler/descrambler.c b/src/descrambler/descrambler.c
index 80dbf24..d247100 100644
--- a/src/descrambler/descrambler.c
+++ b/src/descrambler/descrambler.c
@@ -936,9 +936,10 @@ key_find_struct( th_descrambler_runtime_t *dr,
   int i, pid = extractpid(tsb);
   for (i = 0; i < DESCRAMBLER_MAX_KEYS; i++) {
     tk = &dr->dr_keys[i];
-    if (tk->key_pid == pid) {
-      if (tk != tk_old && tk_old)
-        tk_old->key_csa.csa_flush(&tk_old->key_csa, (mpegts_service_t *)t);
+    if ( pid == 0 ) return NULL;
+    if (tk->key_pid == pid)  {
+     // if (tk != tk_old && tk_old)
+     //   tk_old->key_csa.csa_flush(&tk_old->key_csa, (mpegts_service_t *)t);
       return tk;
     }
   }
@@ -1035,7 +1036,7 @@ descrambler_descramble ( service_t *t,
       if (len2 == 0)
         goto dd_destroy;
       if ((tsb2[3] & 0x80) == 0) {
-        ts_recv_packet2((mpegts_service_t *)t, tsb2, len2);
+        ts_recv_packet0((mpegts_service_t *)t, st, tsb2, len2);
         goto dd_destroy;
       }
       if (dr->dr_key_multipid) {

Again, this is just a quick hack, not a real solution and may cause other problems (very limited testing - 2-3min...)

Jaroslav Kysela wrote:

@Joe Miller User : you should look why tk_old->key_csa.csa_flush is NULL. This code should be never reached. The key descramblers including callbacks should be set in descrambler_keys() - tvhcsa_set_type() calls.

I will try, but probably will not have time to look at it again until next week...

Joe User wrote:

Eric Dec wrote:

Joe User wrote:

@Eric, if you comment out line 941 of src/descrambler/descrambler.c
[...]

then it will work (for at least awhile), but not necessarily fixed. ;)

I commented the line as above, indeed it does not crash, but it does not decode.
(I tried on few releases I had downloaded)
What else could it be? My oscam emu (I also tried the very old one where your change request was implemented)

Sorry, I had added brackets after the if statement before to add the printf. It also crashed sometimes for pid == 0 so I ignore that case.
Try this change:

[...]

Again, this is just a quick hack, not a real solution and may cause other problems (very limited testing - 2-3min...)

Joe? Guess what? It works!!!!!!!!!!!!!

I made the changes that you show in your message on v4.3-734-g4433c27d8 and I have picture and sound.
Thanks a lot.....

Oups, I have been talking too fast.
After 10mn tvheadend crashes again.

I am rerunning it with gdb in log mode....but does not want to crash anymore after 30mn.

I did another quick test to narrow down the problem, and this simplified patch seems to work ok. Still a hack that does not address the problem, but I am not sure why something with pid == 0 is there. Maybe a problem with extractpid?? The real pid 0 packet should not have its scrambled bit set.

diff --git a/src/descrambler/descrambler.c b/src/descrambler/descrambler.c
index 80dbf24..0096961 100644
--- a/src/descrambler/descrambler.c
+++ b/src/descrambler/descrambler.c
@@ -936,6 +936,7 @@ key_find_struct( th_descrambler_runtime_t *dr,
   int i, pid = extractpid(tsb);
   for (i = 0; i < DESCRAMBLER_MAX_KEYS; i++) {
     tk = &dr->dr_keys[i];
+    if ( pid == 0 ) return NULL;
     if (tk->key_pid == pid) {
       if (tk != tk_old && tk_old)
         tk_old->key_csa.csa_flush(&tk_old->key_csa, (mpegts_service_t *)t);

Joe User wrote:

I did another quick test to narrow down the problem, and this simplified patch seems to work ok. Still a hack that does not address the problem, but I am not sure why something with pid 0 is there. Maybe a problem with extractpid?? The real pid 0 packet should not have its scrambled bit set.

[...]

This means I roll back all previous changes and simply add the line " if ( pid 0 ) return NULL;" ?
By the way, in my test of yesterday, I have let it running few hours in debug mode, and it did not crash.
I will try your last change later today, tanks again.

Jaroslav Kysela wrote:

Applied in changeset commit:tvheadend|b2ef96cfca037cd8054913a5c8cc334d7239d765.

You mean you did more changes than what joe was proposing?
I am asking, because today i did some try with joe fix on other poeervu channels, and it kept craching again or not descrambling.
I will try tomorrow your new version.
Thanks for your effort.

Jaroslav Kysela wrote:

Applied in changeset commit:tvheadend|b2ef96cfca037cd8054913a5c8cc334d7239d765.

I have tried this new version. It works fine on the channels i was trying before, but now it crashes on channels from another satellite (with powervu).
Should ii upload logs here or open another bug?

@Eric, I wrote my patch was just a quick hack, not a solution... :)
@Jaroslav. I built new version and some quick tests on extended_cw channels on 9E and 4.9E all worked. Did not test any other encryptions yet, maybe tomorrow.
Thanks

Still crash here with last TVH 4.3-741~g0b24fb883

gdb:

Thread 36 "tvh:mi-main" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe4be8700 (LWP 25933)]
0x0000555555831eba in key_valid (tk=0x0, ki=157 '\235') at src/descrambler/descrambler.c:876
876      return tk->key_valid & mask;

2017-12-03 04:47:12.199 [  ALERT] CRASH: Signal: 11 in PRG: /home/tvh/tvheadend/build.linux/tvheadend (4.3-741~g0b24fb883) [7908324f0ac08fe5b063e35f1e8f9e7fdd413838] CWD: /home/tvh/tvheadend
2017-12-03 04:47:12.199 [  ALERT] CRASH: Fault address 0x91 (Address not mapped)
2017-12-03 04:47:12.199 [  ALERT] CRASH: Loaded libraries: linux-vdso.so.1 /usr/lib/x86_64-linux-gnu/libssl.so.1.1 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 /lib/x86_64-linux-gnu/libz.so.1 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/librt.so.1 /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /lib/x86_64-linux-gnu/libmvec.so.1 /lib/x86_64-linux-gnu/libgcc_s.so.1/lib/x86_64-linux-gnu/libc.so.6/lib64/ld-linux-x86-64.so.2
2017-12-03 04:47:12.199 [  ALERT] CRASH: Register dump [23]: 00005638bcd0c05825a5fd9e86d38a2d61b5d5b3f5642629038d83d28da80cb200007f477d5ef7ae00007f477d5ef7af00005638b762a44000007f475f5fa7000000000000000000000000000000009700007f475f5f951000000000000000000000000000000097000000000000000000007f474800865900007f475f5f951000005638b3e81eba0000000000010202002b0000000000330000000000000004000000000000000efffffffe7ffbba110000000000000091
2017-12-03 04:47:12.199 [  ALERT] CRASH: STACKTRACE
2017-12-03 04:47:12.240 [  ALERT] CRASH: /home/tvh/tvheadend/src/trap.c:148 0x5638b3de633a 0x5638b3ba4000
2017-12-03 04:47:12.295 [  ALERT] CRASH: ??:0 0x7f4785b653b0 0x7f4785b53000
2017-12-03 04:47:12.338 [  ALERT] CRASH: /home/tvh/tvheadend/src/descrambler/descrambler.c:876 0x5638b3e81eba 0x5638b3ba4000
2017-12-03 04:47:12.375 [  ALERT] CRASH: /home/tvh/tvheadend/src/descrambler/descrambler.c:1142 0x5638b3e82a30 0x5638b3ba4000
2017-12-03 04:47:12.415 [  ALERT] CRASH: /home/tvh/tvheadend/src/input/mpegts/tsdemux.c:340 0x5638b3e90daf 0x5638b3ba4000
2017-12-03 04:47:12.457 [  ALERT] CRASH: /home/tvh/tvheadend/src/input/mpegts/mpegts_input.c:1410 (discriminator 2) 0x5638b3e8d9e6 0x5638b3ba4000
2017-12-03 04:47:12.498 [  ALERT] CRASH: /home/tvh/tvheadend/src/input/mpegts/mpegts_input.c:1553 0x5638b3e8e019 0x5638b3ba4000
2017-12-03 04:47:12.538 [  ALERT] CRASH: /home/tvh/tvheadend/src/wrappers.c:161 0x5638b3d95f19 0x5638b3ba4000
2017-12-03 04:47:12.593 [  ALERT] CRASH: ??:0 0x7f4785b5a519 0x7f4785b53000
2017-12-03 04:47:12.593 [  ALERT] CRASH: clone+0x3f  (/lib/x86_64-linux-gnu/libc.so.6)

Joe User wrote:

@Eric, I wrote my patch was just a quick hack, not a solution... :)
@Jaroslav. I built new version and some quick tests on extended_cw channels on 9E and 4.9E all worked. Did not test any other encryptions yet, maybe tomorrow.
Thanks

You are right.
In fact the other crash i have does not seem to be powervu related, seems dealing with epg, i am not sure yet.
Thanks for having helped.

All descrambling related issues should be resolved in the latest master now. Please, create a new issue otherwise.