Tvheadend

Manio add support extended CW to dvbapi plugin:
https://github.com/manio/vdr-plugin-dvbapi/commit/b4cd00a69ca523ef5f76c108f7dd230f5d0c84e5

I finally found some free time so I worked on adding extended_cw support to TVheadend. I also add DES decryption. I have it basically working, but only have done limited testing. Quite possibly it will break "normal" CSA decryption, (but at this time I do not have a way to test.) I have tested it with both CSA and DES powervu channels.

Things to do:
Find a way to set when to use the extended_cw (for now it is forced.)
Fix decrypting multiple channels on the same transponder ( now starting second channel stops first from working.)
Add more/better debug/trace messages.

Note: I used the latest oscam (11272) with the latest "offical" oscam-emu patch (they incorporated all my changes recently) But I sometimes had problems with oscam getting the PMT information. It would work the first time after oscam was started, but then not again. I am not sure if the problem is with oscam or tvheadend, but I added a line to oscam which enables it to work (almost) always: (line 3144 after oscam-emu patch is applied)

        if((cfg.dvbapi_boxtype == BOXTYPE_PC || cfg.dvbapi_boxtype == BOXTYPE_PC_NODMX || cfg.dvbapi_boxtype == BOXTYPE_SAMYGO)
                 && buffer[7] == 0x82 && buffer[8] == 0x02)
        {
                *demux_index = buffer[9]; // it is always 0 but you never know
                *adapter_index = buffer[10]; // adapter index can be 0,1,2
                *ca_mask = (1 << *adapter_index); // use adapter_index as ca_mask (used as index for ca_fd[] array)
+                if (buffer[21]==0x84 && buffer[22]==0x02) *pmtpid = b2i(2, buffer+23);
        }

Source code is available on my github account (https://github.com/joeusercz/tvheadend/tree/extended_cw) if anyone wants to try it.

I did not bother with a pull request because much more testing needs to be done!

I finally found some free time so I worked on adding extended_cw support to TVheadend. I also add DES decryption. I have it basically working, but only have done limited testing. Quite possibly it will break "normal" CSA decryption, (but at this time I do not have a way to test.) I have tested it with both CSA and DES powervu channels.

Things to do:
Find a way to set when to use the extended_cw (for now it is forced.)
Fix decrypting multiple channels on the same transponder ( now starting second channel stops first from working.)
Possibly use openssl DES functions. (currently I added cscrypt from oscam-emu.)
Add more/better debug/trace messages.

Note: I used the latest oscam (11272) with the latest "offical" oscam-emu patch (they incorporated all my changes recently) But I sometimes had problems with oscam getting the PMT information. It would work the first time after oscam was started, but then not again. I am not sure if the problem is with oscam or tvheadend, but I added a line to oscam which enables it to work (almost) always: (line 3144 after oscam-emu patch is applied)

        if((cfg.dvbapi_boxtype == BOXTYPE_PC || cfg.dvbapi_boxtype == BOXTYPE_PC_NODMX || cfg.dvbapi_boxtype == BOXTYPE_SAMYGO)
                 && buffer[7] == 0x82 && buffer[8] == 0x02)
        {
                *demux_index = buffer[9]; // it is always 0 but you never know
                *adapter_index = buffer[10]; // adapter index can be 0,1,2
                *ca_mask = (1 << *adapter_index); // use adapter_index as ca_mask (used as index for ca_fd[] array)
+                if (buffer[21]==0x84 && buffer[22]==0x02) *pmtpid = b2i(2, buffer+23);
        }

Source code is available on my github account (https://github.com/joeusercz/tvheadend/tree/extended_cw) if anyone wants to try it.

I did not bother with a pull request because much more testing needs to be done!

I finally found some free time so I worked on adding extended_cw support to TVheadend. I also add DES decryption. I have it basically working, but only have done limited testing. Quite possibly it will break "normal" CSA decryption, (but at this time I do not have a way to test.) I have tested it with both CSA and DES powervu channels.

Things to do:
Find a way to set when to use the extended_cw (for now it is forced.)
Fix decrypting multiple channels on the same transponder ( now starting second channel stops first from working.)
Possibly use openssl DES functions. (currently I added cscrypt from oscam-emu.)
Add extended_cw functionality to LIBDVBCSA. ( currently I only implemented it with ffdecsa.)
Add more/better debug/trace messages.

Note: I used the latest oscam (11272) with the latest "offical" oscam-emu patch (they incorporated all my changes recently) But I sometimes had problems with oscam getting the PMT information. It would work the first time after oscam was started, but then not again. I am not sure if the problem is with oscam or tvheadend, but I added a line to oscam which enables it to work (almost) always: (line 3144 after oscam-emu patch is applied)

        if((cfg.dvbapi_boxtype == BOXTYPE_PC || cfg.dvbapi_boxtype == BOXTYPE_PC_NODMX || cfg.dvbapi_boxtype == BOXTYPE_SAMYGO)
                 && buffer[7] == 0x82 && buffer[8] == 0x02)
        {
                *demux_index = buffer[9]; // it is always 0 but you never know
                *adapter_index = buffer[10]; // adapter index can be 0,1,2
                *ca_mask = (1 << *adapter_index); // use adapter_index as ca_mask (used as index for ca_fd[] array)
+                if (buffer[21]==0x84 && buffer[22]==0x02) *pmtpid = b2i(2, buffer+23);
        }

Source code is available on my github account (https://github.com/joeusercz/tvheadend/tree/extended_cw) if anyone wants to try it.

I did not bother with a pull request because much more testing needs to be done!

Ok, I did a lot of work in current 4.3 tree. It's almost ready but untested. I choose a bit different implementation way than Joe User. So, if you look to v4.3-156-g85ea455, all pieces are in.

Great work @Jaroslav Kysela,
i tested last 4.3-156-g85ea455, but you already have code for PowerVu channels ? , because i tested with PowerVu channels from 1W and not work.

I am tested also code of @Joe Miller User from here https://github.com/joeusercz/tvheadend/tree/extended_cw and PowerVU channels work nice with his code.

I am readdy to provide logs and somethings else for PowerVu if you needs.

More work in v4.3-161-gb8aae7b. If you use extended_cw_api = 1 in oscam, you should enable 'Extended' CW mode in tvh, too. The 'Extended DES' mode should be autodetected.

I tested last v4.3-161-gb8aae7b with enable extended_cw_api = 1 in oscam and also enable 'Extended' mode in CAPMT in CW mode.

On PowerVu channels, TVH crash:

2017-05-26 18:43:34.936 [ ERROR] descrambler: cannot decode packets for service "Animal Planet HD"
2017-05-26 18:43:36.919 [ ALERT] CRASH: Signal: 11 in PRG: ./build.linux/tvheadend (4.3-161~gb8aae7bdd) [f1dfff650880d6b1a8b13cb8bda2984979894788] CWD: /home/user/tvheadend
2017-05-26 18:43:36.919 [ ALERT] CRASH: Fault address 0x28 (Address not mapped)
2017-05-26 18:43:36.919 [ ALERT] CRASH: Loaded libraries: linux-vdso.so.1 /usr/lib/libdvben50221.so /usr/lib/libdvbapi.so /usr/lib/libucsi.so /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 /lib/x86_64-linux-gnu/libz.so.1 /lib/x86_64-linux-gnu/libpcre.so.3 /usr/lib/x86_64-linux-gnu/liburiparser.so.1 /lib/x86_64-linux-gnu/libdbus-1.so.3 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/librt.so.1 /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libsystemd.so.0 /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libselinux.so.1 /lib/x86_64-linux-gnu/liblzma.so.5 /usr/lib/x86_64-linux-gnu/liblz4.so.1 /lib/x86_64-linux-gnu/libgcrypt.so.20 /lib/x86_64-linux-gnu/libgpg-error.so.0
2017-05-26 18:43:36.919 [ ALERT] CRASH: Register dump [23]: 0000000000280a470000000000173ea40000000000000a280000000000000814000000000000017800007efea00063a0000000000000000000007efea0008a480000000000000020000055a442fb99a0000000000007f99000007efe8021604000007efe802160400000000000000178000000000000017800007efe997f9670000055a4411d74260000000000010206002b0000000000330000000000000004000000000000000efffffffe7ffbba110000000000000028
2017-05-26 18:43:36.919 [ ALERT] CRASH: STACKTRACE
2017-05-26 18:43:36.934 [ ALERT] CRASH: /home/user/tvheadend/src/trap.c:148 0x55a4411727ba 0x55a440fe8000
2017-05-26 18:43:36.949 [ ALERT] CRASH: ??:0 0x7efed22120c0 0x7efed2201000
2017-05-26 18:43:36.964 [ ALERT] CRASH: /home/user/tvheadend/src/descrambler/descrambler.c:849 0x55a4411d7426 0x55a440fe8000
2017-05-26 18:43:36.979 [ ALERT] CRASH: /home/user/tvheadend/src/input/mpegts/tsdemux.c:338 0x55a4411e0331 0x55a440fe8000
2017-05-26 18:43:36.993 [ ALERT] CRASH: /home/user/tvheadend/src/input/mpegts/mpegts_input.c:1412 0x55a4411dd855 0x55a440fe8000
2017-05-26 18:43:37.006 [ ALERT] CRASH: /home/user/tvheadend/src/wrappers.c:161 0x55a44113a871 0x55a440fe8000

Retest with v4.3-174-gfdb364f .

Note that there are two modes for OE 2.2 key algo/mode support. You should select the right one.

git pull
remote: Counting objects: 12, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 12 (delta 9), reused 12 (delta 9), pack-reused 0
Разпакетиране на обектите: 100% (12/12), действието завърши.
От https://github.com/tvheadend/tvheadend
 + c4bba29...fdb364f master     -> origin/master  (принудително обновяване)
Автоматично сливане на „src/descrambler/algo/libdesdec.c“
КОНФЛИКТ (съдържание): Конфликт при сливане на „src/descrambler/algo/libdesdec.c“
Автоматично сливане на „src/descrambler/algo/libaesdec.c“
КОНФЛИКТ (съдържание): Конфликт при сливане на „src/descrambler/algo/libaesdec.c“
Неуспешно сливане — коригирайте конфликтите и подайте резултата.

CC              src/descrambler/algo/libaesdec.o
src/descrambler/algo/libaesdec.c: In function ‘aes_decrypt_packet’:
src/descrambler/algo/libaesdec.c:69:1: error: expected expression before ‘<<’ token
 <<<<<<< HEAD
 ^
src/descrambler/algo/libaesdec.c:86:1: error: expected expression before ‘==’ token
 =======
 ^
src/descrambler/algo/libaesdec.c:97:1: error: expected expression before ‘>>’ token
 >>>>>>> fdb364faba529af672c550d8bc2645bd50e6cc26
 ^
src/descrambler/algo/libaesdec.c:107:1: error: expected declaration or statement at end of input
 }
 ^
src/descrambler/algo/libaesdec.c:107:1: error: expected declaration or statement at end of input
Makefile:638: recipe for target '/opt/tvheadend/build.linux/src/descrambler/algo/libaesdec.o' failed
make[2]: *** [/opt/tvheadend/build.linux/src/descrambler/algo/libaesdec.o] Error 1
make[2]: Leaving directory '/opt/tvheadend'
debian/rules:11: recipe for target 'override_dh_auto_build' failed
make[1]: *** [override_dh_auto_build] Error 2
make[1]: Leaving directory '/opt/tvheadend'
debian/rules:5: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

@Sean Sean acro: git repo is fine. I pushed an old version of one commit, so you should remove it - you merged the old and new code in your local branch, so you're not on v4.3-174-gfdb364f. Do:

$ git reset --hard origin/master

or 

$ git reset --hard HEAD~10
$ git pull origin master

Jaroslav Kysela wrote:

@Sean Sean acro: git repo is fine. I pushed an old version of one commit, so you should remove it - you merged the old and new code in your local branch, so you're not on v4.3-174-gfdb364f. Do:

$ git reset --hard HEAD~10
$ git pull origin master

Last two commands solve problem. My last pull and build was 4.3-173~gc4bba29
on 3 systems same error debian x64, ubuntu x64 and x86

v4.3-174-gfdb364f work good with Extended (OE 2.2), mode follows key for PowerVU channels on 1W and 9E work good, other mode only Extended (OE 2.2) not work for PowerVu i dont for what is.
Thanks, i will make more test ,but for now work good.

Retest with v4.3-206-gb5ec51c .

I think I have the same problem

2017-06-08 22:20:33.534 capmt: OSCam: Starting CAPMT server for service "RTL HD" on adapter 0
2017-06-08 22:20:33.534 subscription: 0078: "192.168.30.38 [ htpc | Kodi Media Center ]" subscribing on channel "RTL HD", weight: 150, adapter: "SAT>IP DVB-C Tuner #3 (192.168.30.200)", network: "DVB-C Network", mux: "138MHz", provider: "Unitymedia", service: "RTL HD", profile="htsp", hostname="192.168.30.38", username="htpc", client="Kodi Media Center"
2017-06-08 22:20:33.613 TS: DVB-C Network/138MHz/RTL HD Transport error indicator (total 1)
2017-06-08 22:20:34.031 TS: DVB-C Network/138MHz/RTL HD: H264 #523: Invalid start code 8f:37:0c
2017-06-08 22:20:34.132 TS: DVB-C Network/138MHz/RTL HD: AC3 #524: Invalid start code f8:fa:33
2017-06-08 22:20:38.371 TS: DVB-C Network/138MHz/RTL HD: H264 #523 Continuity counter error (total 1)
2017-06-08 22:20:38.371 TS: DVB-C Network/138MHz/RTL HD: AC3 #524 Continuity counter error (total 1)
2017-06-08 22:20:38.371 TS: DVB-C Network/138MHz/RTL HD: TELETEXT #528 Continuity counter error (total 1)
2017-06-08 22:20:38.373 TS: DVB-C Network/138MHz/RTL HD: HBBTV #1561 Continuity counter error (total 1)
2017-06-08 22:20:38.375 descrambler: cannot decode packets for service "RTL HD"
2017-06-08 22:20:48.331 TS: DVB-C Network/138MHz/RTL HD: H264 #523 Continuity counter error (total 2492)
2017-06-08 22:20:48.331 TS: DVB-C Network/138MHz/RTL HD: AC3 #524 Continuity counter error (total 2105)
2017-06-08 22:20:48.331 TS: DVB-C Network/138MHz/RTL HD: TELETEXT #528 Continuity counter error (total 1994)
2017-06-08 22:20:48.332 TS: DVB-C Network/138MHz/RTL HD: HBBTV #1561 Continuity counter error (total 52)
2017-06-08 22:20:48.332 descrambler: cannot decode packets for service "RTL HD"
2017-06-08 22:20:48.334 TS: DVB-C Network/138MHz/RTL HD: H264 #523: Invalid start code 4b:28:30
2017-06-08 22:20:48.336 TS: DVB-C Network/138MHz/RTL HD: AC3 #524: Invalid start code b5:01:2d
2017-06-08 22:21:08.373 TS: DVB-C Network/138MHz/RTL HD: H264 #523 Continuity counter error (total 2525)
2017-06-08 22:21:08.373 TS: DVB-C Network/138MHz/RTL HD: AC3 #524 Continuity counter error (total 2133)
2017-06-08 22:21:08.373 TS: DVB-C Network/138MHz/RTL HD: TELETEXT #528 Continuity counter error (total 2021)
2017-06-08 22:21:08.374 TS: DVB-C Network/138MHz/RTL HD: HBBTV #1561 Continuity counter error (total 53)

v4.3-210-g7644886

Try v4.3-211-g1c1f52c . I think that I finally found the right fix.

I dont have problems here... With 4.3-211. Its true that I am using simple descrambling. But since last version is working very, very good.

Perexg has done a good job. :P

v4.3-213-g54f2d47 , it should resolve both 'no key' and 'Invalid cipher' issues

Change '#if 0' to '#if 1' (DEBUG2) in src/descrambler/descrambler.c and create a new (more verbose) log with this.

Add this code, create a new log and show me the contents of the 'service info' dialog (i icon in the service grid).

diff --git a/src/descrambler/descrambler.c b/src/descrambler/descrambler.c
index 05492bf..52bd615 100644
--- a/src/descrambler/descrambler.c
+++ b/src/descrambler/descrambler.c
@@ -1014,7 +1014,10 @@ descrambler_descramble ( service_t *t,
         goto dd_destroy;
       if (dr->dr_key_multipid) {
         tk = key_find_struct(dr, tk, tsb2, t);
-        if (tk == NULL) goto next;
+        if (tk == NULL) {
+          debug2("%p: tk == NULL, pid = %d", dr, extractpid(tsb2));
+          goto next;
+        }
       }
       now = mclk();
 #ifdef DEBUG2

Could you test this change?

diff --git a/src/descrambler/capmt.c b/src/descrambler/capmt.c
index 67d076f..290bad0 100644
--- a/src/descrambler/capmt.c
+++ b/src/descrambler/capmt.c
@@ -2008,6 +2008,7 @@ capmt_update_elementary_stream(capmt_service_t *ct, int *_i,
   case SCT_H264:       type = 0x1b; break;
   case SCT_HEVC:       type = 0x24; break;
   case SCT_DVBSUB:     type = 0x06; break;
+  case SCT_TELETEXT:   type = 0x06; break;
   default:
     if (SCT_ISVIDEO(st->es_type)) type = 0x02;
     else if (SCT_ISAUDIO(st->es_type)) type = 0x04;

Nope, don't decode.

Use 'Science Poland HD' and verify if oscam sends keys for PID 1811 (TELETEXT). I probably need logs from both tvh and oscam to see what's wrong.

Retest with v4.3-218-gea2d899, pls.

crash on this channel

2017-06-12 23:16:49.456 [ ALERT] CRASH: Signal: 11 in PRG: ./build.linux/tvheadend (4.3-218~gea2d89950) [1d6f3d7974e1881036a045dac0e70daddd09bd03] CWD: /home/user/tvheadend
2017-06-12 23:16:49.456 [ ALERT] CRASH: Fault address 0x68 (Address not mapped)
2017-06-12 23:16:49.456 [ ALERT] CRASH: Loaded libraries: linux-vdso.so.1 /usr/lib/libdvben50221.so /usr/lib/libdvbapi.so /usr/lib/libucsi.so /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 /lib/x86_64-linux-gnu/libz.so.1 /lib/x86_64-linux-gnu/libpcre.so.3 /usr/lib/x86_64-linux-gnu/liburiparser.so.1 /lib/x86_64-linux-gnu/libdbus-1.so.3 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/librt.so.1 /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libsystemd.so.0 /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libselinux.so.1 /lib/x86_64-linux-gnu/liblzma.so.5 /usr/lib/x86_64-linux-gnu/liblz4.so.1 /lib/x86_64-linux-gnu/libgcrypt.so.20 /lib/x86_64-linux-gnu/libgpg-error.so.0
2017-06-12 23:16:49.456 [ ALERT] CRASH: Register dump [23]: 00007fb70000551083781dc514080700c14c021223650e2cfdc532239e620431000000000000000000007fb70001e64000007fb70000555000007fb71c008fc800007fb71c006520000000000000071300007fb71c00656800000000000000bc00007fb71c008fc80000000000000040000000000000000000007fb72a1ef6700000560d89aa779f0000000000010246002b0000000000330000000000000004000000000000000efffffffe7ffbba110000000000000068
2017-06-12 23:16:49.456 [ ALERT] CRASH: STACKTRACE
2017-06-12 23:16:49.470 [ ALERT] CRASH: /home/user/tvheadend/src/trap.c:148 0x560d89a4201a 0x560d8989f000
2017-06-12 23:16:49.491 [ ALERT] CRASH: ??:0 0x7fb74c8d80c0 0x7fb74c8c7000
2017-06-12 23:16:49.510 [ ALERT] CRASH: /home/user/tvheadend/src/descrambler/descrambler.c:1170 0x560d89aa779f 0x560d8989f000
2017-06-12 23:16:49.524 [ ALERT] CRASH: /home/user/tvheadend/src/input/mpegts/tsdemux.c:338 0x560d89ab0441 0x560d8989f000
2017-06-12 23:16:49.540 [ ALERT] CRASH: /home/user/tvheadend/src/input/mpegts/mpegts_input.c:1332 0x560d89aad965 0x560d8989f000
2017-06-12 23:16:49.552 [ ALERT] CRASH: /home/user/tvheadend/src/wrappers.c:161 0x560d89a09ee1 0x560d8989f000
2017-06-12 23:16:49.567 [ ALERT] CRASH: ??:0 0x7fb74c8ce494 0x7fb74c8c7000
2017-06-12 23:16:49.567 [ ALERT] CRASH: clone+0x3f (/lib/x86_64-linux-gnu/libc.so.6)

I am unable to test now, but a quick look at the oscam log show the initial pmt generated and sent by tvheadend and the real pmt sent later are not in the same order - this can cause index sync problems between oscam and tvheadend.

2017/06/12 22:16:18 36C3147D c   (dvbapi) Demuxer 0 found pmt type: 81 length: 8 (assuming enigma private descriptor: namespace 0000 tsid 01 onid 5e)
2017/06/12 22:16:18 36C3147D c   (dvbapi) Demuxer 0 ecmpid 0 CAID: 0E00 ECM_PID: 177F PROVID: 000000 
2017/06/12 22:16:18 36C3147D c   (dvbapi) Demuxer 0 stream Videostream (MPEG-4)(type: 1b pid: 0708 length: 0)
2017/06/12 22:16:18 36C3147D c   (dvbapi) Demuxer 0 stream Audiostream (Dolby Digital)(type: 81 pid: 070a length: 0)
2017/06/12 22:16:18 36C3147D c   (dvbapi) Demuxer 0 stream Audiostream (Dolby Digital)(type: 81 pid: 0709 length: 0)
2017/06/12 22:16:18 36C3147D c   (dvbapi) Demuxer 0 stream Audiostream (MPEG-2)(type: 04 pid: 070b length: 0)
2017/06/12 22:16:18 36C3147D c   (dvbapi) Demuxer 0 found 1 ECMpids and 4 STREAMpids in caPMT
2017/06/12 22:16:18 36C3147D c   (dvbapi) Demuxer 0 serving srvid 0708 (Science Poland HD) on adapter 0000 camask 0001 index 0000 pmtpid 0000

2017/06/12 22:16:19 36C3147D c   (dvbapi) pmt:
2017/06/12 22:16:19 36C3147D c   (dvbapi)   02 B0 78 07 08 CB 00 00 E7 08 F0 0C 09 04 0E 00 
2017/06/12 22:16:19 36C3147D c   (dvbapi)   F7 7F 0F 04 53 41 50 53 1B E7 08 F0 17 28 04 4D 
2017/06/12 22:16:19 36C3147D c   (dvbapi)   40 1E 3F 2A 0F FF 7F 00 00 00 01 00 00 01 C2 00 
2017/06/12 22:16:19 36C3147D c   (dvbapi)   00 03 E9 9F 06 E7 09 F0 11 81 05 08 20 03 FF 0F 
2017/06/12 22:16:19 36C3147D c   (dvbapi)   0A 04 70 6F 6C 00 6A 02 00 00 06 E7 0A F0 11 81 
2017/06/12 22:16:19 36C3147D c   (dvbapi)   05 08 20 03 FF 2F 0A 04 65 6E 67 00 6A 02 00 00 
2017/06/12 22:16:19 36C3147D c   (dvbapi)   04 E7 0B F0 06 0A 04 70 6F 6C 00 06 E7 13 F0 07 
2017/06/12 22:16:19 36C3147D c   (dvbapi)   56 05 70 6F 6C 28 58 4A 55 E1 A7 
2017/06/12 22:16:19 36C3147D c   (dvbapi) Demuxer 0 stream Videostream (MPEG-4)(type: 1b pid: 0708 length: 23)
2017/06/12 22:16:19 36C3147D c   (dvbapi) Demuxer 0 stream Data-/Audiostream (Subtitles/VBI and AC-3)(type: 06 pid: 0709 length: 17)
2017/06/12 22:16:19 36C3147D c   (dvbapi) Demuxer 0 stream Data-/Audiostream (Subtitles/VBI and AC-3)(type: 06 pid: 070a length: 17)
2017/06/12 22:16:19 36C3147D c   (dvbapi) Demuxer 0 stream Audiostream (MPEG-2)(type: 04 pid: 070b length: 6)
2017/06/12 22:16:19 36C3147D c   (dvbapi) Demuxer 0 stream Data-/Audiostream (Subtitles/VBI and AC-3)(type: 06 pid: 0713 length: 7)
2017/06/12 22:16:19 36C3147D c   (dvbapi) Demuxer 0 found 1 ECMpids and 5 STREAMpids in PMT
2017/06/12 22:16:19 36C3147D c   (dvbapi) Demuxer 0 serving srvid 0708 (Science Poland HD) on adapter 0000 camask 0001 index 0000 pmtpid 0000

In my code I solved this by never sending the real pmt to oscam.

This also becomes a problem when one uses stream filters to use only certain audio pids...

Could you compile tvh with '--enable-ccdebug' ? I don't see probably the exact crash line with the optimized binary. Provide the crash lines again.

Joe User wrote:

In my code I solved this by never sending the real pmt to oscam.

This also becomes a problem when one uses stream filters to use only certain audio pids...

The new code works only with PID numbers, so the unknown PID (filtered or so) is just ignored. The teletext PID was missing in the first (non-real) PMT sent by tvh, but it's fixed now.

2017-06-12 23:57:53.453 [ ALERT]:CRASH: Signal: 11 in PRG: ./build.linux/tvheadend (4.3-218~gea2d89950) [28d7241e214ff74166da640161d6ad9546c7081f] CWD: /home/home/user/tvheadend
2017-06-12 23:57:53.453 [ ALERT]:CRASH: Fault address 0x68 (Address not mapped)
2017-06-12 23:57:53.453 [ ALERT]:CRASH: Loaded libraries: linux-vdso.so.1 /usr/lib/libdvben50221.so /usr/lib/libdvbapi.so /usr/lib/libucsi.so /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 /lib/x86_64-linux-gnu/libz.so.1 /lib/x86_64-linux-gnu/libpcre.so.3 /usr/lib/x86_64-linux-gnu/liburiparser.so.1 /lib/x86_64-linux-gnu/libdbus-1.so.3 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/librt.so.1 /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libsystemd.so.0 /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libselinux.so.1 /lib/x86_64-linux-gnu/liblzma.so.5 /usr/lib/x86_64-linux-gnu/liblz4.so.1 /lib/x86_64-linux-gnu/libgcrypt.so.20 /lib/x86_64-linux-gnu/libgpg-error.so.0
2017-06-12 23:57:53.453 [ ALERT]:CRASH: Register dump [23]: 00007f4d7803d6f0031e06007ce05b00105c11ad56d5e964d471443cb3d20068000000000000000000007f4dc838988f000000000000000000007f4dceb4604000007f4d7803e72000007f4da0006bb800007f4daa7f269000000000000000000000005abc9e3736000000000000000000005634c0c5183000007f4daa7f261000005634bf29a5f90000000000010246002b0000000000330000000000000004000000000000000efffffffe7ffbba110000000000000068
2017-06-12 23:57:53.453 [ ALERT]:CRASH: STACKTRACE
2017-06-12 23:57:53.467 [ ALERT]:CRASH: /home/home/user/tvheadend/src/trap.c:148 0x5634bf209354 0x5634bf049000
2017-06-12 23:57:53.481 [ ALERT]:CRASH: ??:0 0x7f4dccf8f0c0 0x7f4dccf7e000
2017-06-12 23:57:53.494 [ ALERT]:CRASH: /home/home/user/tvheadend/src/descrambler/descrambler.c:1018 0x5634bf29a5f9 0x5634bf049000
2017-06-12 23:57:53.508 [ ALERT]:CRASH: /home/home/user/tvheadend/src/input/mpegts/tsdemux.c:338 0x5634bf2a8720 0x5634bf049000
2017-06-12 23:57:53.525 [ ALERT]:CRASH: /home/home/user/tvheadend/src/input/mpegts/mpegts_input.c:1332 (discriminator 2) 0x5634bf2a538c 0x5634bf049000
2017-06-12 23:57:53.541 [ ALERT]:CRASH: /home/home/user/tvheadend/src/input/mpegts/mpegts_input.c:1475 0x5634bf2a59bf 0x5634bf049000
2017-06-12 23:57:53.552 [ ALERT]:CRASH: /home/home/user/tvheadend/src/wrappers.c:161 0x5634bf1bcd24 0x5634bf049000
2017-06-12 23:57:53.566 [ ALERT]:CRASH: ??:0 0x7f4dccf85494 0x7f4dccf7e000
2017-06-12 23:57:53.566 [ ALERT]:CRASH: clone+0x3f (/lib/x86_64-linux-gnu/libc.so.6)

v4.3-219-g11b2611

Sorry, I'm just blind for those bugs. v4.3-222-g429679e

v4.3-222-g429679e
Now and other PowerVu channels where, before decode now not decode.

The oscam replies are not perfect. There's mix for DES/CSA key algorithms. If you see 'CSA' key, then something is wrong. Try v4.3-224-g9432ddd - I added another workaround.

Jaroslav Kysela wrote:

Joe User wrote:

In my code I solved this by never sending the real pmt to oscam.

This also becomes a problem when one uses stream filters to use only certain audio pids...

The new code works only with PID numbers, so the unknown PID (filtered or so) is just ignored. The teletext PID was missing in the first (non-real) PMT sent by tvh, but it's fixed now.

I was away for a few weeks and I am quite busy catching up, so I do not have time now to test or look at the new code. But, CA_SET_DESCR and CA_SET_DESCR_MODE messages both only include an index value and not a pid, so I am not sure how you use only pid numbers?? Also, I vaguely remember there was also a possible issue (bug??) internal to oscam affecting the indexes when the pmt changed. Sorry can't remember exact details as it was over a year ago and blocking the real PMT from being sent fixed the problem for me.

Also, I do not recall ever seeing oscam sending conflicting modes and I quick check of my STB shows the IOCTL calls from oscam-emu are consistent for these channels. Although there certainly couls be a difference between the IOCTL calls and the messages sent via network. Again. sorry I do not have time now to review the oscam-emu DVBAPI module.

Last thing, a recent change to oscam switched the order of sending CA_SET_DESCR and CA_SET_DESR_MODE which caused problems for me. I wrote a bug report, but it has been ignored. ([[http://www.streamboard.tv/oscam/ticket/4586]] So, I reverted the change in oscam-emu. The source code for the latest oscam patched with oscam-emu is available on my github account: [[https://github.com/joeusercz/OSCam-with-emu-branch/tree/oscam-emu-merged]] More info here: [[http://www.streamboard.tv/wbb2/thread.php?threadid=45236&hilightuser=142703]]

Now finaly with v4.3-224-g9432ddd three PowerVu channels with teletext 'TLC' 'Animal' 'Science' Poland now decode and work.

With oscam from svn and patched with oscam-emu is many unstable oscam send/replies every time with mix DES/CSA and with more then 2 channels 3rd and more or not decode or must stat stop several time to start work good.

Now i make test with oscam-emu with change from @Joe Miller User and all work perfect for now.

Now i make different test with start stop different channels to see all stable and no more freeze , will make more test ,but for now work good.

Joe User wrote:

I was away for a few weeks and I am quite busy catching up, so I do not have time now to test or look at the new code. But, CA_SET_DESCR and CA_SET_DESCR_MODE messages both only include an index value and not a pid, so I am not sure how you use only pid numbers?? Also, I vaguely remember there was also a possible issue (bug??) internal to oscam affecting the indexes when the pmt changed. Sorry can't remember exact details as it was over a year ago and blocking the real PMT from being sent fixed the problem for me.

You know, there are three commands which controls the descrambling, SET_PID, SET_DESCR_MODE, SET_DESCR. The decoder must know the PID and the associated key. All other things (like indexes for PID/DESCR_MODE/DESCR) are just implementation details which should (and they are) handled only in the dvbapi client. So the current descrambler code receives 'key type', 'key pid', 'odd/oven keys' from the client. No indexes.

Also, I do not recall ever seeing oscam sending conflicting modes and I quick check of my STB shows the IOCTL calls from oscam-emu are consistent for these channels. Although there certainly couls be a difference between the IOCTL calls and the messages sent via network. Again. sorry I do not have time now to review the oscam-emu DVBAPI module.

With your version of oscam-emu, I can see CSA keys instead DSA keys at the stream start - wrong DESCR_MODE (it's rare, but it can be quite easily reproduced with tvh - just do quick wget test multiple times and tvh will print warning that CSA descrambling was changed to DES - first keys were CSA and the later marked as DES).

Last thing, a recent change to oscam switched the order of sending CA_SET_DESCR and CA_SET_DESR_MODE which caused problems for me. I wrote a bug report, but it has been ignored. ([[http://www.streamboard.tv/oscam/ticket/4586]]

The order is really problematic. It's true that DESCR_MODE is static when initially set. So it works with hardware descramblers, but when you add more logic into it, it's really necessary to know, when you have all parameters/info gathered. I just added 'CW Mode' field to the configuration of capmt/dvbapi client in tvh where you can select the incoming order:

   { N_("Standard / auto"),                     CAPMT_CWMODE_AUTO },
   { N_("Extended (OE 2.2)"),                   CAPMT_CWMODE_OE22 },
   { N_("Extended (OE 2.2), mode follows key"), CAPMT_CWMODE_OE22SW },
   { N_("Extended DES (OE 2.0)"),               CAPMT_CWMODE_OE20 },

Most of modes are detected automatically except the 'mode follows key' (which is unfortunately used in the current official oscam repo). I suggested this [[http://www.streamboard.tv/oscam/ticket/4615]] , but the maintainers are not very responsible, indeed.

Hi,

just to clarify, are the two mentioned oscam-branches required, or would they just allow the automatic "mode follows key"-mode detection?

Jaroslav Kysela wrote:

You know, there are three commands which controls the descrambling, SET_PID, SET_DESCR_MODE, SET_DESCR. The decoder must know the PID and the associated key. All other things (like indexes for PID/DESCR_MODE/DESCR) are just implementation details which should (and they are) handled only in the dvbapi client. So the current descrambler code receives 'key type', 'key pid', 'odd/oven keys' from the client. No indexes.

I just found it much simpler and easier to use the index as an index for the array of keys - this is what is done with the oscam-emu stream-relay descrambling. Also I wanted to change as little as possible the existing tvheadend code to make future merges easier.

With your version of oscam-emu, I can see CSA keys instead DSA keys at the stream start - wrong DESCR_MODE (it's rare, but it can be quite easily reproduced with tvh - just do quick wget test multiple times and tvh will print warning that CSA descrambling was changed to DES - first keys were CSA and the later marked as DES).

I just assume CSA until/unless I received a CA_SET_DESCR_MODE telling me otherwise since it is the most commonly used. I can't test now because I gave my only dvb-s2 tuner card to a friend to use...

The order is really problematic. It's true that DESCR_MODE is static when initially set. So it works with hardware descramblers, but when you add more logic into it, it's really necessary to know, when you have all parameters/info gathered. I just added 'CW Mode' field to the configuration of capmt/dvbapi client in tvh where you can select the incoming order:

[...]

Most of modes are detected automatically except the 'mode follows key' (which is unfortunately used in the current official oscam repo). I suggested this [[http://www.streamboard.tv/oscam/ticket/4615]] , but the maintainers are not very responsible, indeed.

I really do not see the point of selecting the extended_cw mode in tvheadend. The extended_cw mode MUST be set in oscam to match the firmware (in this case software) being used. The main difference being that the OE 2.2 firmware supports the sending of the CA_SET_DESCR_MODE messages. The older firmware did not, it used different indexes for the DES keys. (Talking about Dreambox firmware..) So for tvheadend, I chose to implement the OE 2.2 method (much easier) to emulate in tvheadend. (I did the same for the spark7162/7111 firmware changes I made.) There is no need to implement both, let alone have to try to figure out which mode oscam is using. I think it just makes it more confusing for the users. It seems much more simple and straight forward to implement just the OE 2.2 mode and tell the user they MUST set the oscam dvbapi extended_cw mode to 1 (OE 2.2).

Joe User wrote:

In case somebody wants to test, here is an oscam-emu patch which reverts the oscam change referred to above.
With this version of the oscam-eum patch, CA_SET_DESCR_MODE is sent before CA_SET_DESCR.
I will see about getting it pushed to the "official" repo...

In this patch there is something with missing in
module-dvbapi.h

#define MAX_DEMUX 16

#define MAX_DEMUX 16

The code was written primarily to be used with STBs and 15 is far, far more than most people will ever need. Also most STBs have very limited memory, so memory usage of oscam is a concern. In your case, I can see where you would want to increase the value. Just be aware that it is used for a number of things in the code, so there is a chance of something breaking if you increase it and exceed 15 channels. But you should be able to use valgrind to find any problems.

@nemal nemal k: do you have enabled 'extended_cw_api = 1' in oscam? oscam does not send keys to tvh..

dvbapi section is following in oscam.conf file

[dvbapi]
enabled                       = 1
pmt_mode                      = 4
listen_port                   = 9000
delayer                       = 80
user                          = tvheadend
extended_cw_api               = 1
boxtype                       = pc

I think it is somehow related to aarch64 hardware.
The problem with aarch64 is that oscam with oscam-emu.patch doesn't compile for aarch64 (https://github.com/oscam-emu/OSEmu/issues/146) so I had to compile oscam+oscam-emu for armhf.

Now I switched my setup to oscam (without oscam-emu.patch, aarch64) + OSEmu (armhf) setup, decoding itself started working, I can see following lines in tvh log:

2017-06-21 10:13:00.194 [  TRACE]:descrambler: Obtained CSA keys[3200] 70F8268E98921C46:0000000000000000 pid 0C80 from capmt-127.0.0.1-9000 for service

but now I see

Jun 21 10:12:57 tvheadend tvheadend[13433]: TS: thor 1.0W/12303V/Science Europe HD: AC3 @ #3201: Invalid start code 4b:a3:08
Jun 21 10:12:57 tvheadend tvheadend[13433]: TS: thor 1.0W/12303V/Science Europe HD: AC3 @ #3204: Invalid start code 4b:a3:08
Jun 21 10:12:57 tvheadend tvheadend[13433]: TS: thor 1.0W/12303V/Science Europe HD: AC3 @ #3202: Invalid start code 0c:a8:a3
Jun 21 10:12:57 tvheadend tvheadend[13433]: TS: thor 1.0W/12303V/Science Europe HD: AC3 @ #3203: Invalid start code 80:db:8d

Will try on different architecture.

Make sure stream relay is disabled in oscam.conf.

[streamrelay]
stream_relay_enabled          = 0

Joe User wrote:

Make sure stream relay is disabled in oscam.conf.
[...]

Thanks, that did the trick.

I can confirm that issue.

I cannot reproduce here and the logs are not commented, so I'm lost with them. Please, provide commented logs (detailed time where the problem occurs).

I cannot test Jaroslav's version right now, but most likely the problem is that those channels share the same video pid.
For extended_cw there is a limit of 4 audio tracks, so when a feed has more than 4 audio tracks they create another channel with the same video pid but different audio pids. Sometimes audio pids are also used for multiple channels (especially English...)

"Animal Planet HD Russian" just uses a subset of the pids used for "Animal Planet HD"

IIRC I tried with different channels, e.g "Discovery Science HD" and "Discovery Showcase HD".

@Petar : It seems that @Joe Miller User is right. The problem is shared VPID (video PID) - oscam does not handle the ecm filter correctly in this case (it closes the ecmpid when one of the shared service is closed). But it should not be a problem for the standard usage (just use only one from channels with the shared VPID).

@nemal nemal k say say same problem with differend channels, at see same problem.

@nemal nemal k can you share log from tvh and oscam from your situation to seem bug.

hello dear good afternoon I'm trying to configure oscam and tvheadend someone to help me in my last configuration oscam emu and tvheadend 4.3 greetings