Bug #4077
Decryption Inconsistencies
0%
Description
Sorry, this is a bit of a unique problem.
On the Hispasat satellite (30W) there are several transponder containing feeds for the Spanish digital terrestrial network (11222H, 12548.5V, 12671V etc.) Each feed is a full assembled multi programme TS stream (ready to be fed into a DVB-T transmitter) contained within a single PID on the DVB-S2 transponder and encrypted with BISS (so a bit like a Russian doll, TS within a TS).
For example, on 12671V, PID 8006 contains a TS stream with the programmes La 1, La 1 HD, La 2, 24h, and Clan for the Catalonia region.
A few months ago, the version of TVHeadend I had installed was a version where the services were stored as files within directories for each multiplex. I created a fake service for 8006 and put the file in the 12671V directory. I set all of the PIDs (SID, PMT etc) in the file as 8006, and added a 'fake' H264 video PID of 8006. In TVHeadend I then added a CA entry for this service, and set the service to force CA as BISS (2600) so it uses this decryption.
Now when I stream the fake 8006 service through a Python script I have written to strip the outer TS packet headers, I am left with the decrypted data which is a valid transport stream containing the channels I have mentioned above. This works great, and since then I have updated TVHeadend to a newer version where the multiplexes and services aren't directories and files any more but just one GZIP archive for each mux, and it still works fine.
However, I wanted to add all of the other streams containing channels in the same way (more PIDs on other transponders). So yesterday to test this I reinstalled an older version of TVHeadend (on another hard drive in the same hardware setup) with the services still as files in folders rather than the new gzip version. I then added some fake services in the same way as 8006, e.g. 701, 702 and 703 on 12548V as dummy h264 streams with that PID in a fake service file.
These services all appeared in TVH and I added the CA info as before with the correct key. But, now when I run these services through my Python script I do get the actual valid TS content but the picture and sound break up as if the signal is low (the signal is fine). TVHeadend is reporting 'h264 continuity counter error' on the 701, 702 etc PIDs as it decrypts.
I can understand his because the data obviously is not an h264 stream (it's data), but I don't understand why it can ignore this on 8006 and decrypt it 'blind' without there being any errors but not do the same for the new services I have added. I have updated the TVHeadend version once I've added the fake services but the error still occurs. 8006 on my original set-up still works fine.
Is there any way of getting the CA to just decrypt the service without trying to validate the h264 data? Why is it working on one service and not others in the same way? I just want to be able to decrypt the data in each packet without TVHeadend knowing or caring what it contains.
Thanks!
Files
History
Updated by Jaroslav Kysela almost 8 years ago
Updated by Could you grab the original stream to a .ts file (use mux play url) and check it through the support/pid-count.py script for continuity errors for the problematic PID?
Also, you may make your life more easy - the mux config compression can be removed in src/settings.c:
diff --git a/src/settings.c b/src/settings.c
index 3718c48..b1c590c 100644
--- a/src/settings.c
+++ b/src/settings.c
@@ -166,6 +166,7 @@ hts_settings_save(htsmsg_t *record, const char *pathfmt, ...)
pack = 0;
#endif
ok = 1;
+ pack = 0;
if (!pack) {
htsbuf_queue_init(&hq, 0);
Updated by Adam W almost 8 years ago
Updated by 
- File Full TS 12548.5V.ts Full TS 12548.5V.ts added
- File PID 702 Decrypted VLC.ts PID 702 Decrypted VLC.ts added
One more thing to mention, the channels work perfectly if I decrypt using VLC instead of TVH, by streaming the entire transponder from TVHeadend, filtering the PID with ts_filter, decrypting with VLC, then stripping the packet headers with my own Python script. The problem is that only an old version of VLC will allow input and output via pipe, not newer versions for Ubuntu.
I have attached a copy of the perfect TS file I receive for PID 702 on 12.548V using this VLC method via a piped IPTV input in TVHeadend, and also the raw TS file for the whole 12548V transponder (including PIDs 701, 702 and 703) -
TVHeadend IPTV Mux: pipe:///home/adam/mpe2.sh
mpe2.sh Script:
#!/bin/bashcurl http://127.0.0.1:9981/stream/mux/41046c860303aec87c9c0d8ffc4def95?title=12548.5V%20%2F%20Hispasat | ts_filter 702 | cvlc - --ts-csa-ck=******** --ts-csa2-ck=******** --ts-dump-file=- | python3.4 /home/adam/livetsstrip.py
Thank you for your help Jaroslav, I will try the things you've suggested in the next couple of days and report back with some more info.
Updated by Adam W almost 8 years ago
This is the result of the PID count script for 12548V -
0000 ( 0) - 15 (err 0)
0001 ( 1) - 75 (err 0)
0010 ( 16) - 6 (err 0)
0014 ( 20) - 1 (err 0)
005B ( 91) - 220 (err 0)
005C ( 92) - 3786 (err 0)
02BD ( 701) - 101061 (err 0)
02BE ( 702) - 101061 (err 0)
02BF ( 703) - 101062 (err 0)
1645 (5701) - 60 (err 0)
1646 (5702) - 60 (err 0)
1647 (5703) - 60 (err 0)
1FA4 (8100) - 60 (err 0)
Updated by Jaroslav Kysela almost 8 years ago
You may use tvh's pid filter:
curl http://127.0.0.1:9981/stream/mux/41046c860303aec87c9c0d8ffc4def95?pids=702
Updated by Jaroslav Kysela almost 8 years ago
I am trying to reproduce the problem here, but I don't see any discontinuities when I force to descramble the pid 701:
$ wget -O a.ts http://localhost:9981/play/stream/service/648a82db4f6044123ebe3c1b86467437 $ ./support/pid-count.py a.ts 0000 ( 0) - 15 (err 0) 02BD ( 701) - 100224 (err 0) 1645 (5701) - 59 (err 0)
Updated by Adam W almost 8 years ago
Thanks for your help again Jaroslav.
I've built my own TVHeadend as you suggested with the mux compression removed, and now I can see the service details in each mux file.
This is the service I've added -
"c69550fef30c46838aaa5ebe4b77ae08": { "sid": 702, "lcn": 0, "lcn_minor": 0, "lcn2": 0, "srcid": 0, "svcname": "MPE2", "dvb_servicetype": 0, "dvb_ignore_eit": false, "prefcapid": 0, "prefcapid_lock": 1, "force_caid": 9728, "created": 1478951324, "last_seen": 1478951324, "enabled": true, "auto": 1, "priority": 0, "s_type_user": -1, "pcr": 702, "pmt": 702, "stream": [ { "pid": 702, "type": "H264", "position": 0 } ] },
Then a CA entry for it (DES) -
CA ID: 0x2600
Provider ID: 0xea60 (60000)
Transponder ID: 0xea67 (60007)
Service ID: 0x2be (702)
Even Key: [KEY]
Odd Key: [KEY]
But I'm still getting continuity counter errors -
2016-11-19 12:26:28.753 http: 127.0.0.1: using ticket C26156BBF976EFB4FB24EE807F07644F7571454E for /stream/mux/49a3352fb9e3caef4b235a29f5adafc9
2016-11-19 12:26:28.753 mpegts: pipe:///home/adam/mpe2.sh in TDT - tuning on IPTV
2016-11-19 12:26:28.762 spawn: Executing "/home/adam/mpe2.sh"
2016-11-19 12:26:28.763 subscription: 0022: "HTTP" subscribing to mux "pipe:///home/adam/mpe2.sh", weight: 10, adapter: "IPTV", network: "TDT", service: "Raw PID Subscription", hostname="127.0.0.1", client="VLC/2.2.2 LibVLC/2.2.2"
2016-11-19 12:26:28.767 spawn: Filtering pid 702
2016-11-19 12:26:28.772 mpegts: 12548.5V in Hispasat - tuning on TurboSight TBS 6908 DVBS/S2 frontend : DVB-S #0
2016-11-19 12:26:29.402 mpegts: 12548.5V in Hispasat - open PID 02be (702) failed, dupe sub (owner 0x55bf35457200)
2016-11-19 12:26:29.402 subscription: 0023: "HTTP" subscribing to service "Hispasat/12548.5V/MPE2", weight: 100, adapter: "TurboSight TBS 6908 DVBS/S2 frontend : DVB-S #0", network: "Hispasat", mux: "12548.5V", profile="pass", hostname="127.0.0.1", client="curl/7.47.0"
2016-11-19 12:26:30.590 TS: Hispasat/12548.5V/MPE2: H264 @ #702 Continuity counter error (total 1)
2016-11-19 12:26:30.590 spawn: PID 13 - packet incontinuity (packet 13276)- expected 2be, found 0e
2016-11-19 12:26:31.580 spawn: PID 6 - packet incontinuity (packet 26740)- expected 2be, found 04
2016-11-19 12:26:32.588 spawn: PID 9 - packet incontinuity (packet 40201)- expected 2be, found 03
2016-11-19 12:26:33.590 spawn: PID 4 - packet incontinuity (packet 53674)- expected 2be, found 08
2016-11-19 12:26:34.587 spawn: PID 9 - packet incontinuity (packet 67131)- expected 2be, found 05
2016-11-19 12:26:36.593 spawn: PID 4 - packet incontinuity (packet 94122)- expected 2be, found 08
2016-11-19 12:26:37.589 spawn: PID 12 - packet incontinuity (packet 107582)- expected 2be, found 05
2016-11-19 12:26:38.590 spawn: PID 5 - packet incontinuity (packet 121054)- expected 2be, found 0e
2016-11-19 12:26:39.599 spawn: PID 8 - packet incontinuity (packet 134520)- expected 2be, found 09
2016-11-19 12:26:41.591 TS: Hispasat/12548.5V/MPE2: H264 @ #702 Continuity counter error (total 10)
And so on.
It does this on all services I add (701, 703 etc).
How are you decrypting for it to not give errors?
I can send you the key in private if you need it.
Updated by Jaroslav Kysela almost 8 years ago
Adam Wisher wrote:
"pmt": 702,
This is wrong PID for PMT. There's no PMT table on 702. Use another PID (non-existent like 8000 or so).
2016-11-19 12:26:29.402 mpegts: 12548.5V in Hispasat - open PID 02be (702) failed, dupe sub (owner 0x55bf35457200)
This error is caused probably with the above mistake.
Updated by Jaroslav Kysela almost 8 years ago
Jaroslav Kysela wrote:
Adam Wisher wrote:
"pmt": 702,This is wrong PID for PMT. There's no PMT table on PID 702. Use another PID (non-existent like 8000 or so).
2016-11-19 12:26:29.402 mpegts: 12548.5V in Hispasat - open PID 02be (702) failed, dupe sub (owner 0x55bf35457200)
This error is caused probably with the above mistake.
Updated by Adam W almost 8 years ago
- File IMG_0935.PNG IMG_0935.PNG added
That's it! Thank you so much for your help.
I changed the PMT to 8000 and now the services decrypt with no problem. No break-up on any of the encapsulated channels! 
This must be the explanation for why my 8006 service worked. I must have set a different PMT value and not realised.
Updated by Adam W almost 7 years ago
These have now been working until updating to recent 4.3 unstable versions (4.3-912 and 4.3-943 both don't work).
Now TVHeadend will tune the PIDs, but the stream is not being decrypted (either by TVHeadend CW client or by putting the keys in OSCam). The IPTV stream that should contain the decrypted mux data comes back with bandwidth 0 (the tuned service on the DVB-S2 transponder shows bandwidth as expected, e.g. ~23000kbps), and the mapped channels come back with "No signal". OSCam shows the key as successfully found but it seems like TVHeadend is not applying it.
Sometimes the stream will work, seemingly at random, and then mostly it doesn't work at all any more.
It was all still working fine in a 4.3 version from around November 2017 - 4.3-762~gdf43bf0, but this is no longer in the repository to downgrade to.
I have installed 4.2 stable and it's all working perfectly again in this.
Has something changed with the way decryption works? I'm aware that my use case isn't standard, with "fake" h264 PIDs to trick TVHeadend/OSCam into decrypting the stream, but I just thought I'd mention it anyway as it all worked perfectly until recently.
Updated by Jaroslav Kysela almost 7 years ago
Create a new bug and add '--trace descrambler,capmt,cwc' logs..
Updated by Adam W almost 7 years ago
Thanks Jaroslav, I was just about to submit a ticket but I've upgraded to 4.3-956~g0f42197 and it seems to be working fine again! Along with the fixes to setting recordings (no longer crashing!)