Feature #3142
closed
Service descramble pass-through (was: Add SID and PIDs to the mux url of IPTV sources)
Added by B C over 9 years ago. Updated over 9 years ago.
100%
Description
With my effort to interface with OSEmus stream client for PowerVU, I'm now just one small step away from a semi perfect solution. Would it be possible to add the SID and all enumerated PIDs from the selected service as additional parameters to the query string of the mux?
Lets say I have currently the following mux URL:
http://server:someport/play/stream/mux/9d06e2e9989c2706a198579beab659bb?pids=0,17,18
and service with SID 4 uses the following pids:
0x05b4 / 1460 PCR 0x138c / 5004 PMT 1 0x1774 / 6004 CA CAIDS: 0e00:000000 2 0x05b4 / 1460 MPEG2VIDEO 3 0x058c / 1420 MPEG2AUDIO 4 0x058e / 1422 MPEG2AUDIO 5 0x0406 / 1030 MPEG2AUDIO 6 0x0726 / 1830 MPEG2AUDIO
so PIDs 5004,6004,1460,1420,1422,1030,1830 should be appended like eg &spids=5004,6004,1460,1420,1422,1030,1830
and the sid like eg &sid=4
with a final result of
http://server:someport/play/stream/mux/9d06e2e9989c2706a198579beab659bb?pids=0,17,18&spids=5004,6004,1460,1420,1422,1030,1830&sid=4
With a little patch for OSEmu this could be easily parsed and would make adding muxes with PowerVU a matter of seconds. If it is considered I would add a per mux option weather to append this info to the URL so it wouldn't break anything.
As always your thoughts on this would be very much appreciated
Updated by Jaroslav Kysela over 9 years ago
I'm a bit lost. The TVH does not need to know the difference between pids/spids, just add all pids to the pids= list to get them all to OSEmu or whatever else. The additional parameters should be ignored (spids, sid).
BTW: The service pids / CAIDs can be obtained programmatically:
# note: uuid is service uuid - see play link in the service grid wget -O a.json --post-data=uuid=fdfd19e440eeaec00087fbfce6003cf3 http://admin:admin@localhost:9981/api/service/streams # service settings wget -O a.json --post-data=uuid=fdfd19e440eeaec00087fbfce6003cf3 http://admin:admin@localhost:9981/api/idnode/load # services list wget -O a.json --post-data=limit=10000 http://admin:admin@localhost:9981/api/mpegts/service/grid # mux list wget -O a.json --post-data=limit=10000 http://admin:admin@localhost:9981/api/mpegts/mux/grid # network list wget -O a.json --post-data=limit=10000 http://admin:admin@localhost:9981/api/mpegts/network/grid
Updated by B C over 9 years ago
tvh does not need to know the pids, but to stream a whole TS instead of only a few pids is a lot of overhead. The idea is not to descramble the whole mux/transport stream but only one service of it.
Thank you for the examples, that would indeed work for the pids and SID, but how do I know the uuid which was requested, as the mux request does not include it (or am I blind again :-)? Could this peace of info be added like eg ?cuuid=fdfd19e440eeaec00087fbfce6003cf3
Updated by B C over 9 years ago
I did some tests with the complete TS and to my surprise dumping the remaining 35mbit has not really a big negativ effect on performance/cpu usage, but this will only work with devices supporting full TS streams.
Still I have no idea how to know the currently selected SID, descrambling the complete stream would defenitely be a waste of resources.
Updated by Jaroslav Kysela over 9 years ago
The problem is that I don't see your requrement. The /stream/mux url just gives the stream according the PID filters.. But where you need sid in your path ? In IPTV configuration ? I need to see the workflow. My original though is that you can gather all info from tvh using the HTTP API and configure IPTV according this info (where you have everything). Yes, there are no uuid for parents like service-->mux, but you may use mux name to manage the relations..
Updated by Jaroslav Kysela over 9 years ago
OK, I overlooked the requirement in the subject ;-( Anyway, I suggest to wait for https://tvheadend.org/issues/2993 - use HTTP API to gather all info and generate a .m3u file which will be read back by TVH (with the auto-maintenance of IPTV muxes / channels).
Updated by B C over 9 years ago
ok, I have defined an iptv mux to OSEmu (working with some tweaking) which includes as URL the mux of the TS where the encrypted services are broadcast
http://localhost:17999/play/stream/mux/9d06e2e9989c2706a198579beab659bb
this scans fine and evaluates to 39 Services (all encrypted). So whhen I now want to play a stream I need to tell OSEmu the SID of the channel so it findes the right PMT with ECM and ES-Pids. In the current Situation.
Current available solutions:¶
The "clean" (in terms of services) way¶
check what subscription was last added and parse my way with the api to the right service uuid.
with the UUID I can now get the SID and the PIDs.
Drawbacks:
there might have started another subscription in the meantime (in case of predictive tuning with pvr.hts very likely)
service names are not always unique even on the same TS.
The sevice flooding way¶
define a custom mux including SID and PIDs for every service (my current setup)
Drawbacks:
1482 (38*39) ghost services which do not work and spam the db, usability....
there must be a mux created for every service that is of interest manually
Possible solutions with the help of tvh¶
append the UUID of the service to the URL of the mux on the fly¶
with the UUID it is possible to get the SID and the PIDs
Drawbacks:
another roundtrip to tvh at least for the SID
if the traffic should be limited to the pids of interest another tune request with the right pids is necessary
append the SID to the URL of the mux on the fly¶
http://localhost:17999/play/stream/mux/9d06e2e9989c2706a198579beab659bb?sid=4
with the SID Decoding can start immediately
Drawbacks:
if the traffic should be limited to the pids of interest another tune request with the right pids is necessary
append SID and PID list to the URL of the mux on the fly¶
Streaming can start immediately with no overhead what so ever (no subsequent tuning request, no api calls....)
Conclusion¶
so yes it is possible with the current version, although it has drawback either in terms of setup/usability, performance or reliability. As the oiginal channel request code /which calls as a result iptv_input_start_mux later has SID and PID information directly available, passing these on to the mux URL would make it the fastest option to start the stream. It will fail if the selected service PMT was updated, but only for a single switch.
I hope you can follow me now :-)
Updated by B C over 9 years ago
Jaroslav Kysela wrote:
OK, I overlooked the requirement in the subject ;-( Anyway, I suggest to wait for https://tvheadend.org/issues/2993 - use HTTP API to gather all info and generate a .m3u file which will be read back by TVH (with the auto-maintenance of IPTV muxes / channels).
did not read this until after my post. Drawbacks with this is that you must manually sync the m3u file if a service changes and allthough I've never used bouquets before, single channels won't make it into the channel list. But anyway, thank you for all your efforts and the constant improvement of tvh.
Updated by Jaroslav Kysela over 9 years ago
B C wrote:
so yes it is possible with the current version, although it has drawback either in terms of setup/usability, performance or reliability. As the oiginal channel request code /which calls as a result iptv_input_start_mux later has SID and PID information directly available, passing these on to the mux URL would make it the fastest option to start the stream. It will fail if the selected service PMT was updated, but only for a single switch.
Oooh. I though that you'd like to add the sid from the source service not for the IPTV service (which has a valid value after scan, of course). You're right that these services IDs are same, because the stream is not remuxed. But... I don't think that it's the best solution in that case.
I hope you can follow me now :-)
Yes, yes... I just choose a more complicated way regarding this issue :-)
Anyway - I think that your design might be changed to keep the 1:1 (mux:service) mapping for IPTV.
Simply use mpeg-ts passthrough service streaming - http://host/stream/service/_service_uuid_?profile=pass&descramble=0
(Ensure that the selected profile has PMT/SDT rewrite on).
In this case, you get always get only one PMT in the requested MPEG-TS stream, so you can detect the service_id automatically directly.
Updated by Jaroslav Kysela over 9 years ago
- Status changed from New to Fixed
- % Done changed from 0 to 100
Applied in changeset commit:tvheadend|d5fabb89afa91b112e2fc9312b12c4f5f059d75d.
Updated by Jaroslav Kysela over 9 years ago
Note the descramble=0 functionality was added in the previous commit.
Updated by B C over 9 years ago
yeah, that way sounds great. As it is only named PMT/SDT rewrite, I assume also the PAT is with that only single service? The PID of the PMT is unchanged or always on the same PID (thinking of being lazy :-) )
Updated by Jaroslav Kysela over 9 years ago
Note that it's table content filter, so PIDs are not changed. PAT can be rewritten, too. Just check the pass profile config.
Updated by B C over 9 years ago
ok i think I got it, with PAT rewrite enabled PAT will only contain a single pointer to the PMT of the selected service.
Updated by B C over 9 years ago
help me one more time, I still do not have an idea how to get the service_uuid from the mux request to oscemu.
Updated by B C over 9 years ago
can you give me a hint where I find the uuid in function subscription_create
I'm a little bit lost :-)
Updated by B C over 9 years ago
Jaroslav Kysela wrote:
Simply use mpeg-ts passthrough service streaming - http://host/stream/service/_service_uuid_?profile=pass&descramble=0
I think the ecm stream is missing in the TS if I request a service like this
Updated by Jaroslav Kysela over 9 years ago
What's missing CAT PID ? Try v4.1-585-g284a95b .
Updated by B C over 9 years ago
no, cat is not needed, actually I still see not the ECM PID included
/play/stream/mux/9d06e2e9989c2706a198579beab659bb?pids=0,17,18,1560,1520,1522,1530,1532,5005,6005,
Index PID Type Language Details
0x0618 / 1560 PCR
0x138d / 5005 PMT
1 0x0618 / 1560 MPEG2VIDEO
2 0x05f0 / 1520 MPEG2AUDIO
3 0x05f2 / 1522 MPEG2AUDIO
4 0x05fa / 1530 MPEG2AUDIO
5 0x05fc / 1532 MPEG2AUDIO
6 0x1775 / 6005 CA CAIDS: 0e00:000000
/stream/service/9b3f60016e6199ea70e56dfa0c015890?profile=pass&descramble=0
0x0618 / 1560 PCR
0x138d / 5005 PMT
1 0x0618 / 1560 MPEG2VIDEO
2 0x05f0 / 1520 MPEG2AUDIO
3 0x05f2 / 1522 MPEG2AUDIO
4 0x05fa / 1530 MPEG2AUDIO
5 0x05fc / 1532 MPEG2AUDIO
Updated by Jaroslav Kysela over 9 years ago
How do you detect the second table ? If it's loopback to TVH, then TVH doesn't detect CA system (missing CAT table) without patch in v4.1-585-g284a95b .
What says support/pid-count.py utility on ts stream ?
Updated by B C over 9 years ago
both tables are from the tvh webinterface generated the same way, tvheadend --> osemu --> tvheadend --> dvbcard after scanning the mux with the two mentioned urls. Do you want me to run pid-count on the full ts?
Updated by B C over 9 years ago
this is from wget from /stream/service....
0000 ( 0) - 321 (err 0)
0011 ( 17) - 17 (err 0)
0012 ( 18) - 3867 (err 0)
05F0 (1520) - 3011 (err 0)
05F2 (1522) - 3011 (err 0)
05FA (1530) - 3010 (err 0)
05FC (1532) - 3011 (err 0)
0618 (1560) - 47365 (err 0)
138D (5005) - 642 (err 0)
so w/o the ECM pid
Updated by Jaroslav Kysela over 9 years ago
Do you have any CA filter active ? I receive ECM PIDs using /stream/service descramble=0 . Also, the CAT PID should be in the result stream (PID = 1) like:
0x00a2 / 162 PCR
0x0402 / 1026 PMT
1 0x0028 / 40 TELETEXT
2 0x0058 / 88 MPEG2AUDIO cze
3 0x005b / 91 MPEG2AUDIO qaa
4 0x0062 / 98 MPEG2AUDIO cze
5 0x00a2 / 162 MPEG2VIDEO
6 0x0f00 / 3840 CA CAIDS: 0d96:000000
7 0x0f01 / 3841 CA CAIDS: 0624:000000
8 0x0026 / 38 CA CAIDS: 0666:000000
CA filter for 0d96 results with:
0000 ( 0) - 18 (err 0) 0001 ( 1) - 18 (err 0) 0011 ( 17) - 5 (err 0) 0012 ( 18) - 70 (err 0) 0028 ( 40) - 159 (err 0) 0058 ( 88) - 193 (err 0) 00A2 ( 162) - 3009 (err 0) 0402 (1026) - 17 (err 0) 0F00 (3840) - 4 (err 0)
Updated by B C over 9 years ago
in your example I see only one audio stream and one ecm stream? You did the request on the service or the channel?
Updated by Jaroslav Kysela over 9 years ago
I requested service and I have set filters for both audio/CAIDs.
Updated by B C over 9 years ago
ok, some stupidness on my side in the mean time, still a little enhancement would be needed :-)
Allthough CAT is not needed (but good to have it), the CA descriptor pids are needed. Could these be added as well?
Updated by Jaroslav Kysela over 9 years ago
v4.1-593-g054fc2e - CA filter is applied, too . You should add emm=1 parameter to the service url..
Updated by B C over 9 years ago
ok, great thanks. One more thing is incomplete, the PMT is without the ECM ES info, sorry to bother you that much
Updated by B C over 9 years ago
to be more precise, the program_info loop is completely missing, which is carrying the CA descriptor
Updated by Petar Ivanov over 9 years ago
Are when use http://host/stream/service/_service_uuid_?profile=pass&descramble=0 descramble=0 remove all CAs including DES BISS ? I want when use descramble=0 to remove all CAs like play mux and to get same, but only for one Service
Updated by B C over 9 years ago
Petar Ivanov wrote:
Are when use http://host/stream/service/_service_uuid_?profile=pass&descramble=0 descramble=0 remove all CAs including DES BISS ? I want when use descramble=0 to remove all CAs like play mux and to get same, but only for one Service
you will get all pids from the service passed trough, if you need CAT and EMM stream, also add &emm=1 to the URL. Enable PAT and SDT rewrite in the corresponding passthrough profile.
Updated by Jaroslav Kysela over 9 years ago
- Subject changed from Add SID and PIDs to the mux url of IPTV sources to Service descramble pass-through (was: Add SID and PIDs to the mux url of IPTV sources)
I believe that the original requirement is satisfied.
Updated by B C over 9 years ago
can we let this open just to check why it stops for me in the middle or should I open a new ticket for it? I can play the stream for a short while, and then it stops if looped back to tvheadend. Connecting via VLC works just fine, so the passthrough part is ok. Currently analysing a trace all log
Updated by B C over 9 years ago
so I got a little bit further. While the stream is running:
2015-10-13 08:19:59.842 [ TRACE]:capmt: OSCAM Spaz (local): Received message from socket 32 2015-10-13 08:19:59.842 [ TRACE]:capmt: 40 3C 6F 2B 00 00 00 00 01 01 00 00 00 00 00 00 @<o+............ 2015-10-13 08:19:59.842 [ TRACE]:capmt: 00 00 00 00 00 00 00 00 00 FF 00 00 00 00 00 00 ................ 2015-10-13 08:19:59.842 [ TRACE]:capmt: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 2015-10-13 08:19:59.842 [ TRACE]:capmt: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 2015-10-13 08:19:59.842 [ TRACE]:capmt: 04 . 2015-10-13 08:19:59.842 [ TRACE]:capmt: OSCAM Spaz (local): setting filter: adapter=0, demux=0, filter=0, pid=1 2015-10-13 08:19:59.842 [ TRACE]:capmt: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 2015-10-13 08:19:59.842 [ TRACE]:capmt: FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 2015-10-13 08:19:59.842 [ TRACE]:capmt: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 2015-10-13 08:19:59.842 [ TRACE]:mpegts: table: mux 0x56448c0 add emm 00/00 (0) pid 0001 (1) 2015-10-13 08:19:59.842 [ TRACE]:descrambler: mux 0x56448c0 open pid 0001 (1) (flags 0x0000) for 0x573b250
and before it stops:
2015-10-13 08:23:15.393 [ TRACE]:capmt: OSCAM Spaz (local): Received message from socket 32 2015-10-13 08:23:15.393 [ TRACE]:capmt: 40 3C 6F 2B 00 00 00 17 75 80 00 00 00 00 00 00 @<o+....u....... 2015-10-13 08:23:15.393 [ TRACE]:capmt: 00 00 00 00 00 00 00 00 00 F0 00 00 00 00 00 00 ................ 2015-10-13 08:23:15.393 [ TRACE]:capmt: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 2015-10-13 08:23:15.393 [ TRACE]:capmt: 00 00 00 00 00 00 00 00 00 00 00 0B B8 00 00 00 ................ 2015-10-13 08:23:15.393 [ TRACE]:capmt: 04 . 2015-10-13 08:23:15.393 [ TRACE]:capmt: OSCAM Spaz (local): setting filter: adapter=0, demux=0, filter=0, pid=6005 2015-10-13 08:23:15.393 [ TRACE]:capmt: 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 2015-10-13 08:23:15.393 [ TRACE]:capmt: F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 2015-10-13 08:23:15.393 [ TRACE]:capmt: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 2015-10-13 08:23:15.393 [ TRACE]:mpegts: table: mux 0x56448c0 add ecm 00/00 (0) pid 1775 (6005) 2015-10-13 08:23:15.393 [ TRACE]:descrambler: mux 0x56448c0 open pid 1775 (6005) (flags 0x0100) for 0x573b250
and if i block the ecm pid with a ca filter on the loopback service everything stays fine.
Why is it sometimes pid 1 (the cat pid)? I would not know a peace of code in osemu changing that, but who knows :-)
Why does tvh decide it needs to decode the stream which arrives with the scrambling field cleared, just because there is an ecm stream?
Updated by Jaroslav Kysela over 9 years ago
This log is from the IPTV input or from the DVB input ? There should not be such messages for descramble=0 subscriptions.
Updated by B C over 9 years ago
this is the iptv side. The wrong pid (1) is something which only happens very rarely, I think it depends on how packages arrive when you start the stream caused by some old buffer. After 5 min or so tvh seems to reevaluate it and at this time it sees the correct pid and decides that it must decode, which should only happen if the av streams are indeed encrypted. I think that's the culprit
Updated by B C over 9 years ago
after setting the pid of the ecm stream to 0x1FFF (null packet) no more instabilities, so I think this feature request can be closed.
Thank you sooooooo much
Updated by Anonymous over 9 years ago
You already have a way to decode the channels using OSEmu? I am very interested in this issue. You share it on forum?
Updated by B C over 9 years ago
Janusze Biznesu wrote:
You already have a way to decode the channels using OSEmu? I am very interested in this issue. You share it on forum?