Tvheadend

Try this instead:

pipe:///usr/bin/ffmpeg -loglevel fatal -i http://INPUT -map 0:0 -map 0:1 -acodec copy -movflags +faststart -vcodec copy -ignore_unknown -metadata service_provider=PROVIDER -metadata service_name=CHANNEL -tune zerolatency -f mpegts pipe:1

Let us know how steady it is.

Mi Lano wrote:

Try this instead:

pipe:///usr/bin/ffmpeg -loglevel fatal -i http://INPUT -map 0:0 -map 0:1 -acodec copy -movflags +faststart -vcodec copy -ignore_unknown -metadata service_provider=PROVIDER -metadata service_name=CHANNEL -tune zerolatency -f mpegts pipe:1

Let us know how steady it is.

Hi,
I tried this line. Its not faster or anything. I don't think the changes in your line would have an effect anyway. The changed map specifies the first and second stream to map, what "-ignore_unknown" does already (-ignore_unknown leaves also a possible second audio inside). The copy flags have also the same effect as before. "-movflags +faststart" should only affect outputs of mp4 containers.

Thanks for your Answer, but my goal was to avoid the use of the ffmpeg pipe.

have you tried fixing the problem at the source? Enigma2 should not be sending descrambled streams with the scrambled flag set. Do you have:

Include ECM in http streams      unchecked
Descramble sending http streams    checked

Have you tried using a CA filter to block all CAIDs??? (under Configuration->Stream->CA stream filters)

Hi Joe,
the settings at the receiver are like you describe. So it's wired that the receiver sets the flag. I will ask about that problem at the openatv forum.

I tried also the CA filters and can see the filtering in the logs when I start a subscription.

2017-01-31 20:08:29.533 service: esfilter: "IPTV  PIPE/disneycine/{PMT:96}" CA 001 001 06659 ffff ffffffff IGNORE

2017-01-31 20:08:29.533 service: esfilter: "IPTV  PIPE/disneycine/{PMT:96}" CA 002 001 06915 ffff ffffffff IGNORE

2017-01-31 20:08:29.533 service: esfilter: "IPTV  PIPE/disneycine/{PMT:96}" CA 003 001 07939 ffff ffffffff IGNORE

But that changes nothing. I guess these filters are for the final streaming and not for the TVH mux to service translation. I tried to stream and also read the mux with the enabled filter. The filters appear in the logs, only when streaming.

Thanks for your ideas.

So you can play it with VLC as a mux but not a service/channel?

When you play in VLC, under statistics, does it show and data being received?

Are you able to record?

Did you check "Continue even if descrambling fails:" in your stream profile?

If you play the stream with VLC directly from your enigma2 box, does it show the stream as scrambled?

Have you tried enabling the SAT>IP server (tab under general) and set the descrambling limit to 0?

So you can play it with VLC as a mux but not a service/channel?

Yes.

When you play in VLC, under statistics, does it show and data being received?

As mux yes, normal data flow.
As service no, only some kb meta data.

Are you able to record?

No.

Did you check "Continue even if descrambling fails:" in your stream profile?

Yes, on every profile. There is also no TVH log entry with any descrambling fails.

If you play the stream with VLC directly from your enigma2 box, does it show the stream as scrambled?

Yes, the output is the same as from TVH (see my init post, first box) and it does also play the stream like playing the mux over TVH.

Have you tried enabling the SAT>IP server (tab under general) and set the descrambling limit to 0?

I have tried it now and it changed nothing.

To test SAT>IP server you need to use a SAT>IP client (RTSP) There are some free android clients (I personally never used) or DVBViewer for PC (free lite version will work.)

A debug log might be helpful...

Joe User wrote:

To test SAT>IP server you need to use a SAT>IP client (RTSP) There are some free android clients (I personally never used) or DVBViewer for PC (free lite version will work.)

A debug log might be helpful...

Well, I don't want to use RTSP. My goal is to get picture and audio on kodi (HTSP) and VLC (pass). (I use VLC only for testing if TVH is able to do anything with the encrypt flagged channels.)

Problem seams to be that my source (enigma2) sends the encrypted flag. Thats no TVH problem, but I can not override the flag in TVH services, its readonly.

Here is a debug enabled log from reading the mux and playing the service after.

Loglevel debug: enabled

2017-02-02 20:26:37.855 mpegts: disneycine in IPTV  PIPE - tuning on IPTV

2017-02-02 20:26:37.857 subscription: 001B: "scan" subscribing to mux "disneycine", weight: 5, adapter: "IPTV", network: "IPTV  PIPE", service: "Raw PID Subscription" 

2017-02-02 20:26:52.855 mpegts: disneycine in IPTV  PIPE scan complete

2017-02-02 20:26:52.855 subscription: 001B: "scan" unsubscribing

2017-02-02 20:26:52.856 mpegts: disabling service IPTV  PIPE/disneycine/{PMT:98} [sid 007F/127] (missing in PAT/SDT)

2017-02-02 20:26:52.856 mpegts: disabling service IPTV  PIPE/disneycine/{PMT:99} [sid 0072/114] (missing in PAT/SDT)

2017-02-02 20:26:52.856 mpegts: disabling service IPTV  PIPE/disneycine/{PMT:97} [sid 0070/112] (missing in PAT/SDT)

2017-02-02 20:26:52.856 mpegts: disabling service IPTV  PIPE/disneycine/{PMT:100} [sid 006E/110] (missing in PAT/SDT)

2017-02-02 20:27:17.262 http: HOST: using ticket E79230C97BFB9C8B8C93704E19A81C7E4C0D73F9 for /stream/service/6d6e61f0d59d77fa500560043f560f25

2017-02-02 20:27:17.264 subscription: 001C: "HTTP" subscribing to service "IPTV  PIPE/disneycine/{PMT:96}", weight: 100, adapter: "IPTV", network: "IPTV  PIPE", mux: "disneycine", profile="pass", hostname="HOST", client="VLC/2.2.4 LibVLC/2.2.4" 

2017-02-02 20:27:18.975 service: esfilter: "IPTV  PIPE/disneycine/{PMT:96}" CA 001 001 06659 ffff ffffffff IGNORE

2017-02-02 20:27:18.975 service: esfilter: "IPTV  PIPE/disneycine/{PMT:96}" CA 002 001 06915 ffff ffffffff IGNORE

2017-02-02 20:27:18.975 service: esfilter: "IPTV  PIPE/disneycine/{PMT:96}" CA 003 001 07939 ffff ffffffff IGNORE

2017-02-02 20:27:46.656 subscription: 001C: "HTTP" unsubscribing, hostname="HOST", client="VLC/2.2.4 LibVLC/2.2.4" 

The debug log shows nothing more then the normal log.

If someone can give me advice how to change the encrypted flag for a single service in the config over the file system, that would be nice. So I could test if that is the only problem.

Thanks for your time.

need the FULL log (read wiki about debugging) and without using the pipe and not streaming full mux.

Have you tried creating just a single IPTV network (not automatic) then create a mux for that network with a single channel (url for one channel from enigma2)?

In VLC , can you go to tools-> messages (CTRL-M) and set verbosity to 2 (debug) and then try to play the stream and upload the log?

How are you adding the url?

On mine I see:

[   INFO] subscription: 0019: "HTTP" subscribing to service "enigma2/http://192.168.1.12:8001/1:0:19:4333:300C:13E:820000:0:0:0:/{PMT:531}", weight: 100, adapter: "IPTV", network: "enigma2", mux: "testingenigma", profile="all", hostname="192.168.1.16", client="VLC/2.2.4 LibVLC/2.2.4" 

on yours:

[   INFO]:subscription: 002C: "HTTP" subscribing to service "test/disneyCinemagic/{PMT:96}", weight: 100, adapter: "IPTV", network: "test", mux: "disneyCinemagic", profile="pass", hostname="HOST", username="admin", client="VLC/2.2.4 LibVLC/2.2.4" 

Try adding by:
1. configuration-> DVB Inputs-> Networks-> add-> IPTV Network
2. add just name to configuration page (i.e. enigma2)
3. switch to configuration-> DVB Inputs-> Muxes - add -> name from step 2. (i.e enigma2)
4. add just the url (something like: http://192.168.1.12:8001/1:0:19:4333:300C:13E:820000:0:0:0 )

It should then shortly automatically scan the new mux, if not go to configuration-> DVB Inputs-> Networks and select the network (i.e. enigma2) and press "Force Scan".

Then back in Muxes, the "Scan status" of the mux should be idle and the "Scan result" should be ok.

Then in Services you should see at least one service for the network that should be enabled and "Automatic checking" should be "auto check enabled". There may be some other services which are not enabled with "Automatic checking" set to "Missing In PAT/SDT". (referring to other channels on the transponder which are not part of the stream...)

Then try to play that enabled service.

You are right Jon, there is a difference in the log entries. The difference is that I'm adding a service name "disneyCinemagic" to the mux, beside the url, and you not. All other steps you described are the same.

Do you use scrambled (descrambel from the enigma2 box) channels as mux? Can you play these channels? Is VLC Showing the "scrambled" flag "true" when you play the mux or the direct stream from the box?

If yes, yes and yes:
Whats the TVH Version you are using?

I tried adding a name and it did not show up, I guess it did not take affect immediately because it shows it now..

Yes I tested with a scrambled channel which was descrambled by enigma2 with oscam. Scrambled flags are removed from the stream. I guess it is some issue with your CI+ module????

Maybe you can try a tracelog of httpc,tbl-base,service,subscription,descrambler,dvbcam,mpegts,iptv-pcr

Mine:

2017-02-06 21:47:54.247 [  DEBUG] service: enigma2/testmux/testname: Status changed to [Hardware input]
2017-02-06 21:47:54.247 [  DEBUG] service: enigma2/testmux/testname: Status changed to [Hardware input] [Input on service]
2017-02-06 21:47:54.247 [  DEBUG] service: enigma2/testmux/testname: Status changed to [Hardware input] [Input on service] [Demuxed packets]
2017-02-06 21:47:54.247 [  DEBUG] service: enigma2/testmux/testname: Status changed to [Hardware input] [Input on service] [Demuxed packets] [Reassembled packets]

2017-02-06 20:59:02.898 [  DEBUG]:service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] 
2017-02-06 20:59:02.898 [  DEBUG]:service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] 
2017-02-06 20:59:02.898 [  DEBUG]:service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] [No access] 
2017-02-06 20:59:02.904 [  DEBUG]:service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] [Demuxed packets] [No access] 
2017-02-06 20:59:02.904 [  DEBUG]:service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] [Demuxed packets] [Reassembled packets] [No access] 

Not sure what "No access" means, I would have to look at the source code and I do not have time tonight. Maybe someone else knows...

Other difference is that I do have a CA setup and since there are CAIDs present, it does try to find keys even though the stream is already decrypted. (btw - it does not find keys since my CA is not setup for the channel being streamed from enigma2.)

Maybe you can try setting up a fake CA?

Configuration-> CAs-> add-> CAPMT Give some client name and click on enabled. Leave other default settings and press create.

Then try the stream again (with the trace) and see if it works, or at least gives more info.

BTW - You will see continuous errors in log about not being able to connect.

Also, when you play the stream directly in VLC (without tvh) does it start immediately or is there a short delay. (Check messages and verbosity set to 2 - debug to see if some scrambled packets are sent at the beginning.)

And have you tried streaming a channel that was currently being played on the stb (already being decrypted...)?

If you force CAID for the service to 0xffff, TVH won't do any descrambling. v4.1-2419-gf156eb9

Jaroslav Kysela wrote:

If you force CAID for the service to 0xffff, TVH won't do any descrambling. v4.1-2419-gf156eb9

Is the fix contained in the nightlies for xenial or do I have to build it my self?
apt-get update/upgrade won't load a new version.

Paul Brunniem wrote:

Jaroslav Kysela wrote:

If you force CAID for the service to 0xffff, TVH won't do any descrambling. v4.1-2419-gf156eb9

Is the fix contained in the nightlies for xenial or do I have to build it my self?
apt-get update/upgrade won't load a new version.

This is because the build scripts are broken right now. So, unfortunately yes, you'll have to build it.

Fixed in v4.1-2426-gef89ef8

ts debug: new PMT program number=111 version=27 pid_pcr=255
ts debug: * es pid=255 type=27 dr->i_tag=0x52
ts debug: * es pid=255 type=27 fcc=h264
core debug: selecting program id=111
ts debug: * es pid=259 type=6 dr->i_tag=0xa
ts debug: * es pid=259 type=6 dr->i_tag=0x52
ts debug: * es pid=259 type=6 dr->i_tag=0x6a
ts debug: * Stream Component Identifier: 7
ts debug: found language: deu
ts debug: * es pid=259 type=6 fcc=a52 
ts debug: * es pid=260 type=6 dr->i_tag=0xa
ts debug: * es pid=260 type=6 dr->i_tag=0x52
ts debug: * es pid=260 type=6 dr->i_tag=0x6a
ts debug: * Stream Component Identifier: 8
ts debug: found language: eng
ts debug: * es pid=260 type=6 fcc=a52 
core debug: using demux module "ts" 

The pids listed in vlc are different than in tvh - do you have some of the rewrite options selected in you default profile? (or specificed profile for the user) If so, can you try without any rewrite options selected?

Also it seems vlc gives up too quickly, or are you manually stopping the playback?

In your VLC log I still see the wrong pids, did you do it with all the rewrites unchecked?

Joe User wrote:

Then in Services you should see at least one service for the network that should be enabled and "Automatic checking" should be "auto check enabled". There may be some other services which are not enabled with "Automatic checking" set to "Missing In PAT/SDT". (referring to other channels on the transponder which are not part of the stream...)

Then try to play that enabled service.

I tried playing one of the other services for the mux ( one with "Missing In PAT/SDT") and it shows the same behavior you are seeing. Are you making sure to select the channel with "Automatic checking" should be "auto check enabled"??

In the status->stream tab, what does it show for the bandwidth, and what does it show in status->subscriptions when you are streaming?

BTW - my tvheadend info for the channel also shows it is encrypted (probably because ECM info there) but VLC does not show scrambled.

Never mind about wrong pids, I see now you are using a different channel than the original post...

Can you save a short stream and upload it somewhere?

something like this:

 curl http://192.168.1.11:8001/1:0:19:4333:300C:13E:820000:0:0:0: -o test.ts -m 15

Another thing to try: try adding "?descramble=0" to the end of the tvheadend url and see if that helps...

http://USER:PASSWORD@tvhHost:9981/stream/service/4645484bb69606acc77e1e133f6b4f65?descramble=0

I can confirm that "descramble=0" works.

One more thing, after mapping the service to a channel, it plays fine without "descramble=0" and even with the CA client stopped.

Paul Brunniem wrote:

Hi,
unfortunately it does not work for me.
I made the build with the source with this steps:
build: https://tvheadend.org/projects/tvheadend/wiki/Building
start: https://tvheadend.org/projects/tvheadend/wiki/Development

I re-scanned the mux and set the service setting "Force CA ID (e.g. 0x2600):" to "0xffff".

When I play with VLC is does not work.
- no video/audio
- codec info as before
- statistics info as before (no data)

But, the TVH log does not show "no access" anymore -> little success.

It looks like that the MPEG-TS packets are marked as scrambled. Otherwise, I cannot explain this behaviour.

See: https://github.com/tvheadend/tvheadend/blob/master/src/descrambler/descrambler.c#L694

The dr->dr_external should be set when CAID is 0xffff, thus the packets are passed to the next layer, but only when the scrambled bit in the MPEG-TS packet is zero.

Jaroslav Kysela wrote:

It looks like that the MPEG-TS packets are marked as scrambled. Otherwise, I cannot explain this behaviour.

Yes, I am pretty sure this is the case (reason VLC reports stream as scrambled.) That is why I asked for sample to verify this is the case...

Jaroslav Kysela wrote:

The dr->dr_external should be set when CAID is 0xffff, thus the packets are passed to the next layer, but only when the scrambled bit in the MPEG-TS packet is zero.
See: https://github.com/tvheadend/tvheadend/blob/master/src/descrambler/descrambler.c#L694

Yes, that is the code I didn't have time to track down. - thanks!

Where does it get overridden to send the scrambled packets? (descramble=0)

I am also not sure of the internal differences when using an IPTV stream compared to a tuner?

Joe User wrote:

Never mind about wrong pids, I see now you are using a different channel than the original post...

Sorry for that. I changed the channel because the old one does not send after 8 p.m.

Can you save a short stream and upload it somewhere?

Sure, I used the following:
curl http://receiver:8001/1:0:19:6F:D:85:C00000:0:0:0: -o test.ts -m 15

uploaded here:
http://upfile.mobi/Dasz6jvp9b0

Another thing to try: try adding "?descramble=0" to the end of the tvheadend url and see if that helps...

That doesn't change anything for me.

Jaroslav Kysela wrote:

It looks like that the MPEG-TS packets are marked as scrambled. Otherwise, I cannot explain this behaviour.

Yes, I am pretty sure this is the case (reason VLC reports stream as scrambled.) That is why I asked for sample to verify this is the case...

Jaroslav Kysela wrote:

The dr->dr_external should be set when CAID is 0xffff, thus the packets are passed to the next layer, but only when the scrambled bit in the MPEG-TS packet is zero.
See: https://github.com/tvheadend/tvheadend/blob/master/src/descrambler/descrambler.c#L694

Yes, that is the code I didn't have time to track down. - thanks!

Where does it get overridden to send the scrambled packets? (descramble=0)

I am also not sure of the internal differences when using an IPTV stream compared to a tuner?

Is there a chance to get this to work by building TVH without the descrambling module? ("--disable-cwc" or something?)

Compiling with cwc disabled does not work, TVH does not start after that.

Jaroslav Kysela wrote:

See: https://github.com/tvheadend/tvheadend/blob/master/src/descrambler/descrambler.c#L694

The dr->dr_external should be set when CAID is 0xffff, thus the packets are passed to the next layer, but only when the scrambled bit in the MPEG-TS packet is zero.

I changed the line 695 from:

if ((tsb[3] & 0x80) == 0) {

to:

if ((tsb[3] & 0x80) == 0 || 1 == 1) {

compiled again, startet TVH, played the service, and...
works! Got the video/audio in less than a second.

Joe User wrote:

Another thing to try: try adding "?descramble=0" to the end of the tvheadend url and see if that helps...

[...]

Looking at the code, it should bypass the descrambler and just send the packets.

[[https://github.com/tvheadend/tvheadend/blob/master/src/input/mpegts/tsdemux.c#L223]]

"s_scrambled_pass" is set when you add the option "?descrambler=0" to the url request to tvheadend (not url to enigma2...)

I used this to force the sending of scrambled streams to oscam streamrelay and it does work.

For kodi, not sure what you can do, would have to ask on kodi forums.

Yes, cwc is for newcamd connection.

Joe User wrote:

Looking at the code, it should bypass the descrambler and just send the packets.

OK you are right, I made a mistake.

I first tried with the URL param and the link I got from the service tab (also ticket param inside).
But with clean URL it works with the service (I of course rejected the hack I made to the source).

http://receiver:9981/stream/service/4645484bb69606acc77e1e133f6b4f65?descramble=0

But the channel URL does not work.

http://receiver:9981/stream/channel/2c8e588a4a1dc6c34c443dfaa914911a?descramble=0

VLC shows no video/audio and TVH does not show any error.

Other question: If the param descramble=0 works, how can this help me to play the channels in Kodi?

Joe User wrote:

For kodi, not sure what you can do, would have to ask on kodi forums.

Why do you think that Kodi is the problem? TVH seams to have the problem with the input data.

the log says:

2017-02-21 21:33:50.044 [WARNING]:subscription: 000D: service instance is bad, reason: No input detected

Yes, I looked in "src/webui/webui.c" and it does not parse the descramble option for channels, but it could be added.

But.... it will still be sending unscrambled packets falsely flagged as scrambled and I do not know how kodi will handle that.

Aside from the ffmpeg/pipe, you would have to make a hack in tvheadend to clear the scrambled flags for every packet. Probably not simple, but should be doable. Other path is to try to fix the problem at the source. I am not sure which stb you have and what image you are running, although it may not matter if the problem is with the CI+ module (which I expect is the case - the descrambler should clear the flag...)

BTW - why do you not just use the VU+/Enigma2 add-on for kodi to access your stb directly?

Joe User wrote:

Yes, I looked in "src/webui/webui.c" and it does not parse the descramble option for channels, but it could be added.

Would this affect the output of the htsp connections? As I understand the URL param dose only change the behavior of the http streams.

But.... it will still be sending unscrambled packets falsely flagged as scrambled and I do not know how kodi will handle that.

I will test this tonight with the enigma2 plugin in Kodi. So Kodi should get the same as TVH when playing direct from the box.

I am not sure which stb you have and what image you are running...

Its a Megasat force 2 with openatv 5.3 running. I asked in the openatv forum but no answers until now.

BTW - why do you not just use the VU+/Enigma2 add-on for kodi to access your stb directly?

I have more than 1 Client and want to use the great TVH flexibility in input sources/recording/EPG...
The receiver will only be a cheep source giving descrambled DVB-S.

Paul Brunniem wrote:

Ok so VLC works with the little hack, but htsp (Kodi) does not.
TVH runs in timeout with "no input detected". (see attachment)

Remove

  if (tsb2[3] & 0xc0) /* scrambled */
    continue;

condition in src/input/mpegts/tsdemux.c in ts_recv_packet0().

But anyway, your stream don't follow any specification / rules - it is just broken, so these changes cannot be in the tvh upstream code.

Or replace it with:

tsb2[3] &= 0x3F;

which will clear the scrambled flag.

Joe User wrote:

Or replace it with:
[...]

which will clear the scrambled flag.

That does not work because its only readable.

Jaroslav Kysela wrote:

Remove

[...]

condition in src/input/mpegts/tsdemux.c in ts_recv_packet0().

I removed it and it works now with HTTP stream (without the URL param) and htsp in Kodi.

But anyway, your stream don't follow any specification / rules - it is just broken, so these changes cannot be in the tvh upstream code.

What prevents you from using the same flag ("The dr->dr_external should be set when CAID is 0xffff") from the service in the channels?

Anyway, it works now. If that means that I have to build TVH my self and modify a few lines when I update, than it's ok for me.

Paul Brunniem wrote:

But anyway, your stream don't follow any specification / rules - it is just broken, so these changes cannot be in the tvh upstream code.

What prevents you from using the same flag ("The dr->dr_external should be set when CAID is 0xffff") from the service in the channels?

There are basically two expectations for the stream:

1) don't descramble stream at all - for SAT>IP server or special external descrambler (HTTP parameter), the data are passed AS-IS
2) external descrambling (hardware CAM modul) - usually, the hardware CAM modules don't descramble first few MPEG-TS packets, so they should be skipped (they have the scrambling bits set)

It's the difference between s->s_scrambled_pass and dr->dr_external. TVH also should not parse the A/V packets when the scrambling bits are set (garbage).

Thinking more about your problem, the scrambled flag in the MPEG-TS packets should be cleared in the input code before any processing.

Added 'Remove scrambled bits' to the IPTV network settings - v4.1-2445-g3525a9c .

Works like a charm. Thank you all very much for your help (it was greatly appreciated). Also I learned many new things.

Paul Brunniem wrote:

That does not work because its only readable.

Sorry, I was in a hurry and did not notice the ts buffer is passed around as a const.

I was thinking you could make a constant code word entry and then in tvhcsa.c check for caid 0xffff and if so just clear the scrambled bits... but Jaroslav already made a fix.

@Jaroslav, do you think this could be part of the problems in this issue: [[https://tvheadend.org/issues/2794]]???