Project

General

Profile

Actions
Copy link

Feature #3182

closed

Custom user agent for automatic iptv (m3u)

Feature #3182: Custom user agent for automatic iptv (m3u)

Added by Njuskalo Njusko over 10 years ago. Updated over 10 years ago.

Status:
Fixed
Priority:
Normal
Assignee:
-
Category:
IPTV
Target version:
4.2
Start date:
2015-10-20
Due date:
% Done:

100%

Estimated time:

Description

Sometimes, a custom user agent is needed for playing iptv streams. When using pipe/ffmpeg, a command line such as pipe://ffmpeg -user-agent Kodi/14.1 ... was sufficient.

It would be nice if network or mux menu had a selectable or custom user agent string available.

Files

Download all files
trace.log (8.97 KB) trace.log Njuskalo Njusko, 2015-10-29 20:42
trace1.log (9.25 KB) trace1.log Njuskalo Njusko, 2015-10-29 20:58
trace4.log (10.4 KB) trace4.log Njuskalo Njusko, 2015-10-30 09:22
trace5.log (9.47 KB) trace5.log Njuskalo Njusko, 2015-10-30 15:26
trace6.log (9.3 KB) trace6.log Njuskalo Njusko, 2015-10-30 21:24
trace7.log (9.47 KB) trace7.log Njuskalo Njusko, 2015-10-30 22:04
trace8.log (9.47 KB) trace8.log Njuskalo Njusko, 2015-10-30 23:10
trace9.log (9.47 KB) trace9.log Njuskalo Njusko, 2015-10-30 23:15

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #1

  • Target version set to 4.2

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #2

  • Status changed from New to Fixed
  • % Done changed from 0 to 100

Applied in changeset commit:tvheadend|be7381f1d15c12e6d2a5790145d8cb81f7d7d044.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #3

Fixed in v4.1-789-gbe7381f . Syntax for 'Custom HTTP header' is:

User-Agent: my user agent

Note that any HTTP header can be added (one per line - multiple lines are parsed).

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #4

Jaroslav Kysela wrote:

Fixed in v4.1-789-gbe7381f . Syntax for 'Custom HTTP header' is:

User-Agent: my user agent

Note that any HTTP header can be added (one per line - multiple lines are parsed).

Perfect, many thanks. Will try and report.

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #5

Njuskalo Njusko wrote:

Jaroslav Kysela wrote:

Fixed in v4.1-789-gbe7381f . Syntax for 'Custom HTTP header' is:

User-Agent: my user agent

Note that any HTTP header can be added (one per line - multiple lines are parsed).

Perfect, many thanks. Will try and report.

Not working.

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #6

Tried with stream that blocks Kodi, User-Agent: Kodi/14.1, but the stream plays normally

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #7

Njuskalo Njusko wrote:

Tried with stream that blocks Kodi, User-Agent: Kodi/14.1, but the stream plays normally

The above was tested with pipe://

Now I tried with simple http:// link and User-Agent: Mozilla/5.0 but it failed as well.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #8

Try v4.1-794-g3e1e39c . The problem was that the default User-Agent was not removed, so the request contained two User-Agent headers.

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #9

Jaroslav Kysela wrote:

Try v4.1-794-g3e1e39c . The problem was that the default User-Agent was not removed, so the request contained two User-Agent headers.

Still doesn't work.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #10

Show me '--trace httpc' .

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #11

Jaroslav Kysela wrote:

Show me '--trace httpc' .

Please find attached

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #12 

2015-10-29 20:14:48.843 [  TRACE]:httpc: 47 45 54 20 2F 74 73 2F 48 52 54 32 2F 73 74 72 GET /ts/HRT2/str
2015-10-29 20:14:48.843 [  TRACE]:httpc: 65 61 6D 5F 77 69 66 69 5F 31 32 30 30 6B 2E 6D eam_wifi_1200k.m
2015-10-29 20:14:48.843 [  TRACE]:httpc: 33 75 38 20 48 54 54 50 2F 31 2E 31 0D 0A 48 6F 3u8 HTTP/1.1..Ho
2015-10-29 20:14:48.843 [  TRACE]:httpc: 73 74 3A 20 63 64 6E 2E 69 70 74 76 2E 61 6D 69 st: cdn.iptv.ami
2015-10-29 20:14:48.843 [  TRACE]:httpc: 73 2E 6E 65 74 0D 0A 43 6F 6E 6E 65 63 74 69 6F s.net..Connectio
2015-10-29 20:14:48.843 [  TRACE]:httpc: 6E 3A 20 63 6C 6F 73 65 0D 0A 55 73 65 72 2D 41 n: close..User-A
2015-10-29 20:14:48.843 [  TRACE]:httpc: 67 65 6E 74 3A 20 4D 6F 7A 69 6C 6C 61 2F 35 2E gent: Mozilla/5.
2015-10-29 20:14:48.843 [  TRACE]:httpc: 30 0D 0A 0D 0A                                  0....           

GET /ts/HRT2/stream_wifi_1200k.m3u8 HTTP/1.1
Host: cdn.iptv.amis.net
Connection: close
User-Agent: Mozilla/5.0

Seems ok ? But the m3u file is not parsed. Show me the contents of it..

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #13

Jaroslav Kysela wrote:

[...]

Seems ok ? But the m3u file is not parsed. Show me the contents of it..

#EXTM3U
#EXT-X-VERSION:4
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:1349183
#EXTINF:10,
/ts/HRT2/2015-10-29/1210211/HRT2_wifi_1200k-1349183.ts
#EXTINF:10,
/ts/HRT2/2015-10-29/1210211/HRT2_wifi_1200k-1349184.ts
#EXTINF:10,
/ts/HRT2/2015-10-29/1210211/HRT2_wifi_1200k-1349185.ts
#EXTINF:10,
/ts/HRT2/2015-10-29/1210211/HRT2_wifi_1200k-1349186.ts
#EXTINF:10,
/ts/HRT2/2015-10-29/1210211/HRT2_wifi_1200k-1349187.ts

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #14

Try v4.1-796-g2e92cca . I added application/x-mpegurl as valid mime type for m3u.

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #15

Jaroslav Kysela wrote:

Try v4.1-796-g2e92cca . I added application/x-mpegurl as valid mime type for m3u.

Thanks. 4.1-796~g2e92cca produces an error. Please see the log.

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #16

Njuskalo Njusko wrote:

Jaroslav Kysela wrote:

Try v4.1-796-g2e92cca . I added application/x-mpegurl as valid mime type for m3u.

Thanks. 4.1-796~g2e92cca produces an error. Please see the log.

Configured without --enable-libffmpeg_static and it works now.

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #17

Posted in wrong issue thread. Please disregard the above.

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #18

Njuskalo Njusko wrote:

Njuskalo Njusko wrote:

Jaroslav Kysela wrote:

Try v4.1-796-g2e92cca . I added application/x-mpegurl as valid mime type for m3u.

Thanks. 4.1-796~g2e92cca produces an error. Please see the log.

Configured without --enable-libffmpeg_static and it works now.

Still doesn't work.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #19

Could you try v4.1-798-g8185796 ?

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #20

Jaroslav Kysela wrote:

Could you try v4.1-798-g8185796 ?

Crashed. Please see the attached --trace httpc log.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #21

Try v4.1-803-g48f04e3 .

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #22

Jaroslav Kysela wrote:

Try v4.1-803-g48f04e3 .

Still not working. Please see the log.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #23

v4.1-805-g08e57f0

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #24

Jaroslav Kysela wrote:

v4.1-805-g08e57f0

4.1-808~g2cab577

Still not working.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #25

v4.1-809-gcb72dbf

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #26

Jaroslav Kysela wrote:

v4.1-809-gcb72dbf

Not working.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #27

trace7 is from 808

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #28

Jaroslav Kysela wrote:

trace7 is from 808

How so ?

2015-10-30 22:02:44.007 [ NOTICE]:START: HTS Tvheadend version 4.1-809~gcb72dbf started, running as PID:2780 UID:1000 GID:1000, CWD:/home/tommy/tvheadend CNF:/home/tommy/.hts/tvheadend

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #29

Jaroslav Kysela wrote:

trace7 is from 808

4.1-811~g4900abd

Not working as well.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #30

I'm sorry - working on many things together. I need break. Try v4.1-812-ge85a3c7 .

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #31

Jaroslav Kysela wrote:

I'm sorry - working on many things together. I need break. Try v4.1-812-ge85a3c7 .

Didn't work with v4.1-812-ge85a3c7

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #32

v4.1-820-g496a1b9

Updated by Njuskalo Njusko over 10 years ago Actions
Copy link
 #33

Jaroslav Kysela wrote:

v4.1-820-g496a1b9

The stream can be played now, great work, thanks.

The syntax for custom HTTP headers supports escape chars, i.e. 'Kodi/14.1' ?

Updated by Mario D over 10 years ago Actions
Copy link
 #34

Is it supposed to work a channels like the following with both the headers in auto IPTV net?

#EXTINF:-1 tvg-id="0003" group-title="Deutsch" tvg-logo="0004.png",[COLOR orangered]ZDF HD[/COLOR]
http://wownet.ro/str/0004.m3u8|X-Forwarded-For=81.17.18.50&User-Agent=Mozilla/5.0 (Windows NT 10.0; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0

Do I need escaping? It doesn't stream for me.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #35

@Mario: Try v4.1-826-g902f307 (if it does not work - show what TVH has in the custom HTTP headers field in mux).
@Njuskalo: TVH parses whole strings until the delimiter is found (space or | ). No escaping. I'm following these examples: http://xmtvplayer.com/build-m3u-file .

Updated by Mario D over 10 years ago Actions
Copy link
 #36

@Jaroslav Kysela: it continues to fail for me on scan.
The content of the 'custom HTTP headers' in mux is like:

X-Forwarded-For=81.17.18.50&User-Agent=Mozilla/5.0 (Windows NT 10.0; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0

It is the supposed format?

It could help to test a similar m3u list on your system: http://pastebin.com/V4YiW8Zv
The listed channels all worked in a test few minutes ago on kodi using simple IPTV add-on.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #37

@Mario: Try v4.1-827-g55aa465 . The lines should be (without &):

X-Forwarded-For=81.17.18.50
User-Agent=Mozilla/5.0 (Windows NT 10.0; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0

Updated by Mario D over 10 years ago Actions
Copy link
 #38

Ok, now the headers in the field have the expected format.
The mux scan still end with a "FAILED" status, but the service and the channel are created. They work as expected: is it normal that the mux has "FAILED" status?
All the channels in the pastebin list work.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #39

I need to see debug logs - there should be lines with 'complete' and 'incomplete' for DVB service information tables at the end of the scan procedure. Something like:

2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 1D4C (7500) pmt complete
2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 1E14 (7700) pmt complete
2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 1DB0 (7600) pmt complete
2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 1BBC (7100) pmt complete
2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 1B58 (7000) pmt complete
2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 012C (300) pmt complete
2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 00C8 (200) pmt complete
2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 19C8 (6600) pmt complete
2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 0064 (100) pmt complete
2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 0011 (17) sdt complete
2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 0010 (16) nit complete
2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 0001 (1) cat complete
2015-11-01 17:58:09.324 [  DEBUG]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network - 0000 (0) pat complete
2015-11-01 17:58:09.324 [   INFO]:mpegts: Multiplex [onid:233A tsid:4085] in TSfile Network scan complete

Updated by Mario D over 10 years ago Actions
Copy link
 #40

Forcing scan on the provided ZDF mux:

nov 01 18:12:30 sauron tvheadend[3756]: mpegts: test iptv - adding mux prova.m3u - ZDF in test iptv to scan queue weight 6 flags 4000
nov 01 18:12:30 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - add raw service
nov 01 18:12:30 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - tuning on IPTV
nov 01 18:12:30 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - started
nov 01 18:12:30 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - open PID 0012 (18) [20/0x122da100]
nov 01 18:12:30 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - open PID tables subscription [0042/0x122e0ba0]
nov 01 18:12:30 sauron tvheadend[3756]: subscription: 0134: "scan" subscribing to mux "prova.m3u - ZDF", weight: 6, adapter: "IPTV", network: "test iptv", service: "Raw PID Subscription" 
nov 01 18:12:33 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - open PID 0000 (0) [20/0x7f0168003ad0]
nov 01 18:12:33 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - open PID 0001 (1) [16/0x7f0168004f40]
nov 01 18:12:33 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - open PID 0010 (16) [16/0x7f016800bd50]
nov 01 18:12:33 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - open PID 0011 (17) [20/0x7f016800d1c0]
nov 01 18:12:33 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - open PID 0011 (17) [16/0x7f016800e630]
nov 01 18:12:33 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - open PID 0FFF (4095) [16/0x7f01500013f0]
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - close PID 0FFF (4095) [16/0x7f01500013f0]
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - close PID 0011 (17) [20/0x7f016800d1c0]
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - close PID 0010 (16) [16/0x7f016800bd50]
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - close PID 0001 (1) [16/0x7f0168004f40]
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - 0000 (0) pat incomplete
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - scan no data, failed
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - close PID tables subscription [0042/0x122e0ba0]
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - stopping mux
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - close PID 0000 (0) [20/0x7f0168003ad0]
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - close PID 0011 (17) [16/0x7f016800e630]
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: prova.m3u - ZDF in test iptv - close PID 0012 (18) [20/0x122da100]
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: test iptv - removing mux prova.m3u - ZDF in test iptv from scan queue
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: test iptv - adding mux prova.m3u - ZDF in test iptv to scan queue weight 6 flags 4000
nov 01 18:12:45 sauron tvheadend[3756]: mpegts: test iptv - removing mux prova.m3u - ZDF in test iptv from scan queue

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #41

Something is wrong. The PAT table is mandatory - do you see any incoming bandwidth during scan ?

Updated by Mario D over 10 years ago Actions
Copy link
 #42

During a force scan on the ZDF mux, I can't see any bandwidth activities on status->subscriptions. But if I do 'curl http://url.mux' I can see data.
Channels on http://pastebin.com/V4YiW8Zv are publicly available: you could test them if you like.

Updated by Jaroslav Kysela over 10 years ago Actions
Copy link
 #43

I did a check and I see the problem. The provider uses hls/m3u to serve TS segments of 10 seconds each. I'm sorry, but this is not supported. Each stream has only one PAT packet and TVH requires two at minimum to mark the PAT table as good and also, TVH actually expects continuous TS stream - the code does not walk through all m3u segments and then it should probably refetch new m3u list to continue.

Actions
Copy link

Also available in: PDF Atom