IPTV scan subscribing generates too many spawn: Executing "/opt/bin/ffmpeg" on RT-AC68U router.
IPTV scan subscribing generates too many spawn: Executing "/opt/bin/ffmpeg" on RT-AC68U router.Replies (7)
RE: IPTV scan subscribing generates too many spawn: Executing "/opt/bin/ffmpeg" on RT-AC68U router.
-
Added by saen acro over 5 years ago
Are you transcode externally?
It seems as xacell used without correct output network.
RE: IPTV scan subscribing generates too many spawn: Executing "/opt/bin/ffmpeg" on RT-AC68U router.
-
Added by DS Kim over 5 years ago
saen acro wrote:
Are you transcode externally?
It seems as xacell used without correct output network.
Thank you for your reply.
I'm using ffmpeg pipe for mux input.
pipe:///opt/bin/ffmpeg -loglevel fatal -i http://USERID:PASSWORD@localhost:8000/stream/streamer/123 -c copy -f mpegts pipe:1
The streaming URL (served from my server) responds 500 then 404, and it's intended to do so.
The problem is that my server should handle too many requests in short time from tvheadend's scanning subscription.
When I tested same URL on tvheadend 4.2.8 on Debian GNU/Linux(Windows Subsystem for Linux), tvheadend failed to scan and gave up immediately(tried just 2 times).
I think that's normal and expected process.
These are logs from tvheadend 4.2.8 on Debian GNU/Linux(Windows Subsystem for Linux) :
2019-04-14 17:13:45.922 mpegts: TEST_CHANNEL in TEST_PROVIDER - tuning on IPTV
2019-04-14 17:13:45.945 subscription: 0003: "scan" subscribing to mux "TEST_CHANNEL", weight: 6, adapter: "IPTV", network: "TEST_PROVIDER", service: "Raw PID Subscription"
2019-04-14 17:13:45.945 spawn: Executing "/usr/bin/ffmpeg"
2019-04-14 17:13:47.372 spawn: ffmpeg version 3.2.12-1~deb9u1 Copyright (c) 2000-2018 the FFmpeg developers
2019-04-14 17:13:47.372 spawn: built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
2019-04-14 17:13:47.372 spawn: configuration: --prefix=/usr --extra-version='1~deb9u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv
2019-04-14 17:13:47.372 spawn: libavutil 55. 34.101 / 55. 34.101
2019-04-14 17:13:47.372 spawn: libavcodec 57. 64.101 / 57. 64.101
2019-04-14 17:13:47.372 spawn: libavformat 57. 56.101 / 57. 56.101
2019-04-14 17:13:47.372 spawn: libavdevice 57. 1.100 / 57. 1.100
2019-04-14 17:13:47.373 spawn: libavfilter 6. 65.100 / 6. 65.100
2019-04-14 17:13:47.373 spawn: libavresample 3. 1. 0 / 3. 1. 0
2019-04-14 17:13:47.373 spawn: libswscale 4. 2.100 / 4. 2.100
2019-04-14 17:13:47.373 spawn: libswresample 2. 3.100 / 2. 3.100
2019-04-14 17:13:47.373 spawn: libpostproc 54. 1.100 / 54. 1.100
2019-04-14 17:13:47.946 spawn: [http @ 0x7fffe7c6a740] HTTP error 500 Internal Server Error
2019-04-14 17:13:47.946 spawn: http://USERID:PASSWORD@localhost:8000/stream/My_Provider/339: Server returned 5XX Server Error reply
2019-04-14 17:13:47.973 spawn: Executing "/usr/bin/ffmpeg"
2019-04-14 17:13:48.196 spawn: ffmpeg version 3.2.12-1~deb9u1 Copyright (c) 2000-2018 the FFmpeg developers
2019-04-14 17:13:48.196 spawn: built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
2019-04-14 17:13:48.196 spawn: configuration: --prefix=/usr --extra-version='1~deb9u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv
2019-04-14 17:13:48.196 spawn: libavutil 55. 34.101 / 55. 34.101
2019-04-14 17:13:48.196 spawn: libavcodec 57. 64.101 / 57. 64.101
2019-04-14 17:13:48.196 spawn: libavformat 57. 56.101 / 57. 56.101
2019-04-14 17:13:48.196 spawn: libavdevice 57. 1.100 / 57. 1.100
2019-04-14 17:13:48.196 spawn: libavfilter 6. 65.100 / 6. 65.100
2019-04-14 17:13:48.196 spawn: libavresample 3. 1. 0 / 3. 1. 0
2019-04-14 17:13:48.196 spawn: libswscale 4. 2.100 / 4. 2.100
2019-04-14 17:13:48.196 spawn: libswresample 2. 3.100 / 2. 3.100
2019-04-14 17:13:48.196 spawn: libpostproc 54. 1.100 / 54. 1.100
2019-04-14 17:13:48.254 spawn: [http @ 0x7fffb84ed740] HTTP error 404 NOT FOUND
2019-04-14 17:13:48.254 spawn: http://USERID:PASSWORD@localhost:8000/stream/My_Provider/339: Server returned 404 Not Found
2019-04-14 17:13:48.273 iptv: stdin pipe unexpectedly closed: No data
RE: IPTV scan subscribing generates too many spawn: Executing "/opt/bin/ffmpeg" on RT-AC68U router.
-
Added by saen acro over 5 years ago
Your stream point to localhost aka 127.0.0.1, how it come there?
Describe more detailed where stream travels.
RE: IPTV scan subscribing generates too many spawn: Executing "/opt/bin/ffmpeg" on RT-AC68U router.
-
Added by DS Kim over 5 years ago
saen acro wrote:
Your stream point to localhost aka 127.0.0.1, how it come there?
Describe more detailed where stream travels.
Sorry, I modified the URL for privacy.
I'm actually using TLD in the streaming URL like http://xxxx:[email protected]:8000/stream/My_Provider/339.
My proxy server and tvheadend is running on RT-AC68U router.
My Windows laptop(running tvheadend 4.2.8 on Debian GNU/Linux) is conneted to RT-AC68U.
And RT-AC68U is routing router.xxxxxx.ga to itself(localhost) by dnsmasq.
So tvheadend on RT-AC68U and tvheadend on Debian(WSL) can request the same streaming URL to the proxy server on RT-AC68U.
proxy server(8000)
|
RT-AC68U(router.xxxxxx.ga) <-----------> Laptop(192.168.x.x)
| |
tvheadend(9981) tvheadend(9981)
RE: IPTV scan subscribing generates too many spawn: Executing "/opt/bin/ffmpeg" on RT-AC68U router.
-
Added by saen acro over 5 years ago
Why you need proxy in this case?
RE: IPTV scan subscribing generates too many spawn: Executing "/opt/bin/ffmpeg" on RT-AC68U router.
-
Added by DS Kim over 5 years ago
saen acro wrote:
Why you need proxy in this case?
The proxy server is a web crawler.
It scrapes streaming URLs from various remote providers and each of them requires complicated conditions(e.g. login, cookies...).
So I decided to make a proxy server that could handle them programmatically.
Actually, it works!
RE: IPTV scan subscribing generates too many spawn: Executing "/opt/bin/ffmpeg" on RT-AC68U router.
-
Added by DS Kim over 5 years ago
I think I found a solution.
126 line @ src/input/mpegts/iptv/iptv_pipe.c - 4.2
if (mclk() < im->mm_iptv_respawn_last + sec2mono(2)) {
tvherror(LS_IPTV, "stdin pipe unexpectedly closed: %s",
r < 0 ? strerror(errno) : "No data");
} else {
As I understand it, 'mclk()' is now, and 'im->mm_iptv_respawn_last' is the last started time of a spawn thread. And it checks whether 2 seconds have passed after the last spawn started. If it takes more than 2 seconds to reach the if statement then a new thread starts, otherwise the while loop is broken.
In my case, the condition of the if statement always returned False. That's why too many spawns are generated until timeout.
I think it's because 'im->mm_iptv_respawn_last' is 'long int', but the others are 'long long int'.
I tried to log them with :
tvherror(LS_IPTV, "## mclk() : %lld", mclk());
tvherror(LS_IPTV, "## im->mm_iptv_respawn_last : %lld", (long long int)im->mm_iptv_respawn_last);
tvherror(LS_IPTV, "## sec2mono(2) : %lld", sec2mono(2));
tvherror(LS_IPTV, "## im->mm_iptv_respawn_last + sec2mono(2): %lld", im->mm_iptv_respawn_last + sec2mono(2));
tvherror(LS_IPTV, "## abs(mclk()), abs(im->mm_iptv_respawn_last), abs(sec2mono(2)) : %d, %d, %d", abs(mclk()), abs(im->mm_iptv_respawn_last), abs(sec2mono(2)));
And these are what I'm got :
2019-04-19 16:15:59.445 iptv: ## mclk() : 179301691855
2019-04-19 16:15:59.445 iptv: ## im->mm_iptv_respawn_last : -1087559463
2019-04-19 16:15:59.445 iptv: ## sec2mono(2) : 2000000
2019-04-19 16:15:59.445 iptv: ## im->mm_iptv_respawn_last + sec2mono(2): -1085559463
2019-04-19 16:15:59.445 iptv: ## abs(mclk()), abs(im->mm_iptv_respawn_last), abs(sec2mono(2)) : 1086934577, 1087559463, 2000000
......
2019-04-19 16:15:59.539 iptv: ## mclk() : 179301791965
2019-04-19 16:15:59.539 iptv: ## im->mm_iptv_respawn_last : -1086934577
2019-04-19 16:15:59.539 iptv: ## sec2mono(2) : 2000000
2019-04-19 16:15:59.539 iptv: ## im->mm_iptv_respawn_last + sec2mono(2): -1084934577
2019-04-19 16:15:59.539 iptv: ## abs(mclk()), abs(im->mm_iptv_respawn_last), abs(sec2mono(2)) : 1086834467, 1086934577, 2000000
......
I don't know what values returned on the other systems, the 'im->mm_iptv_respawn_last' returned a negative time value on my RT-AC68U so far. That made a comparison of 'a positive value < a negative value' in the if statement. Therefore, the condition of the if statement always returns False.
My solution is using absolute values of the times, since they are just needed to check duration of a thread but not correct date and time. I'm totally new in C programming for Linux. So this solution has a chance not to work properly.
--- a/src/input/mpegts/iptv/iptv_pipe.c
+++ b/src/input/mpegts/iptv/iptv_pipe.c
@@ -123,7 +123,7 @@ iptv_pipe_read ( iptv_mux_t *im )
spawn_kill(pid, iptv_pipe_kill_sig(im), im->mm_iptv_kill_timeout);
im->mm_iptv_fd = -1;
im->im_data = NULL;
- if (mclk() < im->mm_iptv_respawn_last + sec2mono(2)) {
+ if (abs(mclk()) < abs(im->mm_iptv_respawn_last + sec2mono(2))) {
tvherror(LS_IPTV, "stdin pipe unexpectedly closed: %s",
r < 0 ? strerror(errno) : "No data");
} else {