Transcoding choking on 4x quadcore Xeon server
Added by Broc Miller almost 8 years ago
Trying to set up a box to stream to 5-10 clients with transcoding.
Backend Specs:
-Dell POWEREDGE R900
-Processor 4 x Intel Quad Core 2.93GHz 8MB 1066MHz X7350
-32 x 4GB Memory Sticks
-5 X 1TB 7.2K SATA 3.5" Hard Drives (Configured for RAID 5)
-YR352 Quad Port GB NIC
-Ubuntu X64
$lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.1 LTS
Release: 16.04
Codename: xenial
-tvheadend --version tvheadend: version 4.1-2401~g1f894a6~xenial
Transcode Settings:
-Container MPEG-TS/av-lib
-Resolution 1080
-Video Codec libx264: libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
-Video codec preset: superfast: h264 / h265
-Video bitrate: 2782
-Audio codec: aac: AAC (Advanced Audio Coding)
-Audio bitrate: 128
I can run 1 client off of this machine with no problems, as soon as I try a second transcode client, both clients begin to stop/start, buffer and adding more clients only make it worse. I have a desktop machine with a quad core i7-3770 in it that does just fine up to 4-5 clients then CPU hits wall with same transcode settings. What gives? Have 16 cores and am having problems. Anyone know where to even start looking?
Thanks,
Replies (18)
RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago
Shortly X7350 have tons of missing instructions latest Atom's are more powerful
try to install intel microcode
few years ago am try to compare hp Dl580 G2 (4 cpus @2.7Ghz) with Xeon [email protected] big diference
RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago
Updated microcode (was already at most recent state for CPU's) still same result.
Here are the capabilities as reported by libav when opening a stream.
-Xeon processor (Intel® Xeon® Processor X7350 (8M Cache, 2.93 GHz, 1066 MHz FSB)
-using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
-libav: 264 - core 148 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x3 me=dia subme=1 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=22 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=300 keyint_min=30 scenecut=40 intra_refresh=0 rc_lookahead=0 rc=abr mbtree=0 bitrate=2872 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=3477 vbv_bufsize=10432 nal_hrd=none filler=0 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00
-i7-3770
-using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
-libav: 264 - core 148 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x3 me=dia subme=1 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=300 keyint_min=30 scenecut=40 intra_refresh=0 rc_lookahead=0 rc=abr mbtree=0 bitrate=2872 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=3590 vbv_bufsize=10770 nal_hrd=none filler=0 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00
Any other ideas or am I just using the wrong CPU's?
RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago
Looking at $htop command with 2 streams running it doesn't look like I'm hitting a wall because of CPU. See attached picture.!!
Server.png (554 KB) Server.png |
RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago
So changed
Video Codec Preset: slower: h264 / h265 / qsv(h264)
Was using "Super Fast" before and I'm getting lots more cpu usage but still having problems with stuttering (only 1 stream this time with the higher profile encoding preset). I'm thinking I must have some kind of bottleneck somewhere else.
Did some memory tests
Xeon system:
mbw -b 4096 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04529 MiB: 32.00000 Copy: 706.484 MiB/s
AVG Method: DUMB Elapsed: 0.04674 MiB: 32.00000 Copy: 684.649 MiB/s
AVG Method: MCBLOCK Elapsed: 0.03987 MiB: 32.00000 Copy: 802.590 MiB/s
i7-3770 system:
mbw -b 4096 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.00691 MiB: 32.00000 Copy: 4633.853 MiB/s
AVG Method: DUMB Elapsed: 0.00616 MiB: 32.00000 Copy: 5191.855 MiB/s
AVG Method: MCBLOCK Elapsed: 0.00422 MiB: 32.00000 Copy: 7580.064 MiB/s
Could memory Bandwidth be where my issue lies? Any ideas or experience with this out there?
Server2.png (511 KB) Server2.png |
RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago
Phoronix Test Suite
Nice tool to test and compare
RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago
Intel Xeon E5620 @ 2.40GHz
sysbench --test=cpu --cpu-max-prime=20000 run sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Doing CPU performance benchmark Threads started! Done. Maximum prime number checked in CPU test: 20000 Test execution summary: total time: 27.6008s total number of events: 10000 total time taken by event execution: 27.5999 per-request statistics: min: 2.72ms avg: 2.76ms max: 4.59ms approx. 95 percentile: 3.02ms Threads fairness: events (avg/stddev): 10000.0000/0.00 execution time (avg/stddev): 27.5999/0.00
atom d410 @1.6
sysbench --test=cpu --cpu-max-prime=20000 run sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Doing CPU performance benchmark Threads started! Done. Maximum prime number checked in CPU test: 20000 Test execution summary: total time: 193.5603s total number of events: 10000 total time taken by event execution: 193.5506 per-request statistics: min: 18.14ms avg: 19.36ms max: 113.48ms approx. 95 percentile: 24.31ms Threads fairness: events (avg/stddev): 10000.0000/0.00 execution time (avg/stddev): 193.5506/0.00
RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago
Quad X7350's @ 2.93Ghz
sysbench --test=cpu --cpu-max-prime=20000 run sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Doing CPU performance benchmark Threads started! Done. Maximum prime number checked in CPU test: 20000 Test execution summary: total time: 40.2467s total number of events: 10000 total time taken by event execution: 40.2408 per-request statistics: min: 4.02ms avg: 4.02ms max: 7.45ms approx. 95 percentile: 4.02ms Threads fairness: events (avg/stddev): 10000.0000/0.00 execution time (avg/stddev): 40.2408/0.00
@saen acro Are you transcoding with Xeon rig?
RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago
16 treads ~
four 4K channels to 1080p
10 1080p to 720p
30 1080p to 480p
but mostly stay shut down
With compromise on quality of output
experiment with
http://www.ipvideotrans.com/live-video-transcoding.html
just to understand idea of bitrate:size:quality (cpu affinity)
then experiment plex's ffmpeg (gpu powered)
always try to keep totoal cpu usage under 85%
later
https://01.org/linuxgraphics/community/vaapi (only for i7-3770)
RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago
I have tried the ipvideotrans and it works great! On the Xeon machine I can do 9+ SD streams no problem and all the CPU's show it. If I try similar settings in TvHeadend for SD, processors are only ~8% for one stream. If I try to do another stream at the same time, VLC player stutters on both streams and turns grey then picture will come and freeze.
I know from using ipvideotrans that I have enough cpu to transcode more than 1 stream at a time but am having difficulty finding out where my problem is with tvheadend and live transcoding. If I turn up the profile in Tvheadend then the CPU use will rise but again cannot do more than 2 simultaneous streams even though there is lots of CPU left.
Is there a buffer I need to increase somewhere? Multi-thread option for libx264? Just can't figure it out.
RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago
There also some treads around in forum for hardware acceleration for build in ffmpeg and etc.
Defensively ffmpeg is reason and problem can be there.
RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago
Tried on an i5-6600k Vaapi acceleration with tvheadend and this fork https://github.com/lekma/tvheadend was disappointing at best. Most of the adjustments didn't work or crashed TVH. 2 streams at 1080 in and 720 out was pretty much half of gpu. Also tried NVENC on the same machine with a GeForce 1070 which worked really well but is locked down to 2 concurrent transcodes (apparently a professional GeForce card does not have this limitation but they are also $500-800).
I'm thinking I would like to play with some of the options passed to libx264 and libav but cannot find anywhere in TVH source where these are. Specifically I would like to change these options in TVH when a stream is opened:
-threads=12
-lookahead_threads=2
-sliced_threads=0
Any idea on how to do that?
I know my hardware has the horsepower to do what I need because Ipvideotrans worked very well.
RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago
https://github.com/tvheadend/tvheadend/blob/master/Makefile.ffmpeg
this is where can be controlled compilation.
for rest settings need UI support aka new request ticket.
RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago
From this post https://tvheadend.org/boards/5/topics/13892?page=3 it seems that maybe some optimizations in how transcoding is implemented in TVH being more "serialized" than "parallel" is where the problem lies. That posts mentions the RasPi hardware encoding but has implications that this would benefit all. Maybe someone could rework transcoding.c in such a way. I don't have the skills for such.
RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago
After lots of hours delving into this problem I have fixed it by using pipe:/// to ffmpeg in tvheadend muxes. Using HdHomerun Prime (with cable card) as source.
EG:
In the "URL:" section of Muxes
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v528 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
You can automate adding all of these by making an HDHomerunPrime1.m3u file (like below) adding in "Networks Tab" a new "IPTV Automatic Network" and providing a link to your m3u file in the "URL:" section of your new network.
#EXTM3U #EXTINF:-1,[USAeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v524 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[AMCeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v525 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[MTVeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v526 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[CNNHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v527 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[A&EeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v528 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[TWCHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v529 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[ESPN2HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v530 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[ESPNHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v531 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[TLCeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v533 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[TNTeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v534 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[FNCHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v535 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[E!eHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v536 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[TRVLeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v537 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[CMTeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v538 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[TCMHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v540 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[NICKeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v541 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[ANPLeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v542 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[TBSeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v543 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[SYFYeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v544 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[FOODeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v547 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[DISCeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v548 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[HGTVeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v549 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[HISTeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v550 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[ALTUTHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v551 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[FAMwHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v555 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[CNBCHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v557 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[VH1eHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v558 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[CMDYwHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v559 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[HALLeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v560 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[TRUeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v561 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[FXwHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v563 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[MNBCHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v564 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[OXYGeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v566 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[E-Vine HD (Shop] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v573 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[JWLY HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v575 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[MTV 2 HD east] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v577 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[TVLD HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v579 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[ID HD e] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v580 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[BET HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v581 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[BRVOeHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v582 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[LIFEe HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v584 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[FX Movie Channe] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v593 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[HALLMARK MOVIE] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v594 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[DISNEY XD HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v602 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[Bloomberg HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v613 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[HLN HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v614 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[FOX BUSINESS HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v615 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[CSPAN HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v617 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[CSPAN2 HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v618 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[CSPAN3 HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v619 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[NBC SPORTS HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v646 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[FXX HD east] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v647 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[OUTDOOR HD east] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v650 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[FOX SPORTS 1 HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v652 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[Fox Sports 2 HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v653 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[ALT2HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v654 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[NATIONAL GEO HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v665 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[Liquidation HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v677 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[Son Life HD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v678 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1 #EXTINF:-1,[VLCTYHD] pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v801 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
This works really well, I no longer choke on more than 4 streams and my CPU will go all the way to 100% @ 6 streams at the same time. Now I just need better hardware to be able to transcode all 9 tuners (2/3 the way there with an i7-3770). Not sure what the problem is with TVH as far as not being able to do more than 4 streams with the built in method and libav on this hardware but, this works. Hopefully someone will find this useful in the future and save themselves a bunch of time.
Cheers
RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago
http://rodrigopolo.com/ffmpeg/
play little bit with this
Next is NGINX HLS output with transcoding
RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago
I did change my command a bit.
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v528 -vcodec libx264 -preset superfast -crf 28 -maxrate 2600k -bufsize 3500k -vf yadif=0,scale=-2:720 -acodec aac -b:a 128K -f mpegts pipe:1
Went from a constant rate to varible rate and it's alot better on bw.
RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago
ultrafast +5% more bitrate is almost same but cpu usage is more reduced
mp3 audio or mpeg2 is mostly same quality but cpu is less used
RE: Transcoding choking on 4x quadcore Xeon server - Added by norman staffd almost 5 years ago
Broc Miller wrote:
After lots of hours delving into this problem I have fixed it by using pipe:/// to ffmpeg in tvheadend muxes. Using HdHomerun Prime (with cable card) as source.
EG:
In the "URL:" section of Muxes
[...]You can automate adding all of these by making an HDHomerunPrime1.m3u file (like below) adding in "Networks Tab" a new "IPTV Automatic Network" and providing a link to your m3u file in the "URL:" section of your new network.
[...]
This works really well, I no longer choke on more than 4 streams and my CPU will go all the way to 100% @ 6 streams at the same time. Now I just need better hardware to be able to transcode all 9 tuners (2/3 the way there with an i7-3770). Not sure what the problem is with TVH as far as not being able to do more than 4 streams with the built in method and libav on this hardware but, this works. Hopefully someone will find this useful in the future and save themselves a bunch of time.
Cheers
You shod be OK, if you add 2 GPU card, dont use CPU for transcoding , I have same solution but using GPU P5000 Nvidia 2 work very good.,, Question how did you pull out all this stream from that device, I have same only let me pull 3 stream ?