Project

General

Profile

Forums » General »

Transcoding choking on 4x quadcore Xeon server

Added by Broc Miller almost 8 years ago

Trying to set up a box to stream to 5-10 clients with transcoding.

Backend Specs:
-Dell POWEREDGE R900
-Processor 4 x Intel Quad Core 2.93GHz 8MB 1066MHz X7350
-32 x 4GB Memory Sticks
-5 X 1TB 7.2K SATA 3.5" Hard Drives (Configured for RAID 5)
-YR352 Quad Port GB NIC
-Ubuntu X64
$lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.1 LTS
Release: 16.04
Codename: xenial
-tvheadend --version tvheadend: version 4.1-2401~g1f894a6~xenial

Transcode Settings:
-Container MPEG-TS/av-lib
-Resolution 1080
-Video Codec libx264: libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
-Video codec preset: superfast: h264 / h265
-Video bitrate: 2782
-Audio codec: aac: AAC (Advanced Audio Coding)
-Audio bitrate: 128

I can run 1 client off of this machine with no problems, as soon as I try a second transcode client, both clients begin to stop/start, buffer and adding more clients only make it worse. I have a desktop machine with a quad core i7-3770 in it that does just fine up to 4-5 clients then CPU hits wall with same transcode settings. What gives? Have 16 cores and am having problems. Anyone know where to even start looking?

Thanks,


Replies (18)

RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago

Shortly X7350 have tons of missing instructions latest Atom's are more powerful
try to install intel microcode

few years ago am try to compare hp Dl580 G2 (4 cpus @2.7Ghz) with Xeon big diference

RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago

Updated microcode (was already at most recent state for CPU's) still same result.

Here are the capabilities as reported by libav when opening a stream.
-Xeon processor (Intel® Xeon® Processor X7350 (8M Cache, 2.93 GHz, 1066 MHz FSB)
-using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
-libav: 264 - core 148 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x3 me=dia subme=1 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=22 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=300 keyint_min=30 scenecut=40 intra_refresh=0 rc_lookahead=0 rc=abr mbtree=0 bitrate=2872 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=3477 vbv_bufsize=10432 nal_hrd=none filler=0 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00

-i7-3770
-using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
-libav: 264 - core 148 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x3 me=dia subme=1 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=300 keyint_min=30 scenecut=40 intra_refresh=0 rc_lookahead=0 rc=abr mbtree=0 bitrate=2872 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=3590 vbv_bufsize=10770 nal_hrd=none filler=0 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00

Any other ideas or am I just using the wrong CPU's?

RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago

Looking at $htop command with 2 streams running it doesn't look like I'm hitting a wall because of CPU. See attached picture.!!

Server.png (554 KB) Server.png

RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago

So changed
Video Codec Preset: slower: h264 / h265 / qsv(h264)

Was using "Super Fast" before and I'm getting lots more cpu usage but still having problems with stuttering (only 1 stream this time with the higher profile encoding preset). I'm thinking I must have some kind of bottleneck somewhere else.

Did some memory tests

Xeon system:
mbw -b 4096 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04529 MiB: 32.00000 Copy: 706.484 MiB/s
AVG Method: DUMB Elapsed: 0.04674 MiB: 32.00000 Copy: 684.649 MiB/s
AVG Method: MCBLOCK Elapsed: 0.03987 MiB: 32.00000 Copy: 802.590 MiB/s

i7-3770 system:
mbw -b 4096 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.00691 MiB: 32.00000 Copy: 4633.853 MiB/s
AVG Method: DUMB Elapsed: 0.00616 MiB: 32.00000 Copy: 5191.855 MiB/s
AVG Method: MCBLOCK Elapsed: 0.00422 MiB: 32.00000 Copy: 7580.064 MiB/s

Could memory Bandwidth be where my issue lies? Any ideas or experience with this out there?

Server2.png (511 KB) Server2.png

RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago

Phoronix Test Suite
Nice tool to test and compare

RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago

Intel Xeon E5620 @ 2.40GHz


sysbench --test=cpu --cpu-max-prime=20000 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000

Test execution summary:
    total time:                          27.6008s
    total number of events:              10000
    total time taken by event execution: 27.5999
    per-request statistics:
         min:                                  2.72ms
         avg:                                  2.76ms
         max:                                  4.59ms
         approx.  95 percentile:               3.02ms

Threads fairness:
    events (avg/stddev):           10000.0000/0.00
    execution time (avg/stddev):   27.5999/0.00

atom d410 @1.6
sysbench --test=cpu --cpu-max-prime=20000 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000

Test execution summary:
    total time:                          193.5603s
    total number of events:              10000
    total time taken by event execution: 193.5506
    per-request statistics:
         min:                                 18.14ms
         avg:                                 19.36ms
         max:                                113.48ms
         approx.  95 percentile:              24.31ms

Threads fairness:
    events (avg/stddev):           10000.0000/0.00
    execution time (avg/stddev):   193.5506/0.00

RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago

Quad X7350's @ 2.93Ghz

sysbench --test=cpu --cpu-max-prime=20000 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000

Test execution summary:
    total time:                          40.2467s
    total number of events:              10000
    total time taken by event execution: 40.2408
    per-request statistics:
         min:                                  4.02ms
         avg:                                  4.02ms
         max:                                  7.45ms
         approx.  95 percentile:               4.02ms

Threads fairness:
    events (avg/stddev):           10000.0000/0.00
    execution time (avg/stddev):   40.2408/0.00

@saen acro Are you transcoding with Xeon rig?

RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago

16 treads ~
four 4K channels to 1080p
10 1080p to 720p
30 1080p to 480p
but mostly stay shut down ;)

With compromise on quality of output

experiment with
http://www.ipvideotrans.com/live-video-transcoding.html
just to understand idea of bitrate:size:quality (cpu affinity)

then experiment plex's ffmpeg (gpu powered)

always try to keep totoal cpu usage under 85%

later
https://01.org/linuxgraphics/community/vaapi (only for i7-3770)

RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago

I have tried the ipvideotrans and it works great! On the Xeon machine I can do 9+ SD streams no problem and all the CPU's show it. If I try similar settings in TvHeadend for SD, processors are only ~8% for one stream. If I try to do another stream at the same time, VLC player stutters on both streams and turns grey then picture will come and freeze.

I know from using ipvideotrans that I have enough cpu to transcode more than 1 stream at a time but am having difficulty finding out where my problem is with tvheadend and live transcoding. If I turn up the profile in Tvheadend then the CPU use will rise but again cannot do more than 2 simultaneous streams even though there is lots of CPU left.

Is there a buffer I need to increase somewhere? Multi-thread option for libx264? Just can't figure it out.

RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago

There also some treads around in forum for hardware acceleration for build in ffmpeg and etc.
Defensively ffmpeg is reason and problem can be there.

RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago

Tried on an i5-6600k Vaapi acceleration with tvheadend and this fork https://github.com/lekma/tvheadend was disappointing at best. Most of the adjustments didn't work or crashed TVH. 2 streams at 1080 in and 720 out was pretty much half of gpu. Also tried NVENC on the same machine with a GeForce 1070 which worked really well but is locked down to 2 concurrent transcodes (apparently a professional GeForce card does not have this limitation but they are also $500-800).

I'm thinking I would like to play with some of the options passed to libx264 and libav but cannot find anywhere in TVH source where these are. Specifically I would like to change these options in TVH when a stream is opened:

-threads=12
-lookahead_threads=2
-sliced_threads=0

Any idea on how to do that?
I know my hardware has the horsepower to do what I need because Ipvideotrans worked very well.

RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago

https://github.com/tvheadend/tvheadend/blob/master/Makefile.ffmpeg
this is where can be controlled compilation.

for rest settings need UI support aka new request ticket.

RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago

From this post https://tvheadend.org/boards/5/topics/13892?page=3 it seems that maybe some optimizations in how transcoding is implemented in TVH being more "serialized" than "parallel" is where the problem lies. That posts mentions the RasPi hardware encoding but has implications that this would benefit all. Maybe someone could rework transcoding.c in such a way. I don't have the skills for such.

RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller almost 8 years ago

After lots of hours delving into this problem I have fixed it by using pipe:/// to ffmpeg in tvheadend muxes. Using HdHomerun Prime (with cable card) as source.

EG:

In the "URL:" section of Muxes

pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v528 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1

You can automate adding all of these by making an HDHomerunPrime1.m3u file (like below) adding in "Networks Tab" a new "IPTV Automatic Network" and providing a link to your m3u file in the "URL:" section of your new network.

#EXTM3U
#EXTINF:-1,[USAeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v524 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[AMCeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v525 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[MTVeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v526 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[CNNHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v527 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[A&EeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v528 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[TWCHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v529 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[ESPN2HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v530 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[ESPNHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v531 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[TLCeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v533 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[TNTeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v534 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[FNCHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v535 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[E!eHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v536 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[TRVLeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v537 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[CMTeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v538 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[TCMHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v540 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[NICKeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v541 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[ANPLeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v542 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[TBSeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v543 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[SYFYeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v544 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[FOODeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v547 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[DISCeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v548 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[HGTVeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v549 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[HISTeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v550 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[ALTUTHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v551 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[FAMwHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v555 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[CNBCHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v557 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[VH1eHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v558 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[CMDYwHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v559 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[HALLeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v560 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[TRUeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v561 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[FXwHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v563 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[MNBCHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v564 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[OXYGeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v566 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[E-Vine HD (Shop]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v573 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[JWLY HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v575 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[MTV 2 HD east]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v577 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[TVLD HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v579 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[ID HD e]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v580 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[BET HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v581 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[BRVOeHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v582 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[LIFEe HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v584 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[FX Movie Channe]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v593 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[HALLMARK MOVIE]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v594 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[DISNEY XD HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v602 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[Bloomberg HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v613 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[HLN HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v614 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[FOX BUSINESS HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v615 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[CSPAN HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v617 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[CSPAN2 HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v618 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[CSPAN3 HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v619 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[NBC SPORTS HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v646 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[FXX HD east]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v647 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[OUTDOOR HD east]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v650 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[FOX SPORTS 1 HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v652 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[Fox Sports 2 HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v653 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[ALT2HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v654 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[NATIONAL GEO HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v665 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[Liquidation HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v677 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[Son Life HD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v678 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1
#EXTINF:-1,[VLCTYHD]
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v801 -vcodec libx264 -preset superfast -vf yadif=0,scale=-2:720 -b:v 2600k -acodec aac -b:a 128K -copyinkf -f mpegts pipe:1

This works really well, I no longer choke on more than 4 streams and my CPU will go all the way to 100% @ 6 streams at the same time. Now I just need better hardware to be able to transcode all 9 tuners (2/3 the way there with an i7-3770). Not sure what the problem is with TVH as far as not being able to do more than 4 streams with the built in method and libav on this hardware but, this works. Hopefully someone will find this useful in the future and save themselves a bunch of time.

Cheers

RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro almost 8 years ago

http://rodrigopolo.com/ffmpeg/
play little bit with this ;)

Next is NGINX HLS output with transcoding :P

RE: Transcoding choking on 4x quadcore Xeon server - Added by Broc Miller over 7 years ago

I did change my command a bit.

pipe:///usr/bin/ffmpeg -loglevel fatal -i http://10.7.7.100:5004/auto/v528 -vcodec libx264 -preset superfast -crf 28 -maxrate 2600k -bufsize 3500k -vf yadif=0,scale=-2:720 -acodec aac -b:a 128K -f mpegts pipe:1

Went from a constant rate to varible rate and it's alot better on bw.

RE: Transcoding choking on 4x quadcore Xeon server - Added by saen acro over 7 years ago

ultrafast +5% more bitrate is almost same but cpu usage is more reduced
mp3 audio or mpeg2 is mostly same quality but cpu is less used

RE: Transcoding choking on 4x quadcore Xeon server - Added by norman staffd almost 5 years ago

Broc Miller wrote:

After lots of hours delving into this problem I have fixed it by using pipe:/// to ffmpeg in tvheadend muxes. Using HdHomerun Prime (with cable card) as source.

EG:

In the "URL:" section of Muxes
[...]

You can automate adding all of these by making an HDHomerunPrime1.m3u file (like below) adding in "Networks Tab" a new "IPTV Automatic Network" and providing a link to your m3u file in the "URL:" section of your new network.

[...]

This works really well, I no longer choke on more than 4 streams and my CPU will go all the way to 100% @ 6 streams at the same time. Now I just need better hardware to be able to transcode all 9 tuners (2/3 the way there with an i7-3770). Not sure what the problem is with TVH as far as not being able to do more than 4 streams with the built in method and libav on this hardware but, this works. Hopefully someone will find this useful in the future and save themselves a bunch of time.

Cheers

You shod be OK, if you add 2 GPU card, dont use CPU for transcoding , I have same solution but using GPU P5000 Nvidia 2 work very good.,, Question how did you pull out all this stream from that device, I have same only let me pull 3 stream ?

    (1-18/18)