Feature #3869
h264_omx encoding for raspberry pi 3
0%
Description
Please implement h264_omx encoder on for raspberry pi 3 to enhance transcoding. Raspberry Pi supports hardware h264 encoding, makes perfect sense for tvheadend to make use of it.
Files
History
Updated by danny skjodt over 8 years ago
Updated by drhans drhans wrote:
Please implement h264_omx encoder on for raspberry pi 3 to enhance transcoding. Raspberry Pi supports hardware h264 encoding, makes perfect sense for tvheadend to make use of it.
Im not sure anyone ever made the pi produce a proper h264 bitstream.
Updated by drhans drhans over 8 years ago
Updated by Raspberry can do hardware h264 encoding: https://www.raspberrypi.org/blog/new-video-features/
Can't tvheadend use this feature?
Updated by Joseph ZACHARIE over 8 years ago
Updated by To enable Hardware encode for Raspberry PI (h264_omx)
We should adapt the compilation of ffmpeg.
The current version in tvheadend is allready compatible. ffmpeg 3.1 accept OMX and mmal param for Raspberry PI 2 and 3
Could you please update the ffmpeg static to user ffmpeg 3.1.1 and enable the param in configuration to : --enable-omx-rpi --enable-mmal
?
Pre req are :
for OpenMAX : libomxil-bellagio-dev (Package)
for MMAL : https://github.com/raspberrypi/userland (to compile)
I have try to adapt the code, but I'm definitly not a dev. (Probably due of a patch of ffmpeg during the compilation)
I'm currenlty using only a PI3 to all my sat system, and I'm pretty sure with HW encoding my PI will be able to transcode to my cell phone ower WAN.
REF: https://ubuntu-mate.community/t/hardware-h264-video-encoding-with-libav-openmax-il/4997/6
Updated by Joseph ZACHARIE over 8 years ago
- File Stream Profiles.png Stream Profiles.png added
So I have finaly compilter the ffmpeg for TVHEADEND adn run some test :
root@osmc314:/mnt/jonas/partage/Disney# ffmpeg -i Vice-Versa\ \(2015\).avi -c:a copy -c:v h264_omx -b:v 128k trailer-sample.mp4
ffmpeg version N-81078-ge83a01d Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 4.9.2 (Debian 4.9.2-10)
configuration: --enable-libfreetype --enable-gpl --enable-nonfree --enable-libx264 --enable-libass --enable-libfaac --enable-libmp3lame --bindir=/usr/local/bin --enable-static --enable-omx-rpi --enable-mmal
libavutil 55. 28.100 / 55. 28.100
libavcodec 57. 50.100 / 57. 50.100
libavformat 57. 44.100 / 57. 44.100
libavdevice 57. 0.102 / 57. 0.102
libavfilter 6. 47.100 / 6. 47.100
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 1.100 / 2. 1.100
libpostproc 54. 0.100 / 54. 0.100
Input #0, avi, from 'Vice-Versa (2015).avi':
Duration: 01:34:34.37, start: 0.000000, bitrate: 1641 kb/s
Stream #0:0: Video: mpeg4 (Advanced Simple Profile) (XVID / 0x44495658), yuv420p, 720x400 [SAR 1:1 DAR 9:5], 1499 kb/s, 30 fps, 30 tbr, 30 tbn, 30 tbc
Stream #0:1: Audio: mp3 (U0[0][0] / 0x0055), 48000 Hz, stereo, s16p, 128 kb/s
File 'trailer-sample.mp4' already exists. Overwrite ? [y/N] y
[h264_omx 0x27e1820] Using OMX.broadcom.video_encode 0x2855c50] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
[mp4
Last message repeated 1 times
Output #0, mp4, to 'trailer-sample.mp4':
Metadata:
encoder : Lavf57.44.100
Stream #0:0: Video: h264 (h264_omx) ([33][0][0][0] / 0x0021), yuv420p, 720x400 [SAR 1:1 DAR 9:5], q=2-31, 128 kb/s, 30 fps, 15360 tbn, 30 tbc
Metadata:
encoder : Lavc57.50.100 h264_omx
Stream #0:1: Audio: mp3 (i0[0][0] / 0x0069), 48000 Hz, stereo, 128 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mpeg4 (native) -> h264 (h264_omx))
Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=170232 fps=163 q=-0.0 Lsize= 180769kB time=01:34:34.36 bitrate= 261.0kbits/s dup=3 drop=0 speed=5.45x
video:88653kB audio:88653kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.953115%
root@osmc314:/mnt/jonas/partage/Disney# ffmpeg -i Vice-Versa\ \(2015\).avi -c:a copy -c:v h264 -b:v 404k trailer-sample.mp4
ffmpeg version N-81078-ge83a01d Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 4.9.2 (Debian 4.9.2-10)
configuration: --enable-libfreetype --enable-gpl --enable-nonfree --enable-libx264 --enable-libass --enable-libfaac --enable-libmp3lame --bindir=/usr/local/bin --enable-static --enable-omx-rpi --enable-mmal
libavutil 55. 28.100 / 55. 28.100
libavcodec 57. 50.100 / 57. 50.100
libavformat 57. 44.100 / 57. 44.100
libavdevice 57. 0.102 / 57. 0.102
libavfilter 6. 47.100 / 6. 47.100
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 1.100 / 2. 1.100
libpostproc 54. 0.100 / 54. 0.100
Input #0, avi, from 'Vice-Versa (2015).avi':
Duration: 01:34:34.37, start: 0.000000, bitrate: 1641 kb/s
Stream #0:0: Video: mpeg4 (Advanced Simple Profile) (XVID / 0x44495658), yuv420p, 720x400 [SAR 1:1 DAR 9:5], 1499 kb/s, 30 fps, 30 tbr, 30 tbn, 30 tbc
Stream #0:1: Audio: mp3 (U0[0][0] / 0x0055), 48000 Hz, stereo, s16p, 128 kb/s
File 'trailer-sample.mp4' already exists. Overwrite ? [y/N] y
[libx264 0x155d820] using SAR=1/1 0x155d820] using cpu capabilities: ARMv6 NEON
[libx264
[libx264 0x155d820] profile High, level 3.0 0x155d820] 264 - core 142 r2431 a5831aa - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=404 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[libx264
[mp4 0x15d1c50] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead. 0x155d820] frame I:3 Avg QP:18.26 size: 1884
Last message repeated 1 times
Output #0, mp4, to 'trailer-sample.mp4':
Metadata:
encoder : Lavf57.44.100
Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 720x400 [SAR 1:1 DAR 9:5], q=-1--1, 404 kb/s, 30 fps, 15360 tbn, 30 tbc
Metadata:
encoder : Lavc57.50.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/404000 buffer size: 0 vbv_delay: -1
Stream #0:1: Audio: mp3 (i[0][0][0] / 0x0069), 48000 Hz, stereo, 128 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mpeg4 (native) -> h264 (libx264))
Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame= 320 fps= 18 q=-1.0 Lsize= 736kB time=00:00:10.77 bitrate= 559.6kbits/s dup=1 drop=0 speed=0.596x
video:558kB audio:168kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.360326%
[libx264
[libx264 0x155d820] frame P:153 Avg QP:18.37 size: 2921 0x155d820] frame B:164 Avg QP:22.13 size: 720
[libx264
[libx264 0x155d820] consecutive B-frames: 25.3% 16.9% 6.6% 51.2% 0x155d820] mb I I16..4: 78.5% 19.4% 2.0%
[libx264
[libx264 0x155d820] mb P I16..4: 8.8% 19.0% 0.1% P16..4: 42.1% 6.5% 2.7% 0.0% 0.0% skip:20.8% 0x155d820] mb B I16..4: 0.2% 0.5% 0.0% B16..8: 33.4% 1.0% 0.1% direct: 1.1% skip:63.8% L0:41.0% L1:57.1% BI: 1.9%
[libx264
[libx264 0x155d820] final ratefactor: 22.28 0x155d820] 8x8 transform intra:65.1% inter:93.3%
[libx264
[libx264 0x155d820] coded y,uvDC,uvAC intra: 18.0% 43.6% 7.4% inter: 6.1% 17.4% 0.2% 0x155d820] i16 v,h,dc,p: 41% 35% 14% 10%
[libx264
[libx264 0x155d820] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 29% 48% 1% 1% 1% 1% 0% 1% 0x155d820] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 29% 28% 2% 5% 4% 4% 2% 2%
[libx264
[libx264 0x155d820] i8c dc,h,v,p: 52% 31% 15% 3% 0x155d820] Weighted P-Frames: Y:15.0% UV:13.7%
[libx264
[libx264 0x155d820] ref P L0: 72.4% 3.3% 19.6% 4.5% 0.2% 0x155d820] ref B L0: 89.7% 9.0% 1.2%
[libx264
[libx264 0x155d820] ref B L1: 97.5% 2.5% 0x155d820] kb/s:427.95
[libx264
Exiting normally, received signal 2.
So with software h264 I've got 18 fps with OMX hardware encode I've got 163 fps with the same file.
Updated by Joseph ZACHARIE over 8 years ago
I've compilted a version without Encoder profile but with h264_omx.
The good part is :
2016-07-25 13:33:01.943 subscription: 010F: "HTTP" subscribing on channel "BFM BUSINESS", weight: 125, adapter: "Technisat SkyStar USB HD (DVB-S/S2) : DVB-S #0", network: "Astra", mux: "11895V", provider: "CSAT", service: "BFM BUSINESS", profile="H264_OMX", hostname="192.168.0.106", username="joseph", client="Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
2016-07-25 13:33:02.652 transcode: 0029: 1:H264 1440x1080 ==> H264 512x384 (h264_omx)
2016-07-25 13:33:02.776 transcode: 0029: missing meta data for H264
2016-07-25 13:33:02.911 libav: mmco: unref short failure
2016-07-25 13:33:02.911 transcode: 0029: Using preset faster
2016-07-25 13:33:02.916 libav: Using OMX.broadcom.video_encode
2016-07-25 13:33:02.934 libav: Increasing reorder buffer to 1
2016-07-25 13:33:03.003 libav: Increasing reorder buffer to 2
2016-07-25 13:33:03.138 libav: mmco: unref short failure
2016-07-25 13:33:03.375 libav: Increasing reorder buffer to 3
The sad part is I still use 170% of CPU in PI3. So I still have freeze during transcode.
So maybe I've done a mistake by removing the profile in transcode.c :
// All modern devices should support "high" profile
// av_dict_set(&opts, "profile", "high", 0);Witout removing the profile, I had a crash during the transcode.
Now, I'll let a real dev work on it.
Updated by Antonio S almost 8 years ago
Updated by 
Hi Joseph
I see you have made some tests for the RPi3 h264 hardware transcoding.
I want to do the same, so I have ordered a RPi3 that should arrive soon.
I'm also a Linux embedded SW engineer, I never worked on video part, but maybe I can work on it.
First, I wanted to enable omx_h264 codec in tvheadend. So I have built a git source code, with --enable-mmal and --enable-omx-rpi in Makefile.ffmpeg and with enable-ffmpeg-static on a bananapi, which is actually CPU compatible but without OMX GPU.
The result of the compilation is ok, but I don't get the h264_omx profile available, as you have shown in your feature requests:
https://tvheadend.org/issues/3869#change-19299
I have also resumed the thread at:
https://tvheadend.org/boards/5/topics/13892?r=22417#message-22417
there is the chance that ffmpeg actually checks if the HW is omx capable or not, and if tvheadend ask ffpmeg for the available codec, it doesn't report it, since I run on a banana pi, I will check this when RPi3 will arrive.
Now, coming back to your results:
I suspect that the reason why the cpu usage is so high, is that the mpeg2 is software decoded.
I see that your ffmpeg test use an avi as source, maybe this avi is a divx or in general an mp4 video. While with tvheadend I assume you try to transcode a DVB-S stream, that normally is MPEG2
So, please remember that the omx hardware decoding of mpeg2 on RPi3 is locked, to unlock it you must buy a (very cheap) mpeg2 license. see here:
http://www.raspberrypi.com/mpeg-2-license-key/
Do you have the license? If not, for the low price it costs, maybe you can just buy it and try. I'm not familiar with the codec pickup logic in tvheaded, so I cannot guarantee that it automatically switch to hardware mpeg2 decoding, but still the license is so cheap...
Another thing is the resize: are you resizing the source?
Updated by Joseph ZACHARIE almost 8 years ago
Hi,
To enable h264_omx I have modify the file https://github.com/tvheadend/tvheadend/blob/master/src/plumbing/transcoding.c
search :
case SCT_H264:
if (!strcmp(ocodec->name, "nvenc") || !strcmp(ocodec->name, "h264_qsv"))
and adapt to enable h264_omx. Moreover I had remove some profile param to to enable the transcode.
I had the mpeg2 licence and I did resize div by 4 and try bitrate limitation.
My test video was a h264 file.
For me the limitation as HW due we can't read the DVB by the USB and transcode in the same time.
When I read the flow from the sdcard or a mounted share It was working well.
I'd plan to retest when I'll have free time ans a more stable tvheadend version. Currently I can't compile it due of my OSMC env not compatible.
Regards
Updated by Antonio S almost 8 years ago
Hello Joseph
regarding this row:
if (!strcmp(ocodec->name, "nvenc") || !strcmp(ocodec->name, "h264_qsv"))
octx->pix_fmt = AV_PIX_FMT_NV12;
else
octx->pix_fmt = AV_PIX_FMT_YUV420P;
do you mean that also h264_omx shall pick up pixel format as AV_PIX_FMT_NV12?