Tvheadend

Tvheadend does not construct commands but uses a the libav-library to do transcoding. The advantage is obviously that this way the startup time is reduced.

Flole Systems wrote:

Tvheadend does not construct commands but uses a the libav-library to do transcoding. The advantage is obviously that this way the startup time is reduced.

API is not a crystal ball, need to construct commant to tell what library need to do.
I can't program on C but am program on low level where logical decision is more important.
Main lag of starting is based on 2 things:
analyse of input stream , currently TVH do not know what is stream parameters,
and start analyzing when transcode start,
with can be solved particularly when service is mapped and/or watched later.
video and audio codec, stream size, bitrate.
secondly Output stream forming : frame size, buffer, GOP, B-frames etc.

It still doesn't construct any command. Constructing a command would mean calling the ffmpeg binary with parameters that are generated by Tvheadend. But that is not what's happening. Tvheadend is starting the transcoding super fast, it's ffmpeg which is slow. That's why the native libav transcoding is better than using external commands like you suggested.

I'm not suggest to use external programs,
external programs in moment solve TVH problems.

If not send command to libav, then how you tell it to compress Mpeg2 video with mpeg1 layer2 audio to H264 with AAC with 480p resolution? (without crystal ball)
Simple logic is to assemble request (aka command) to libav to do what you want, else result all time will be random.
https://libav.org/documentation/doxygen/master/dir_687bdf86e8e626c2168c3a2d1c125116.html

You set the parameters and pass a codec parameter to the encoder. If you specify illegal parameters it won't work as expected. It's entirely different from constructing a command, which would mean something like assembling an ffmpeg command which then gets executed.

Flole Systems Systems, sorry I was so busy with my work and it took me sometimes to try out our last commit. The NVENC transcoding is indeed working but I'm hitting another bug.

For odd reason, the GPU run out of memory if I switch channel several time. It seems TVH or maybe FFMPEG create a new NVENC stream everytime I switch channel but doesn't remove the old one. Here an example below:

2021-02-26 04:47:44.170 libav: AVCodecContext: dl_fn->cuda_dl->cuCtxCreate(&ctx->cu_context_internal, 0, cu_device) failed
2021-02-26 04:47:44.170 libav: AVCodecContext: -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
2021-02-26 04:47:44.170 libav: AVCodecContext:
2021-02-26 04:47:44.170 libav: AVCodecContext: No capable devices found

--------------------------------------------------------------------------- | NVIDIA-SMI 460.39 Driver Version: 460.39 CUDA Version: 11.2 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 GeForce GTX 165... Off | 00000000:01:00.0 Off | N/A | | 35% 44C P8 10W / 100W | 3874MiB / 3909MiB | 0% Default | | | | N/A |
---------------------------------------------------+----------------------+

--------------------------------------------------------------------------- | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 27439 C tvheadend 3859MiB |
---------------------------------------------------------------------------

Hi Andre,

Can you please share your config for tvheadend and the version of ffmpeg you are using? I am still unable to get this to work fully.

here is mine:

for FFMPEG:

dorin@dorin:~$ ffmpeg
ffmpeg version 4.3.2 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 7 (Ubuntu 7.5.0-6ubuntu2)
configuration: --enable-nonfree --enable-nvenc
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100

for tvheadend config:

Configure arguments:
--build=x86_64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-option-checking --disable-silent-rules --libexecdir=${prefix}/lib/tvheadend --disable-maintainer-mode --disable-dependency-tracking --disable-ffmpeg_static --disable-libfdkaac_static --disable-libtheora_static --disable-libopus_static --disable-libvorbis_static --disable-libvpx_static --disable-libx264_static --disable-libx265_static --disable-libfdkaac --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --disable-avahi --disable-dbus_1 --disable-bintray_cache --enable-hdhomerun_static --enable-hdhomerun_client --enable-libav --enable-pngquant --enable-trace --enable-vaapi --enable-nvenc --jobs=8

Compiler:
Using C compiler: cc
Using LD flags: -ldvbcsa
Build for arch: x86_64

Binaries:
Using PYTHON: python
Using GZIP: gzip
Using BZIP2: bzip2

Options:
pie yes
ccdebug no
cardclient yes
cwc yes
cccam yes
capmt yes
constcw yes
linuxdvb yes
satip_server yes
satip_client yes
hdhomerun_client yes
hdhomerun_static yes
iptv yes
tsfile yes
dvbscan yes
timeshift yes
trace yes
avahi no
zlib yes
libav yes
ffmpeg_static no
libx264 yes
libx264_static no
libx265 yes
libx265_static no
libvpx yes
libvpx_static no
libtheora yes
libtheora_static no
libvorbis yes
libvorbis_static no
libfdkaac no
libfdkaac_static no
libopus yes
libopus_static no
nvenc yes
vaapi yes
mmal no
omx no
inotify yes
epoll yes
pcre no
pcre2 yes
uriparser yes
ccache no
tvhcsa yes
bundle no
pngquant yes
kqueue no
dbus_1 no
android no
gtimer_check no
slow_memoryinfo no
libsystemd_daemon no
pcloud_cache yes
ddci yes
cclang_threadsan no
gperftools no
execinfo yes
mmx yes
sse2 yes
W_unused_result yes
f_stack_protector yes
f_stack_protector_strong yes
f_stack_check yes
f_PIE yes
fdatasync yes
getloadavg yes
atomic32 yes
atomic64 yes
atomic_time_t yes
atomic_ptr yes
bitops64 yes
lockowner yes
qsort_r yes
gmtoff yes
recvmmsg yes
sendmmsg yes
ifnames yes
py_gzip yes
bin_pkg_config yes
bin_xgettext yes
bin_msgmerge yes
bin_gzip yes
bin_bzip2 yes
ssl yes
linuxdvbapi yes
linuxdvb_ca yes
upnp yes
hwaccels yes
inotify_h yes
inotify_init1 yes
dvbcsa yes
epoll_create1 yes
mpegts yes
mpegts_dvb yes

Packages:
openssl 1.1.1f
zlib 1.2.11
libpcre2-8 10.34
liburiparser 0.9.3
libavfilter 7.85.100
libswresample 3.7.100
libavresample 4.0.0
libswscale 5.7.100
libavformat 58.45.100
libavcodec 58.91.100
libavutil 56.51.100
libva 1.7.0
libva-drm 1.7.0

Thank you,
Dorin

Andre-Guy Otis wrote:

For odd reason, the GPU run out of memory if I switch channel several time. It seems TVH or maybe FFMPEG create a new NVENC stream everytime I switch channel but doesn't remove the old one. Here an example below:

I also notice similar behaviour, but with vaapi and amd rx570!

[drm:amdgpu_vce_validate_handle [amdgpu]] ERROR No more free VCE handles!

this is my log from dmesg

Andre-Guy Otis wrote:

Flole Systems Systems, sorry I was so busy with my work and it took me sometimes to try out our last commit. The NVENC transcoding is indeed working but I'm hitting another bug.

For odd reason, the GPU run out of memory if I switch channel several time. It seems TVH or maybe FFMPEG create a new NVENC stream everytime I switch channel but doesn't remove the old one. Here an example below:

2021-02-26 04:47:44.170 libav: AVCodecContext: dl_fn->cuda_dl->cuCtxCreate(&ctx->cu_context_internal, 0, cu_device) failed
2021-02-26 04:47:44.170 libav: AVCodecContext: -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
2021-02-26 04:47:44.170 libav: AVCodecContext:
2021-02-26 04:47:44.170 libav: AVCodecContext: No capable devices found

 +-----------------------------------------------------------------------------+
 | NVIDIA-SMI 460.39       Driver Version: 460.39       CUDA Version: 11.2     |
 |-------------------------------+----------------------+----------------------+
 | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
 | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
 |                               |                      |               MIG M. |
 |===============================+======================+======================|
 |   0  GeForce GTX 165...  Off  | 00000000:01:00.0 Off |                  N/A |
 | 35%   44C    P8    10W / 100W |   3874MiB /  3909MiB |      0%      Default |
 |                               |                      |                  N/A |
 +-------------------------------+----------------------+----------------------+

 +-----------------------------------------------------------------------------+
 | Processes:                                                                  |
 |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
 |        ID   ID                                                   Usage      |
 |=============================================================================|
 |    0   N/A  N/A     27439      C   tvheadend                        3859MiB |
 +-----------------------------------------------------------------------------+

i just can't make it to work. tried the command line that saen provided, just to make sure there is not an issue with the card, but that one works.

i also purged the whole tvheadend + rebuilt using external ffmpeg. you can see the config above. I am getting the same result. tried various combinations of containers and settings, but to no avail. I think I am too stupid for this, therefore I'll stick to vaapi. or try again later.

I have the same problem,

Sat Feb 27 14:21:50 2021

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro K4200        Off  | 00000000:01:00.0 Off |                  N/A |
| 33%   55C    P0    40W / 110W |   3134MiB /  4036MiB |      9%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     19921      C   /usr/bin/tvheadend               3123MiB |
+-----------------------------------------------------------------------------+

I need to restart tvheadend to free up memory.

System:    Host: tvh Kernel: 4.15.0-136-generic x86_64 bits: 64 Console: tty 0 Distro: Ubuntu 18.04.5 LTS
Machine:   Device: desktop Mobo: Micro-Star model: H310-A PRO (MS-7B83) v: 1.0 serial: N/A
           UEFI [Legacy]: American Megatrends v: 1.A0 date: 10/12/2020
CPU:       6 core Intel Core i5-9400 (-MCP-) cache: 9216 KB
           clock speeds: max: 4100 MHz 1: 1986 MHz 2: 1824 MHz 3: 1565 MHz 4: 1668 MHz 5: 1801 MHz 6: 1627 MHz
Graphics:  Card: NVIDIA GK104GL [Quadro K4200]
           Display Server: X.org 1.19.6 driver: nvidia tty size: 128x44 Advanced Data: N/A out of X
Audio:     Card-1 Intel Cannon Lake PCH cAVS driver: snd_hda_intel Sound: ALSA v: k4.15.0-136-generic
           Card-2 NVIDIA GK104 HDMI Audio Controller driver: snd_hda_intel
Network:   Card: Intel Ethernet Connection (7) I219-V driver: e1000e
           IF: eno1 state: up speed: 1000 Mbps duplex: full mac: 30:9c:23:af:e0:53
Drives:    HDD Total Size: 120.0GB (29.6% used)
           ID-1: /dev/sda model: Samsung_SSD_850 size: 120.0GB
Partition: ID-1: / size: 110G used: 34G (32%) fs: ext4 dev: /dev/sda2
RAID:      No RAID devices: /proc/mdstat, md_mod kernel module present
Sensors:   System Temperatures: cpu: 32.0C mobo: N/A
           Fan Speeds (in rpm): cpu: N/A
Info:      Processes: 179 Uptime: 2 days Memory: 5457.3/32104.3MB Init: systemd runlevel: 5
           Client: Shell (bash) inxi: 2.3.56

lukas fuka K, question for you. Do you have hardware accelaration checked under you nvidia profile? I can also get nvidia "transcoding" using the GPU but not hardware accelerated.

Hardware acceleration is only for vaapi, it doesn't do anything for nvidia at the moment. You should not mix nvidia and vaapi, that's usually not a good idea.

I have a patch ready that enabled nvidia decoding acceleration but that is not in yet. I might add an additional checkbox for it, or a drop-down menu so it's possible to choose between nvdec and vaapi.

saen acro wrote:

Andre-Guy Otis wrote:

Flole Systems Systems, sorry I was so busy with my work and it took me sometimes to try out our last commit. The NVENC transcoding is indeed working but I'm hitting another bug.

For odd reason, the GPU run out of memory if I switch channel several time. It seems TVH or maybe FFMPEG create a new NVENC stream everytime I switch channel but doesn't remove the old one. Here an example below:

2021-02-26 04:47:44.170 libav: AVCodecContext: dl_fn->cuda_dl->cuCtxCreate(&ctx->cu_context_internal, 0, cu_device) failed
2021-02-26 04:47:44.170 libav: AVCodecContext: -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
2021-02-26 04:47:44.170 libav: AVCodecContext:
2021-02-26 04:47:44.170 libav: AVCodecContext: No capable devices found

[...]
Try in place of nvidia-smi to use NVTOP

you can kill zombie transcode process there

I run my tvheadend on a headless server and this is the only process that use the GPU. For some reason, TVH create a new nvenc session on the GPU everytime I switch channel but doesn't flush the older session from the GPU, I end up to run out of memory and I had to restart TVH to clear that out.

I'm using a GTX 1650ti which is limited to 3 nvenc sessions by default so I got this out of memory error only after switching my channel 4 times but on the log files, I got no detail, here's an example.

libav: AVCodecContext: out of memory: (no detail)
libav: AVCodecContext: No capable devices found

You can unlock this nvenc sessions limit by just apply a patch on the driver with this tool: https://github.com/keylase/nvidia-patch. I was now able to switch channel more than 4 times but I noticed the GPU memory just keep increasing everytime I switch to a new channel. For testing, I keep doing it until a reach out the max memory and I finally got the cuda error above.

@dorin ilut, like Flole Systems said, the hardware acceleration is for decoding only. You can rely on your CPU for decoding and your GPU for encoding which I'm currently doing. If your nvenc still doesn't work, I can help you out. Please let me know what is your nvidia driver version, your linux distro and the error you get from the logs.

Flole Systems An other thing I found out is the scaling doesn't work. My ATSC signal is 1080p and I tried to scale it down to 720p but it still showing in 1080p when I watch it over Kodi. That's not a big deal for me but may matter for others.

If you use HTSP for kodi that is perfectly normal. HTSP is always passthrough, no transcoding possible.

I don't really understand why the sessions are not cleaned up, for me it is working perfectly fine. I start a channel, Tvheadend is now visible in nvidia-smi, I stop it and Tvheadend completely disappears from nvidia-smi.

Flole Systems Sorry, my post may be wasn't clear. I'm aware the HTSP for Kodi doesn't do any transcoding but I set a stream profile for my kodi user under TVH and this profile has the transcoding on it and is working as intended but the scalling do not work from master git, the stable release 4.2.8 on the other hand work perfectly.

For the session not cleaned up is a big mystery. Do you run the same nvidia driver as I do? I'm using the version 460.39 with lastest CUDA 11.2 and ffmpeg git.

I tried with the internal ffmpeg aswell as the git master ffmpeg, didn't seem to make a difference. Also I was using all kinds of different drivers as I usually upgrade when new versions are available, so also the one you mentioned. I suggest that someone who has the issue does some debugging, including trace of libav and transcoding. Also it's usually a good idea to add more verbose messages if things aren't working as expected.

Flole Systems wrote:

If you use HTSP for kodi that is perfectly normal. HTSP is always passthrough, no transcoding possible.

blablabla this is not true
set transcode profile in user profile and will transcode

I don't really understand why the sessions are not cleaned up, for me it is working perfectly fine. I start a channel, Tvheadend is now visible in nvidia-smi, I stop it and Tvheadend completely disappears from nvidia-smi.

share your config specs /drivers etc.

Flole Systems wrote:

I tried with the internal ffmpeg aswell as the git master ffmpeg, didn't seem to make a difference. Also I was using all kinds of different drivers as I usually upgrade when new versions are available, so also the one you mentioned. I suggest that someone who has the issue does some debugging, including trace of libav and transcoding. Also it's usually a good idea to add more verbose messages if things aren't working as expected.

here are my attempts:

Nvidia profile with acceleration enabled ( I know you said to not enable that yet )
2021-02-28 10:58:40.316 subscription: 0031: "188.60.141.85 [ dorin | Kodi Media Center ]" subscribing on channel "AMC HD", weight: 100, adapter: "Silicon Labs Si2168 #2 : DVB-C #0", network: "rcs", mux: "522MHz", provider: "AMC Networks Int CE", service: "AMC HD", profile="dt_nv", hostname="188.60.141.85", username="dorin", client="Kodi Media Center"
2021-02-28 10:58:40.479 transcode: 0002: 01:H264: > Using profile dt_nv
2021-02-28 10:58:40.479 transcode: 0002: 02:MPEG2AUDIO: > Copy
2021-02-28 10:58:40.479 transcode: 0002: 03:MPEG2AUDIO: > Copy
2021-02-28 10:58:40.479 transcode: 0002: 04:DVBSUB: > Copy
2021-02-28 10:58:41.058 libav: AVCodecContext: mmco: unref short failure
2021-02-28 10:58:41.071 libav: AVCodecContext: reference picture missing during reorder
2021-02-28 10:58:41.071 libav: AVCodecContext: Missing reference picture, default is 2147483647
2021-02-28 10:58:41.071 libav: AVCodecContext: mmco: unref short failure
2021-02-28 10:58:41.200 transcode: 0002: 01:H264: [h264 => h264_nvenc]: opts: gpu=0,b=2000000,quality=0,tvh_filter_deint=1,width=1920,height=1080,pix_fmt=0,bf=3,tvh_require_meta=1
2021-02-28 10:58:41.321 transcode: 0002: 01:H264: [h264 => h264_nvenc]: filters: source args: 'video_size=1920x1080:pix_fmt=vaapi_vld:time_base=1/50:pixel_aspect=1/1'
2021-02-28 10:58:41.321 transcode: 0002: 01:H264: [h264 => h264_nvenc]: filters: filters: 'deinterlace_vaapi'
2021-02-28 10:58:41.321 libav: TVHGraph: Impossible to convert between the formats supported by the filter 'Parsed_deinterlace_vaapi_0' and the filter 'auto_scaler_0'
2021-02-28 10:58:41.321 transcode: 0002: 01:H264: [h264 => h264_nvenc]: filters: failed to config filter graph
2021-02-28 10:58:41.321 transcode: 0002: 01:H264: [h264 => h264_nvenc]: opts: quality=0
2021-02-28 10:59:29.771 subscription: 0031: "188.60.141.85 [ dorin | Kodi Media Center ]" unsubscribing from "AMC HD", hostname="188.60.141.85", username="dorin", client="Kodi Media Center"

Using nvidia profile without hardware acceleration enabled
2021-02-28 10:59:58.874 mpegts: 522MHz in rcs - tuning on Silicon Labs Si2168 #2 : DVB-C #0
2021-02-28 10:59:58.874 subscription: 0032: "188.60.141.85 [ dorin | Kodi Media Center ]" subscribing on channel "AMC HD", weight: 100, adapter: "Silicon Labs Si2168 #2 : DVB-C #0", network: "rcs", mux: "522MHz", provider: "AMC Networks Int CE", service: "AMC HD", profile="dt_nv", hostname="188.60.141.85", username="dorin", client="Kodi Media Center"
2021-02-28 10:59:59.037 transcode: 0003: 01:H264: > Using profile dt_nv
2021-02-28 10:59:59.037 transcode: 0003: 02:MPEG2AUDIO: > Copy
2021-02-28 10:59:59.037 transcode: 0003: 03:MPEG2AUDIO: > Copy
2021-02-28 10:59:59.037 transcode: 0003: 04:DVBSUB: > Copy
2021-02-28 10:59:59.445 transcode: 0003: 01:H264: [h264 => h264_nvenc]: opts: gpu=0,b=2000000,quality=0,tvh_filter_deint=1,width=1920,height=1080,pix_fmt=0,bf=3,tvh_require_meta=1
2021-02-28 10:59:59.542 transcode: 0003: 01:H264: [h264 => h264_nvenc]: filters: source args: 'video_size=1920x1080:pix_fmt=yuv420p:time_base=1/50:pixel_aspect=1/1'
2021-02-28 10:59:59.542 transcode: 0003: 01:H264: [h264 => h264_nvenc]: filters: filters: 'yadif'
2021-02-28 10:59:59.543 transcode: 0003: 01:H264: [h264 => h264_nvenc]: opts: quality=0
2021-02-28 11:00:40.453 subscription: 0032: "188.60.141.85 [ dorin | Kodi Media Center ]" unsubscribing from "AMC HD", hostname="188.60.141.85", username="dorin", client="Kodi Media Center"

Using the first one does not show in nvtop. using the second one shows in nvtop, but once the stream is stopped it does not go away. Stays in and just accumulates memory with every play.

I found that the memory is freed when playing via VLC player. If I use KODI, each program switch will increase the allocated GPU memory.

Lukas G wrote:

I found that the memory is freed when playing via VLC player. If I use KODI, each program switch will increase the allocated GPU memory.

Good observation, thanks! I also confimed that behaivour! In kodi 19, tvheadend addon have option: "use http for streaming channels" .When use this option, and channel stops, vram is flushed! When is disabled, vram increase! Maybe this will help for developers to identify problem... Im not use nvidia card, but situation is exactly the same, with AMD GPU!

HTSP does not offer transcoding at the moment, that's why it's not possible to create a HTSP profile with transcoding. Kodi doesn't allow selecting a non-HTSP protocol and will reject that. If you found some hack that allows this then don't be surprised if things don't work as expected.

Flole Systems wrote:

HTSP does not offer transcoding at the moment, that's why it's not possible to create a HTSP profile with transcoding. Kodi doesn't allow selecting a non-HTSP protocol and will reject that. If you found some hack that allows this then don't be surprised if things don't work as expected.

But then if I may ask, how come vaapi transcoding is working without any issues with Kodi? And the if HTSP does not offer trasncoding would it then not work with streaming profiles? Cause if I enable the streaming profile within Kodi, I get vaapi hardware trasncoding. or am I mixing goats and cabbages.

Flole Systems, I dont know... For some reason, kodi tvheadend addon, have access to my created transcoding profile, via htsp protocol... Pass profile, and other predefined is not allowed... Im using latest master from git!

In kodi there used to be a warning if a non-HTSP profile was specified. If someone removed that warning I don't understand why that was done. That explains why you might think that it's supported to configure it this way.

Some people apparently forced a user to a streaming profile and then used that for kodi transcoding, that's what I meant with "some hack". It seemed to work to some extend but obviously not completely. There is a reason why the HTSP profile exists and why it doesn't offer transcoding (because it's not properly implementer).