Tvheadend

Provide '--trace timeshift' - https://tvheadend.org/projects/tvheadend/wiki/Traces .

Jaroslav Kysela wrote:

Provide '--trace timeshift' - https://tvheadend.org/projects/tvheadend/wiki/Traces .

Since posting, I've updated to tvheadend 4.1-2088. The problem is still there
Log file uploaded to http://sprunge.us/aZfi

OK, the timeshift code does not try to allocate RAM segments. It looks like another bug - hidden memory leak - which might be similar to #3770 . Unfortunately, I'm out of time to play with wetek now.

OK, so what happens with this issue moving forward ?
Is it 'on the radar' to be investigated, or is the "I'm out of time to play with wetek now" the end of the story ?

Unfortunately, these devices do not have tools to debug this issue in LE/OE, so I cannot instruct you what to do, unless you install debian or any other full distro on it (chroot might be sufficient).

I fixed some other memory leak issues, so you may give a try with newer versions of tvh.

Thanks. I am staying up to date with the releases from http://build.mycvh.de/ but unfortunately, the memory issues that have been fixed have not cured this problem.
I appreciate it is difficult to debug these issues on ARM devices with stripped down OS running.

There is one further observation I have made, and maybe this might help shed more light on the possible cause.
If you record a TV programme without even watching a live broadcast, even with the "On-demand (no first rewind)" enabled, RAM is consumed until exhausted.

Jaroslav Kysela wrote:

Unfortunately, these devices do not have tools to debug this issue in LE/OE, so I cannot instruct you what to do, unless you install debian or any other full distro on it (chroot might be sufficient).

I fixed some other memory leak issues, so you may give a try with newer versions of tvh.

Unfortunately this issue seems to be there still. I'm running HTS Tvheadend 4.1-2277~gbf75b27 and tried to use timeshift with Kodi 16.1, but even without pausing it just eats up all my RAM.

I'm using Tvheadend server + pvr.hts client on LibreELEC (https://forum.libreelec.tv/thread/2156-8-0-1l-libreelec-8-0-for-s905-s905x), and had a similar issue.
Independent of any Tvheadend timeshift settings, the OS would go OOM after having paused live TV for a while. `top` showed that the kodi process' mem increased, while Tvheadend remained unchanged and also no timeshift files were created.
So I assumed that the 'pause' command did not go through to Tvheadend, and Kodi did the buffering...

But strangely, after having recently updated LibreELEC (from 8.0.1k to l) and thus Kodi, and w/o the Tvheadend server and client addons having been updated iirc, the issue is gone. Memory consumption is not increasing beyond the configured limit, and timeshift files are created after that.

Weird, but you might want to try again with recent tvheadend server+client + Kodi. :-)

I believe I'm having the exact same problem.
Using HTS Tvheadend 4.2.8 with Kodi 18.2 with tvheadend plugin and timeshift activated.

I first thought this was driver related:
https://github.com/b-rad-NDi/Embedded-MediaDrivers/issues/5
but now I strongly believe it's a tvheadend issue.

To add to this, I've tested multiple variances over the timeshift option.
In all cases it ended-up generating an OOM killer over tvheadend.
Duration varied depending of selected options.

• Enabled
• Max period: 60 minutes
• Storage Path: /volume1/TV
• Maximum size (MB): 10000
• Maximum RAM size (MB): 4000 (note that I have 16GB ram)

Nothing else selected (e.g. RAM only, On-demand (no first rewind), Unlimited time, Unlimited size, Fit to RAM (cut rewind), Include teletext)

My first assumptions are that setting Max period to 0 (e.g. unlimited) or selecting Unlimited time and/or Unlimited size did resulted in OOM killer due to memory leaking.

Also note that I am running on a Synology DS918+ with the following:

$ uname -a
Linux th0ma7-nas 4.4.59+ #24922 SMP PREEMPT Thu Mar 28 11:07:03 CST 2019 x86_64 GNU/Linux synology_apollolake_918+

Still getting OOM killer, it just takes longer with above parameters (a few more hours to a few days) before it do hapen.

Let me know how I can help to gather data and catch where the bug is.
Cheers.

Use valgrind to see if there is a memory leak.

Also you can use gdb to see what exactly is eating up the memory. You need to do this before the crash though.

I can certainly do that! Having a few hints on how to proceed?

I can easily install gdb and/or valgrind as needed:

$ sudo /opt/bin/opkg list | grep -i -e valgrind -e gdb
gdb - 8.2.1-1 - GDB, the GNU Project debugger
valgrind - 3.15.0-1 - Valgrind...

Install valgrind and follow the steps in the wiki. Everything will be slow but it's normally sufficient to start a stream and pause it to cause timeshift to start. Only if that doesnt find anything you can use GDB, you want to look into the allocated memory there to see whats used for what.

Humm, so I tried using either valgrind or gdb but without much luck (not using theses toold that often neither).
Synology boxes might not be that user friendly neither as I'm guessing debug libs may to be missing (guessing)?
A few hints into getting debug tools to work would be quite handy...

Valgrind output:

==23225== Memcheck, a memory error detector
==23225== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==23225== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==23225== Command: /volume1/@appstore/tvheadend/bin/tvheadend -f -u sc-tvheadend -g tvheadend --http_port 9981 --htsp_port 9982 -c /volume1/@appstore/tvheadend/var -p /volume1/@appstore/tvheadend/var/tvheadend.pid
==23225== 

valgrind:  Fatal error at startup: a function redirection
valgrind:  which is mandatory for this platform-tool combination
valgrind:  cannot be set up.  Details of the redirection are:
valgrind:  
valgrind:  A must-be-redirected function
valgrind:  whose name matches the pattern:      strlen
valgrind:  in an object with soname matching:   ld-linux-x86-64.so.2
valgrind:  was not found whilst processing
valgrind:  symbols from the object with soname: ld-linux-x86-64.so.2
...

And here is GDB attached to the PID:

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
0x00007fcfd8b0ac48 in pthread_cond_timedwait () from /lib/libpthread.so.0
(gdb) set logging on
Copying output to gdb.txt.
(gdb) set pagination off
(gdb) bt full
#0  0x00007fcfd8b0ac48 in pthread_cond_timedwait () from /lib/libpthread.so.0
No symbol table info available.
#1  0x00005617f8f3b5c7 in ?? ()
No symbol table info available.
#2  0x00007fcfd826c030 in __libc_start_main () from /lib/libc.so.6
No symbol table info available.
#3  0x00005617f8f3cb71 in ?? ()
No symbol table info available.
(gdb) continue
Continuing.
[Detaching after fork from child process 25805]
[New LWP 25813]
[New LWP 25814]
[LWP 25813 exited]
[LWP 24797 exited]
[New LWP 25945]
[New LWP 25946]
[New LWP 25947]