Tvheadend

I found the answer.

If anyone else is interest then have a look at config_timeshift.html.
Openelec's Tvheadend addon is a good example how to set this https://github.com/OpenELEC/OpenELEC.tv/tree/master/packages/addons/service/multimedia/tvheadend/config

That only relates to if you want timeshift etc...

Generally TVH is pretty lightweight. The installation size (if you use --bundle) is about 5M (I think, from memory). My own configuration (DVB-S dual tuner with ~150 channels is about 40M, 12M of that is EPG data). It has no other temporary files, but will need a few 10M's min for memory.

If you want timeshift and/or recordings then you'll obviously need some disk space somewhere, but that could either be USB/network attached etc...

Adam

Thanks Adam,
Sounds good.

Adam Sutton wrote:

That only relates to if you want timeshift etc...

Generally TVH is pretty lightweight. The installation size (if you use --bundle) is about 5M (I think, from memory). My own configuration (DVB-S dual tuner with ~150 channels is about 40M, 12M of that is EPG data). It has no other temporary files, but will need a few 10M's min for memory.

If you want timeshift and/or recordings then you'll obviously need some disk space somewhere, but that could either be USB/network attached etc...

Adam

I think I have a issue that relates to this. I'm not sure, I'm a newbee so forgive me if I'm talking rubbish.
I'm using Build: 4.1-2309~g0736ebc on my pi2 and when I'm doing 3 recordings at the same time (I've got a hdhomerun with 4 tuners), the pi seems to run out of memory (top: KiB Mem, Free is less than 30MB, but I don't know exactly how much) crashing tvheadend.

Is it normal that tvheadend uses continuously more memory? When I do ps_mem, tvheadend starts with 28MB and ends up more than 700MB's. Is this normal? Or is this "a memory leak"?

I'm not doing timeshifting and recordings are saved to external hard disk attached via usb.

Any help appreciated.

Wim

Could it be related to this post:
https://tvheadend.org/boards/5/topics/3501?r=5049#message-5049

I do have continuity errors most of my recordings. Or is above post not related to that? (if so, sorry).

Ah yes, forgot this too: when I stop and start tvheadend again, stop the recordings, free memory goes up again.

Wim. Your problem sounds like you're overloading the Pi's single USB bus, which is shared with all ports and Ethernet. Try recording just two programmes & see if that makes a difference.

Hi Mark,

Thanks for responding.

I just started 2 recordings, I'll post the result later.

I heard that the usb is indeed the bottleneck of the pi.
I read that a banana pi has dedicated sata and thus doesn't have that problem. Would you advise a banana pi? Or is than not having sse2 (on arm cpu's) the next bottleneck?

I thought my problem was a memory problem so I changed these:
In boot/config.txt I added: gpu_mem_1024=64

And in /etc/dphys-swapfile I changed:
CONF_SWAPSIZE=100 to 1024
and I changed the location of the swap-file to an external harddisk:
CONF_SWAPFILE=/mnt/NDAS/swap.file

I guess that last change made my problem even bigger? Would you advise to make these changes undone? Both?
I must add to that that normally, not the last few weeks when I was having my issue above, I'm running pocketmine. I guess this needs a larger gpu_mem? Or is the gpu_mem only for vnc-viewer?

Thanks.

Wim

Oh, I forgot one more question.
Is there a program I could use to monitor in realtime the USB-bus load?

Wim K wrote:

Hi Mark,

Thanks for responding.

I just started 2 recordings, I'll post the result later.

I heard that the usb is indeed the bottleneck of the pi.
I read that a banana pi has dedicated sata and thus doesn't have that problem. Would you advise a banana pi? Or is than not having sse2 (on arm cpu's) the next bottleneck?

I have a Banana Pi M1. I've ran Tvheadend on it without much issue in the past, the only issue was something related to Armbian's kernel.

I thought my problem was a memory problem so I changed these:
In boot/config.txt I added: gpu_mem_1024=64

And in /etc/dphys-swapfile I changed:
CONF_SWAPSIZE=100 to 1024
and I changed the location of the swap-file to an external harddisk:
CONF_SWAPFILE=/mnt/NDAS/swap.file

I guess that last change made my problem even bigger? Would you advise to make these changes undone? Both?

I personally wouldn't have done it as your just increasing the transfer.

I must add to that that normally, not the last few weeks when I was having my issue above, I'm running pocketmine. I guess this needs a larger gpu_mem? Or is the gpu_mem only for vnc-viewer?

All mine are headless so I can't answer that :p

Thanks.

Wim

Wim K wrote:

Oh, I forgot one more question.
Is there a program I could use to monitor in realtime the USB-bus load?

http://unix.stackexchange.com/questions/10671/usb-performance-traffic-monitor

Mark,

With my limited knowledge of linux, I couldn't get the programs in the link going. Maybe you have another link for me, please?

Yesterday I tried 2 simultaneous sd recordings, that failed. I don't know what's different today, but it didn't crash tvheadend today.
I'll report back about that.

Wim

Wim K wrote:

Mark,

With my limited knowledge of linux, I couldn't get the programs in the link going. Maybe you have another link for me, please?

I don't unfortunately

Yesterday I tried 2 simultaneous sd recordings, that failed. I don't know what's different today, but it didn't crash tvheadend today.

One last thing you can try is setting the DVR profile "stream profile" to mkv (or pass if it's already set to mkv) and "Cache scheme" to don't keep.

I'll report back about that.

Wim

If changing the profile as suggested doesn't work you may want to get a device with a giga Ethernet & more than one USB bus.

Out of interest, can you actually play 3 streams at the same time without issue? Or is it just recording?

Mark Clarkstone wrote:

Wim K wrote:

Mark,

With my limited knowledge of linux, I couldn't get the programs in the link going. Maybe you have another link for me, please?

I don't unfortunately

I'm trying wireshark now.

Yesterday I tried 2 simultaneous sd recordings, that failed. I don't know what's different today, but it didn't crash tvheadend today.

One last thing you can try is setting the DVR profile "stream profile" to mkv (or pass if it's already set to mkv) and "Cache scheme" to don't keep.

I read the help pages for "cache scheme":
"Don't keep: Tell the system that you’re not expecting to re-use the data soon, so don’t keep it in cache. The data will still be buffered for writing. Useful e.g. in a RAM-limited system like a Pi (given that you’re unlikely to be watching while recording, so data can be discarded now and read back from disc later)."

I guess this means that when you have a Pi, you should know you shouldn't watch delayed? I didn't know it would keep the buffer in ram.
I was just about to test, and this confirms that this may be the problem, watching 1 recording with a delay (start playing the recording, press pause, put the kids to bed and than press play) and the other just recording. That was what happened 3 fridays evenings in a row.
Would that not be possible with cache scheme "Don't keep"? Wouldn't tvheadend read it from external harddrive again?

I'll report back about that.

Wim

If changing the profile as suggested doesn't work you may want to get a device with a giga Ethernet & more than one USB bus.

That's why I was thinking banana pi.

---

Out of interest, can you actually play 3 streams at the same time without issue? Or is it just recording?

I'm going to test a bit more. Will let you know. Much easier to do with a monitor-program, so I'll try to do that first.

I set the swap-file back to original state.

I tested almost everything. I can record max 3 shows simultaneously without RAM going too high and I can watch 1 of them at the same time, pause it, etc. Changing the cache scheme (was set to system) didn't make a difference (stream profile was already set to pass).
I can watch at least (haven't been able to try more at the moment) 2 different streams.

The only thing I haven't tested is using the androidbox I used when the problem occurred 3 fridays in a row. Maybe that's were the problem is?
Perhaps later today or friday.

Will let you know.

Thanks so far,

Wim
ps In this thread you referred to: http://unix.stackexchange.com/questions/10671/usb-performance-traffic-monitor

did you mean the 2 shell scripts on this website:

http://pipe.oliveira-carvalho.com/blog/tech/monitorusbthroughput

or did you mean usbtop (if you meant this, I will try it again because wireshark is no good for me)?

EDIT: got usbtop working :-)
Checking for USB problems now.

TVheadend just crashed (18:31 PM). SysLog attached.

Wim

In the test from the previous post: Pi ran out of memory after maybe half an hour recording, cache scheme: system.

Today I just tried again with cache scheme: don't keep and it crashed too.

Which of the following are the relevant memory items to watch in top: VIRT, RES, Swap free, Mem free?
VIRT is already pretty high when not recording at all.

Wim K wrote:

In the test from the previous post: Pi ran out of memory after maybe half an hour recording, cache scheme: system.

Today I just tried again with cache scheme: don't keep and it crashed too.

Which of the following are the relevant memory items to watch in top: VIRT, RES, Swap free, Mem free?
VIRT is already pretty high when not recording at all.

https://www.maketecheasier.com/linux-top-explained/.

Looking at those logs Tvheadend hadn't crashed. What is happening is that it keeps being killed as you're running out of memory. If it was a crash you'd have seen a line starting with "CRASH:" from tvheadend before it exits.

On my Pi2 this is all I have running.

UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 Feb13 ?        00:00:11 /sbin/init
root         2     0  0 Feb13 ?        00:00:00 [kthreadd]
root         3     2  0 Feb13 ?        00:01:40 [ksoftirqd/0]
root         5     2  0 Feb13 ?        00:00:00 [kworker/0:0H]
root         7     2  0 Feb13 ?        00:00:12 [rcu_sched]
root         8     2  0 Feb13 ?        00:00:00 [rcu_bh]
root         9     2  0 Feb13 ?        00:00:00 [migration/0]
root        10     2  0 Feb13 ?        00:00:00 [migration/1]
root        11     2  0 Feb13 ?        00:00:02 [ksoftirqd/1]
root        13     2  0 Feb13 ?        00:00:00 [kworker/1:0H]
root        14     2  0 Feb13 ?        00:00:00 [migration/2]
root        15     2  0 Feb13 ?        00:00:01 [ksoftirqd/2]
root        17     2  0 Feb13 ?        00:00:00 [kworker/2:0H]
root        18     2  0 Feb13 ?        00:00:00 [migration/3]
root        19     2  0 Feb13 ?        00:00:01 [ksoftirqd/3]
root        21     2  0 Feb13 ?        00:00:00 [kworker/3:0H]
root        22     2  0 Feb13 ?        00:00:00 [kdevtmpfs]
root        23     2  0 Feb13 ?        00:00:00 [netns]
root        24     2  0 Feb13 ?        00:00:00 [perf]
root        25     2  0 Feb13 ?        00:00:00 [khungtaskd]
root        26     2  0 Feb13 ?        00:00:00 [writeback]
root        27     2  0 Feb13 ?        00:00:00 [crypto]
root        28     2  0 Feb13 ?        00:00:00 [bioset]
root        29     2  0 Feb13 ?        00:00:00 [kblockd]
root        31     2  0 Feb13 ?        00:00:00 [rpciod]
root        33     2  0 Feb13 ?        00:00:00 [kswapd0]
root        34     2  0 Feb13 ?        00:00:00 [vmstat]
root        35     2  0 Feb13 ?        00:00:00 [fsnotify_mark]
root        36     2  0 Feb13 ?        00:00:00 [nfsiod]
root        45     2  0 Feb13 ?        00:00:00 [kthrotld]
root        46     2  0 Feb13 ?        00:00:00 [bioset]
root        47     2  0 Feb13 ?        00:00:00 [bioset]
root        48     2  0 Feb13 ?        00:00:00 [bioset]
root        49     2  0 Feb13 ?        00:00:00 [bioset]
root        50     2  0 Feb13 ?        00:00:00 [bioset]
root        51     2  0 Feb13 ?        00:00:00 [bioset]
root        52     2  0 Feb13 ?        00:00:00 [bioset]
root        53     2  0 Feb13 ?        00:00:00 [bioset]
root        54     2  0 Feb13 ?        00:00:00 [bioset]
root        55     2  0 Feb13 ?        00:00:00 [bioset]
root        56     2  0 Feb13 ?        00:00:00 [bioset]
root        57     2  0 Feb13 ?        00:00:00 [bioset]
root        58     2  0 Feb13 ?        00:00:00 [bioset]
root        59     2  0 Feb13 ?        00:00:00 [bioset]
root        60     2  0 Feb13 ?        00:00:00 [bioset]
root        61     2  0 Feb13 ?        00:00:00 [bioset]
root        62     2  0 Feb13 ?        00:00:00 [bioset]
root        63     2  0 Feb13 ?        00:00:00 [bioset]
root        64     2  0 Feb13 ?        00:00:00 [bioset]
root        65     2  0 Feb13 ?        00:00:00 [bioset]
root        66     2  0 Feb13 ?        00:00:00 [bioset]
root        67     2  0 Feb13 ?        00:00:00 [bioset]
root        68     2  0 Feb13 ?        00:00:00 [bioset]
root        69     2  0 Feb13 ?        00:00:00 [bioset]
root        70     2  0 Feb13 ?        00:00:00 [VCHIQ-0]
root        71     2  0 Feb13 ?        00:00:00 [VCHIQr-0]
root        72     2  0 Feb13 ?        00:00:00 [VCHIQs-0]
root        73     2  0 Feb13 ?        00:00:00 [iscsi_eh]
root        74     2  0 Feb13 ?        00:00:00 [dwc_otg]
root        75     2  0 Feb13 ?        00:00:00 [DWC Notificatio]
root        77     2  0 Feb13 ?        00:00:00 [VCHIQka-0]
root        78     2  0 Feb13 ?        00:00:00 [deferwq]
root        81     2  0 Feb13 ?        00:00:00 [bioset]
root        82     2  0 Feb13 ?        00:00:05 [mmcqd/0]
root        84     2  0 Feb13 ?        00:00:01 [jbd2/mmcblk0p2-]
root        85     2  0 Feb13 ?        00:00:00 [ext4-rsv-conver]
root        87     2  0 Feb13 ?        00:00:00 [ipv6_addrconf]
root       130     1  0 Feb13 ?        00:00:14 /lib/systemd/systemd-journald
root       137     1  0 Feb13 ?        00:00:00 /lib/systemd/systemd-udevd
root       194     2  0 Feb13 ?        00:00:00 [rc0]
root       232     2  0 Feb13 ?        00:00:00 [kworker/2:1H]
root       234     1  0 Feb13 ?        00:00:00 dhclient -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/
root       255     1  0 Feb13 ?        00:00:00 /usr/sbin/sshd -D
root       256     1  0 Feb13 ?        00:00:00 /usr/sbin/cron -f
root       258     1  0 Feb13 ?        00:00:02 /lib/systemd/systemd-logind
message+   264     1  0 Feb13 ?        00:00:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork
ntp        284     1  0 Feb13 ?        00:00:37 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 105:109
root       311     1  0 Feb13 ?        00:00:02 /usr/sbin/rsyslogd -n
root       317     1  0 Feb13 tty1     00:00:00 /sbin/agetty --noclear tty1 linux
root       319     1  0 Feb13 ttyAMA0  00:00:00 /sbin/agetty --keep-baud 115200 38400 9600 ttyAMA0 vt102
hts        325     1 15 Feb13 ?        10:06:04 /usr/bin/tvheadend -f -u hts -g video
root       350     2  0 Feb13 ?        00:00:00 [kworker/0:1H]
root       351     2  0 Feb13 ?        00:00:30 [kdvb-ad-1-fe-0]
root       388     2  0 Feb13 ?        00:00:00 [kworker/1:1H]
root       477     2  0 Feb13 ?        00:00:13 [kdvb-ad-0-fe-0]
root       522     2  0 Feb13 ?        00:00:00 [kworker/3:1H]
root      2682     2  0 Feb14 ?        00:00:00 [kworker/u8:2]
root      2697     2  0 Feb14 ?        00:00:01 [kworker/2:2]
root      3637     2  0 Feb15 ?        00:00:00 [kworker/2:1]
root      3677     2  0 Feb15 ?        00:00:03 [kworker/0:0]
root      4075     2  0 03:56 ?        00:00:02 [kworker/u8:1]
root      4307     2  0 06:39 ?        00:00:00 [kworker/0:1]
root      4387     2  0 08:26 ?        00:00:00 [kworker/3:0]
root      4390     2  0 08:31 ?        00:00:01 [kworker/3:1]
root      4391     2  0 08:33 ?        00:00:02 [kworker/1:2]
root      4392     2  0 08:40 ?        00:00:00 [kworker/1:1]
root      4395     2  0 08:48 ?        00:00:00 [kworker/1:0]
root      4398   255  0 08:50 ?        00:00:00 sshd: root@pts/0    
root      4403     1  0 08:51 ?        00:00:00 /lib/systemd/systemd --user
root      4406  4403  0 08:51 ?        00:00:00 (sd-pam)  
root      4409  4398  0 08:51 pts/0    00:00:00 -bash
root      4414  4409  1 08:51 pts/0    00:00:01 top
root      4415     2  0 08:51 ?        00:00:00 [kworker/u8:0]
root      4417     2  0 08:52 ?        00:00:00 [kworker/3:2]
root      4418     2  0 08:52 ?        00:00:00 [kworker/2:0]
root      4419   255  1 08:52 ?        00:00:00 sshd: root@pts/1    
root      4425  4419  0 08:52 pts/1    00:00:00 -bash
root      4430     2  0 08:52 ?        00:00:00 [kworker/1:3]
root      4433  4425  0 08:52 pts/1    00:00:00 ps -ef

$ free -m
             total       used       free     shared    buffers     cached
Mem:           973        221        751         12         23        105
-/+ buffers/cache:         91        881
Swap:            0          0          0

  325 hts       20   0  339.5m  52.4m   5.1m S  13.5  5.4 606:22.91 tvheadend    

The RES (52.4m) is actually very small, this is with two usb tuners grabbing EPG & being idle the rest of the day. On my other system with 5 tuners (2 pci cards (1 dual tuner), 2 usb) Tvheadend is using:

  533 hts       20   0 1415.8m 114.3m   6.0m S   0.3  3.8  90:35.33 tvheadend 

I think you may have two problems. The big one, the pi's usb bus being a bottleneck, especially shoving 4 streams through it and blurting some back out again to your hard drive :o.

I mean look what happens to my Pi just streaming two HD services..

Notice the high bit rate & the errors..

Ok, done some testing using RPI-monitor, top, etc.

Watching four streams at the same time: no problem.

Recording 3 (sd) streams gives after some time:
Processor load increases.
Network traffic stays the same all the time.
"mpegts: too much queued table input data (over 2MB), discarding new"
Available memory (don't know what that would be though) goes down, free memory remains the same.
Process tvheadend is terminated.

IMHO it looks like streaming to other devices via ethernet is not a problem, getting the data through the usb is.
So what if I connect the harddisk via ethernet (I have a NDAS)? I'm going to give that a try.

Wim K wrote:

Ok, done some testing using RPI-monitor, top, etc.

Watching four streams at the same time: no problem.

By four streams you mean all unique and not on the same mux (using all tuners).

Recording 3 (sd) streams gives after some time:
Processor load increases.
Network traffic stays the same all the time.
"mpegts: too much queued table input data (over 2MB), discarding new"
Available memory (don't know what that would be though) goes down, free memory remains the same.
Process tvheadend is terminated.

IMHO it looks like streaming to other devices via ethernet is not a problem, getting the data through the usb is.
So what if I connect the harddisk via ethernet (I have a NDAS)? I'm going to give that a try.

Won't hurt to try

Mark Clarkstone wrote:

Wim K wrote:

Ok, done some testing using RPI-monitor, top, etc.

Watching four streams at the same time: no problem.

By four streams you mean all unique and not on the same mux (using all tuners).

Recording 3 (sd) streams gives after some time:
Processor load increases.
Network traffic stays the same all the time.
"mpegts: too much queued table input data (over 2MB), discarding new"
Available memory (don't know what that would be though) goes down, free memory remains the same.
Process tvheadend is terminated.

IMHO it looks like streaming to other devices via ethernet is not a problem, getting the data through the usb is.
So what if I connect the harddisk via ethernet (I have a NDAS)? I'm going to give that a try.

Won't hurt to try

Remember, though: The USB and Ethernet shame the same bus. Plus, the Pi only has 100Mbps ethernet, not GbE.

Mark Clarkstone wrote:

Wim K wrote:

Ok, done some testing using RPI-monitor, top, etc.

Watching four streams at the same time: no problem.

By four streams you mean all unique and not on the same mux (using all tuners).

Recording 3 (sd) streams gives after some time:
Processor load increases.
Network traffic stays the same all the time.
"mpegts: too much queued table input data (over 2MB), discarding new"
Available memory (don't know what that would be though) goes down, free memory remains the same.
Process tvheadend is terminated.

IMHO it looks like streaming to other devices via ethernet is not a problem, getting the data through the usb is.
So what if I connect the harddisk via ethernet (I have a NDAS)? I'm going to give that a try.

Won't hurt to try

I'll try again with multiple channels watching and make sure these are different muxes.

Robert Cameron wrote:

Mark Clarkstone wrote:

Wim K wrote:

Ok, done some testing using RPI-monitor, top, etc.

Watching four streams at the same time: no problem.

By four streams you mean all unique and not on the same mux (using all tuners).

Recording 3 (sd) streams gives after some time:
Processor load increases.
Network traffic stays the same all the time.
"mpegts: too much queued table input data (over 2MB), discarding new"
Available memory (don't know what that would be though) goes down, free memory remains the same.
Process tvheadend is terminated.

IMHO it looks like streaming to other devices via ethernet is not a problem, getting the data through the usb is.
So what if I connect the harddisk via ethernet (I have a NDAS)? I'm going to give that a try.

Won't hurt to try

Remember, though: The USB and Ethernet shame the same bus. Plus, the Pi only has 100Mbps ethernet, not GbE.

I understand, but looking at the graphs in RPI-monitor, ethernet is not a big problem at all. Besides, I can stream to 4 different devices at the same time, no problem for the cpu.
I think that pushing via usb makes the difference for the cpu, causing the ram problem. Then cpu or ram (I don't know which) issue is stopping the process tvheadend.

Well, that's not so easy although I've gotten pretty far already I think.

BUT in order to make sure first it's worth all the trouble I thought it would be a good idea to mount a share from my windows pc first to see if it works in theory.
So as user hts, I have mounted a windows share, but no matter what I try user hts is not getting permission to write to that directory although I have setup the share correct in windows (everybody, read-write access).
Could anyone please help me with this?

Wim

Never mind, sorry, I've got it. Testing wright now.