Bug #2309
Another memory leak?
Status:
Fixed
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
2014-09-19
Due date:
% Done:
0%
Estimated time:
Found in version:
3.9.1526~g4bce3dc
Affected Versions:
Description
TVHeadend Build: 3.9.1526~g4bce3dc
Intel NUC i5, two sundtek DVB-C usb sticks, Arch Linux running 3.16.2-1, 8GB Ram, Samsung Evo 256GB 2,5" SSD.
Partitions:
Number Start (sector) End (sector) Size Code Name 1 2048 1050623 512.0 MiB EF00 EFI System 2 1050624 66867199 31.4 GiB 8300 Linux filesystem 3 66867200 486299647 200.0 GiB 8300 Linux filesystem 4 486299648 488397134 1.0 GiB 8200 Linux swap
root@TVServer ~ > lslkb -f NAME FSTYPE LABEL UUID MOUNTPOINT sda ├─sda1 vfat EFI 31D8-D2FA /boot ├─sda2 btrfs arch 8a168fb7-b757-428e-8553-408cf7e844ab /run/btrfs-root ├─sda3 ext4 video 7002a830-ddba-4622-98ec-2a472d011d0d /home └─sda4 swap swap 0326181a-ff7e-48e8-a337-28a705881e72 [SWAP]
sda2 is root running btrfs and sda3 is mounted on home for recordings. 200GB left. Harddisk is a 2,5 SSD Samsung Evo 256GB.
root@TVServer /home/hts/.hts/tvheadend/dvr/config > cat /etc/fstab # # /etc/fstab: static file system information # # <file system> <dir> <type> <options> <dump> <pass> # /dev/sdb2 LABEL=arch UUID=8a168fb7-b757-428e-8553-408cf7e844ab / btrfs rw,nodev,relatime,ssd,space_cache,subvol=root 0 0 # mount BTRFS root UUID=8a168fb7-b757-428e-8553-408cf7e844ab /run/btrfs-root btrfs nodev,nosuid,noexec 0 0 # /dev/sdb1 LABEL=EFI UUID=31D8-D2FA /boot vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro 0 2 # /dev/sdb3 LABEL=video UUID=7002a830-ddba-4622-98ec-2a472d011d0d /home ext4 rw,relatime,data=ordered 0 2
I have TVH running for about 2 months now and its getting more and more perfect for my needs.
But one thing makes me wonder quite a bit... I never really used recordings in the past, but now I wanted to give it a try.
After a night of three (scrambled) records of HD channels my memory consumption looks like this:
root@TVServer ~ > free -m
total used free shared buffers cached
Mem: 7917 7638 279 8 2 90
-/+ buffers/cache: 7545 372
Swap: 1024 103 920top - 07:44:44 up 9:15, 1 user, load average: 0,05, 0,11, 0,16 Tasks: 105 total, 1 running, 104 sleeping, 0 stopped, 0 zombie %Cpu(s): 1,3 us, 1,2 sy, 0,0 ni, 97,5 id, 0,0 wa, 0,0 hi, 0,0 si, 0,0 st KiB Mem: 8107844 total, 7823048 used, 284796 free, 2600 buffers KiB Swap: 1048736 total, 106468 used, 942268 free. 93468 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 390 hts 20 0 9906080 7,192g 7576 S 6,6 93,0 13:55.05 tvheadend 388 ntp 20 0 109428 15388 5952 S 0,0 0,2 0:02.08 ntpd 144 root 20 0 131952 15232 15092 S 0,0 0,2 0:03.00 systemd-journal 412 root 20 0 457960 12720 10024 S 4,7 0,2 13:48.41 mediasrv 1090 root 20 0 84592 6176 5276 S 0,0 0,1 0:00.05 sshd
root@TVServer ~ > cat/proc/meminfo MemTotal: 8107844 kB MemFree: 286580 kB MemAvailable: 251020 kB Buffers: 2600 kB Cached: 93468 kB SwapCached: 52232 kB Active: 6363132 kB Inactive: 1320756 kB Active(anon): 6325776 kB Inactive(anon): 1276972 kB Active(file): 37356 kB Inactive(file): 43784 kB Unevictable: 15380 kB Mlocked: 15380 kB SwapTotal: 1048736 kB SwapFree: 942276 kB Dirty: 0 kB Writeback: 0 kB AnonPages: 7552492 kB Mapped: 34444 kB Shmem: 8964 kB Slab: 31072 kB SReclaimable: 16668 kB SUnreclaim: 14404 kB KernelStack: 2400 kB PageTables: 17328 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 5102656 kB Committed_AS: 489048 kB VmallocTotal: 34359738367 kB VmallocUsed: 352800 kB VmallocChunk: 34359376892 kB HardwareCorrupted: 0 kB AnonHugePages: 3807232 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 41404 kB DirectMap2M: 1982464 kB DirectMap1G: 6291456 kB
So this is no disk cache in my opinion. TVH grabs an incredible amount of 7GB RAM without freeing it. Records are written on disk.
My default recording profile looks like this:
root@TVServer /home/hts/.hts/tvheadend/dvr/config > cat 984b43e397babcd498718a714df6fad5
{
"enabled": true,
"name": "",
"container": 1,
"cache": 1,
"retention-days": 31,
"pre-extra-time": 1,
"post-extra-time": 4,
"episode-duplicate-detection": true,
"epg-update-window": 86400,
"storage": "/home/hts/video",
"file-permissions": "0664",
"charset": "UTF-8",
"rewrite-pat": false,
"rewrite-pmt": false,
"tag-files": false,
"skip-commercials": false,
"directory-permissions": "0775",
"day-dir": false,
"channel-dir": false,
"title-dir": false,
"channel-in-title": true,
"date-in-title": true,
"time-in-title": true,
"episode-in-title": false,
"episode-before-date": false,
"subtitle-in-title": false,
"omit-title": false,
"clean-title": false,
"whitespace-in-title": false
}
Playing around with cache settings in my recording profile doesn't seem to change anything. When looking at the memory consumption during recording the system uses disk cache correctly assigned as cached mem but TVH also assigns memory without freeing it.
Let me start a record...
Sep 19 07:58:37 TVServer tvheadend[390]: dvr: "Guten Morgen Deutschland" on "RTL HD" recorder starting Sep 19 07:58:37 TVServer tvheadend[390]: subscription: "DVR: Guten Morgen Deutschland" subscribing on "RTL HD", weight: 300, adapter: "Sundtek DVB-C : DVB-C #0", network: "Kabel Deutschland", mux: "346", provider: "Digital Free", service: "RTL HD" Sep 19 07:58:38 TVServer tvheadend[390]: dvr: /home/hts/video/RTL HD-Guten Morgen Deutschland.2014-09-19.06-00.mkv from adapter: "Sundtek DVB-C : DVB-C #0", network: "Kabel Deutschland", mux: "346", provider: "Digital Free", service: "RTL HD" Sep 19 07:58:38 TVServer tvheadend[390]: dvr: # type lang resolution aspect ratio sample rate channels Sep 19 07:58:38 TVServer tvheadend[390]: dvr: 1 DVBSUB ger Sep 19 07:58:38 TVServer tvheadend[390]: dvr: 2 H264 1920x1080 16:9 Sep 19 07:58:38 TVServer tvheadend[390]: dvr: 3 AC3 ger 48000 2 Sep 19 07:58:38 TVServer tvheadend[390]: dvr: 4 TELETEXT <disabled, no valid input> Sep 19 07:58:38 TVServer tvheadend[390]: dvr: 5 CA Sep 19 07:58:38 TVServer tvheadend[390]: dvr: 6 CA
root@TVServer /home/hts/.hts/tvheadend/dvr/config > iostat 1
Linux 3.16.2-1-ARCH (TVServer) 19.09.2014 _x86_64_ (4 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0,57 0,00 0,50 0,07 0,00 98,86
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 5,87 469,99 837,40 16088764 28665716
avg-cpu: %user %nice %system %iowait %steal %idle
3,03 0,00 1,52 0,00 0,00 95,45
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 0,00 0,00 0,00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
3,30 0,00 1,27 0,25 0,00 95,18
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 2,00 0,00 12,00 0 12
avg-cpu: %user %nice %system %iowait %steal %idle
3,05 0,00 1,27 0,00 0,00 95,69
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 0,00 0,00 0,00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
2,79 0,00 1,02 0,00 0,00 96,19
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 0,00 0,00 0,00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
3,05 0,00 0,76 0,00 0,00 96,19
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 0,00 0,00 0,00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
3,30 0,00 1,02 0,00 0,00 95,69
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 0,00 0,00 0,00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
3,54 0,00 1,27 0,00 0,00 95,19
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 0,00 0,00 0,00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
2,54 0,00 1,27 0,00 0,00 96,18
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 37,00 0,00 17348,00 0 17348
avg-cpu: %user %nice %system %iowait %steal %idle
3,02 0,00 1,26 0,00 0,00 95,72
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 0,00 0,00 0,00 0 0
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 1 0 106424 150352 800 109728 0 1 118 211 192 55 1 0 99 0 0 0 0 106424 148000 800 111112 0 0 0 0 1936 3271 3 1 96 0 0 0 0 106424 146904 808 111172 0 0 0 12 1881 3328 3 1 96 0 0 0 0 106424 144220 808 112880 0 0 0 0 1950 3239 3 1 96 0 0 0 0 106424 141500 808 114416 0 0 0 0 1793 3247 3 1 95 0 0 0 0 106424 140016 808 114456 0 0 0 0 1806 3281 3 1 96 0 0 1 0 106424 137520 808 115900 0 0 0 11260 1809 3223 3 1 96 0 0 0 0 106424 134712 808 117440 0 0 0 0 1781 3212 3 1 96 0 0 0 0 106424 132108 816 118752 0 0 0 20 1852 3259 4 1 95 0 0 0 0 106424 130944 816 118848 0 0 0 0 1791 3243 2 1 97 0 0 0 0 106424 128048 816 120828 0 0 0 0 1793 3261 2 1 97 0 0 0 0 106424 126924 816 120808 0 0 0 0 1774 3264 3 1 96 0 0 0 0 106424 124236 816 122296 0 0 0 0 1717 3205 3 1 96 0 0 0 0 106424 121772 816 123716 0 0 0 0 1970 3347 3 1 96 0 0 0 0 106424 120540 824 123708 0 0 0 12 1974 3360 3 1 96 0 0 1 0 106424 145572 824 97476 0 0 0 0 1877 3287 3 1 96 0 0 0 0 106424 143868 824 99068 0 0 0 11128 1807 3250 3 2 95 0 0 0 0 106424 141536 824 100464 0 0 0 0 1800 3263 3 1 96 0 0 0 0 106424 139244 824 101444 0 0 0 0 1827 3249 3 1 97 0 0 0 0 106424 137972 824 101548 0 0 0 0 1829 3204 2 2 96 0 0 0 0 106424 135368 832 102924 0 0 0 20 1801 3243 2 1 96 0 0 0 0 106424 132776 832 104360 0 0 0 0 1831 3274 3 1 97 0 0 0 0 106424 131492 832 104360 0 0 0 0 1801 3251 3 2 95 0 0 0 0 106424 128856 832 106144 0 0 0 0 1757 3230 2 1 97 0 0 0 0 106424 126512 832 107684 0 0 0 0 1741 3170 3 1 96 0 0 0 0 106424 123820 840 108876 0 0 0 10192 1777 3232 2 2 96 0 0 1 0 106424 122788 840 109104 0 0 0 0 1766 3224 2 1 97 0 0 0 0 106424 120868 840 110448 0 0 0 0 1818 3270 2 1 96 0 0 0 0 108148 148932 836 89020 0 1724 0 1724 1869 3275 3 1 95 0 0
After some minutes:
top - 08:01:33 up 9:32, 2 users, load average: 0,21, 0,22, 0,17
Tasks: 119 total, 1 running, 118 sleeping, 0 stopped, 0 zombie
%Cpu(s): 2,7 us, 1,3 sy, 0,0 ni, 96,0 id, 0,0 wa, 0,0 hi, 0,1 si, 0,0 st
KiB Mem: 8107844 total, 7970248 used, 137596 free, 384 buffers
KiB Swap: 1048736 total, 109668 used, 939068 free. 80156 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
390 hts 20 0 9898,1m 7,379g 7216 S 13,0 95,4 15:05.31 tvheadend
412 root 20 0 457960 13016 9548 S 3,7 0,2 14:27.79 mediasrv
1090 root 20 0 84592 6176 5276 S 0,3 0,1 0:00.13 sshd
1 root 20 0 29264 4252 2788 S 0,0 0,1 0:01.33 systemd
2 root 20 0 0 0 0 S 0,0 0,0 0:00.00 kthreadd
3 root 20 0 0 0 0 S 0,0 0,0 0:00.52 ksoftirqd/0
5 root 0 -20 0 0 0 S 0,0 0,0 0:00.00 kworker/0:0H
7 root 20 0 0 0 0 S 0,0 0,0 0:01.10 rcu_preempt
Aborting now as the swapout is rising. Soon the process will be killed.
I must admit I never really watched memory consumption before I raised memory from 4GB to 8GB and thinking about a ramdisk for my TVH. I played around with it and it worked pretty well but then I saw that memory consumption during a record in combination with timeshift is noticably too high so I started to sort things out. It wasn't the tmpfs for TS, it was the recording itself. Right now the ramdisk and TS itself is disabled cause its hard to find the best amout of assigned memory right in this situation.
So actually I have no clue. Do you have any idea what is going on here?
Updated by