Bug #3466
closed
Out of Memory -> KILLED
Added by Alfred Zastrow over 9 years ago. Updated over 8 years ago.
0%
Description
Just as an adhoc info, it happend during a recording & live view:
[ 5780.489863] Out of memory: Kill process 913 (tvh:lnxdvb-fron) score 960 or sacrifice child
[ 5780.489930] Killed process 913 (tvh:lnxdvb-fron) total-vm:5666264kB, anon-rss:3593032kB, file-rss
:0kB
My tvh-server has 4 GB and this never happend.
Sorry for no-debug. I will install the debug version tomorrow, today I'm back to 1264 because I need to reccord for my wife (WAF-important :-))
Alfred
Files
| Bildschirmfoto vom 2016-01-02 06-56-41.png (46.3 KB) Bildschirmfoto vom 2016-01-02 06-56-41.png | Alfred Zastrow, 2016-01-02 07:06 | ||
| vg.log (1.49 MB) vg.log | C K, 2016-01-13 15:12 |
Updated by Alfred Zastrow over 9 years ago
This was logged yesterday, I will try latest today.
[ 5780.486718] tvh:tshift-rd invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
[ 5780.486723] tvh:tshift-rd cpuset=/ mems_allowed=0
[ 5780.486730] CPU: 2 PID: 919 Comm: tvh:tshift-rd Tainted: G O 4.3.0-1-amd64 #1 Debian 4.3.3-2
[ 5780.486732] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Q2900M, BIOS P1.00 07/24/2014
[ 5780.486734] 0000000000000000 000000008251b1cd ffffffff812ddc89 ffff880036a743c0
[ 5780.486739] ffffffff811c9ff6 ffff8800a8dd8918 0000000000000000 0000000000000000
[ 5780.486742] ffff8800a851a800 ffffffffa0544a5d ffff8800a8dd0000 000000008251b1cd
[ 5780.486746] Call Trace:
[ 5780.486755] [<ffffffff812ddc89>] ? dump_stack+0x40/0x57
[ 5780.486760] [<ffffffff811c9ff6>] ? dump_header+0x83/0x20a
[ 5780.486786] [<ffffffffa0544a5d>] ? i915_gem_shrinker_oom+0x19d/0x1f0 [i915]
[ 5780.486790] [<ffffffff8116282d>] ? oom_kill_process+0x1fd/0x400
[ 5780.486793] [<ffffffff81162201>] ? oom_unkillable_task+0xc1/0x100
[ 5780.486798] [<ffffffff8126103e>] ? security_capable_noaudit+0x3e/0x60
[ 5780.486801] [<ffffffff81162c7a>] ? out_of_memory+0x1fa/0x470
[ 5780.486804] [<ffffffff811684cd>] ? __alloc_pages_nodemask+0x8bd/0xa10
[ 5780.486808] [<ffffffff811abfba>] ? alloc_pages_current+0x8a/0xf0
[ 5780.486812] [<ffffffff81161301>] ? filemap_fault+0x151/0x430
[ 5780.486815] [<ffffffff81189cef>] ? __do_fault+0x4f/0xe0
[ 5780.486819] [<ffffffff8118eb4b>] ? handle_mm_fault+0xecb/0x16b0
[ 5780.486824] [<ffffffff81060338>] ? __do_page_fault+0x198/0x430
[ 5780.486828] [<ffffffff812131a3>] ? SyS_epoll_wait+0x93/0xd0
[ 5780.486832] [<ffffffff815834b8>] ? page_fault+0x28/0x30
[ 5780.486835] Mem-Info:
[ 5780.486842] active_anon:902427 inactive_anon:2166 isolated_anon:0
active_file:69 inactive_file:123 isolated_file:0
unevictable:0 dirty:0 writeback:0 unstable:0
slab_reclaimable:6015 slab_unreclaimable:4757
mapped:317 shmem:2323 pagetables:2434 bounce:0
free:6488 free_pcp:313 free_cma:0
Updated by Alfred Zastrow over 9 years ago
With HTS Tvheadend 4.1-1284~ga5085c8 I couldn't reproduce the crash so far but what I've seen is that in time shift mode old data is not deleted. I've configured 1 GB as Ram-Buffer and even with a time shift of some seconds this seems to be filled over the time (screenshoot) and then the next buffers are written on the disc.
I will cancel my ram buffer to not running into short of memory situations anymore.
Maybe you can close this ticket.
btw. the advantage of the memory buffer is smaler now, because the disc is needed anyway after some time.
Updated by Alfred Zastrow over 9 years ago
I'm back to 1264 - this is the last "stable" version for me.
1284 also as 1286 hangs without a usable message after few seconds :-(
Updated by Alfred Zastrow over 9 years ago
without using timeshift, only two clients.
Updated by Njuskalo Njusko over 9 years ago
4.1-1258~gb480c92 is the last stable version in my case, using IPTV only. In all subsequent releases after zapping a while the tvh process gets unresponsive. Not even using timeshift nor recording. Using Kodi Jarvis B4 as a client.
Updated by Mark Clarkstone over 9 years ago
Njuskalo Njusko wrote:
4.1-1258~gb480c92 is the last stable version in my case, using IPTV only. In all subsequent releases after zapping a while the tvh process gets unresponsive. Not even using timeshift nor recording. Using Kodi Jarvis B4 as a client.
This is pretty much the same for me but DVB-S/S2/T/T2, I will continue to test git master.
Updated by Jaroslav Kysela over 9 years ago
- Status changed from New to Fixed
I believe, it's fixed in current master.
Updated by C K over 9 years ago
Jaroslav Kysela wrote:
I believe, it's fixed in current master.
Hmmm, just got a crash.
2016-01-13 15:08:43.192 [ INFO] htsp: Got connection from 192.168.178.22 2016-01-13 15:08:43.599 [ INFO] htsp: 192.168.178.22: Welcomed client software: Kodi Media Center (HTSPv18) 2016-01-13 15:08:43.878 [ INFO] transcode: 0001: 1:H264 1920x1080 ==> H264 1280x720 (libx264) 2016-01-13 15:08:43.884 [ INFO] transcode: 0001: 2:AC3 ==> Filtered 2016-01-13 15:08:43.888 [ INFO] transcode: 0001: 3:CA ==> Filtered 2016-01-13 15:08:43.888 [ INFO] transcode: 0001: 4:CA ==> Filtered tvheadend: src/plumbing/globalheaders.c:284: gh_hold: Zusicherung »ssc != ((void *)0)« nicht erfüllt. 2016-01-13 15:08:44.208 [ ALERT] CRASH: Signal: 6 in PRG: ./build.linux/tvheadend (4.1-1354~g5966e61) [8f2762c52b2cb566663da57a27737fd119d48d86] CWD: /home/waldmeister/src/tvheadend 2016-01-13 15:08:44.212 [ ALERT] CRASH: Fault address 0x3e800004322 (N/A) 2016-01-13 15:08:44.213 [ ALERT] CRASH: Loaded libraries: /usr/lib/valgrind/vgpreload_core-amd64-linux.so /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so /lib/x86_64-linux-gnu/libssl.so.1.0.0 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 /lib/x86_64-linux-gnu/libz.so.1 /usr/lib/liburiparser.so.1 /usr/lib/x86_64-linux-gnu/libavahi-common.so.3 /usr/lib/x86_64-linux-gnu/libavahi-client.so.3 /lib/x86_64-linux-gnu/libdbus-1.so.3 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/librt.so.1 /lib/x86_64-linux-gnu/libc.so.6 /lib64/ld-linux-x86-64.so.2 2016-01-13 15:08:44.214 [ ALERT] CRASH: Register dump [23]: 000000000000006a0000000000000000000000000000000800000000000000010000000000b678340000000000b679b80000000037d81180000000001e910ae0000000000000432200000000000043590000000004061feb000000000402c00000000000000000060000000000000000000000000686ccc90000000037d80ec8000000000686ccc9000000000000000400000000000000000000000000000006000000000000000e0000000000000000000003e800004322 2016-01-13 15:08:44.215 [ ALERT] CRASH: STACKTRACE 2016-01-13 15:08:44.285 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/trap.c:148 0x46bed9 2016-01-13 15:08:44.335 [ ALERT] CRASH: ??:0 0x611a340 2016-01-13 15:08:44.336 [ ALERT] CRASH: gsignal+0x39 (/lib/x86_64-linux-gnu/libc.so.6) 2016-01-13 15:08:44.336 [ ALERT] CRASH: abort+0x148 (/lib/x86_64-linux-gnu/libc.so.6) 2016-01-13 15:08:44.360 [ INFO] htsp: 192.168.178.22 [ Kodi Media Center ]: Identified as user 'oe-buero' 2016-01-13 15:08:44.364 [ INFO] htsp: 192.168.178.22 [ oe-buero | Kodi Media Center ]: Privileges updated 2016-01-13 15:08:44.382 [ ALERT] CRASH: ??:0 0x6865b86 2016-01-13 15:08:44.423 [ ALERT] CRASH: ??:0 0x6865c32 2016-01-13 15:08:44.468 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/plumbing/globalheaders.c:284 0x4a1023 2016-01-13 15:08:44.521 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/plumbing/transcoding.c:858 0x519992 2016-01-13 15:08:44.590 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/plumbing/transcoding.c:1497 0x5189f8 2016-01-13 15:08:44.640 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/plumbing/tsfix.c:278 0x49fabf 2016-01-13 15:08:44.684 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/plumbing/tsfix.c:298 0x4a029a 2016-01-13 15:08:44.727 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/parsers/parsers.c:1826 0x491a98 2016-01-13 15:08:44.777 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/parsers/parsers.c:1402 0x4933e5 2016-01-13 15:08:44.825 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/parsers/parsers.c:437 0x492884 2016-01-13 15:08:44.879 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/input/mpegts/tsdemux.c:101 0x4d068e 2016-01-13 15:08:44.922 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/input/mpegts/tsdemux.c:268 0x4d0afe 2016-01-13 15:08:44.979 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/descrambler/tvhcsa.c:77 0x51a83d 2016-01-13 15:08:45.024 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/descrambler/tvhcsa.c:164 0x51a97d 2016-01-13 15:08:45.069 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/descrambler/descrambler.c:766 0x4c8af7 2016-01-13 15:08:45.120 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/input/mpegts/tsdemux.c:233 0x4d0a5d 2016-01-13 15:08:45.167 [ ALERT] CRASH: /home/waldmeister/src/tvheadend/src/input/mpegts/mpegts_input.c:1300 0x4ceac6
see attached valgrind log for more infos
Updated by James Huxtable almost 9 years ago
Jaroslav Kysela wrote:
I believe, it's fixed in current master.
Is this definitely fixed? I was testing a recording last night after I had enabled a new profile and timeshifting. Before finishing the recording the process was killed as all memory had been used (including swap).
It appears as if the buffer wasn't being written to the HDD and freed properly, meaning it kept growing until it was killed.
My build is:
Build: 4.1-1945~g5374573~vivid (2016-04-28T23:44:34+0000)
[204365.820566] tvheadend invoked oom-killer: gfp_mask=0x24201ca, order=0, oom_score_adj=0
[204365.820569] tvheadend cpuset=/ mems_allowed=0
[204365.820573] CPU: 1 PID: 7474 Comm: tvheadend Not tainted 4.4.0-21-generic #37-Ubuntu
[204365.820574] Hardware name: Xen HVM domU, BIOS 4.4.0 04/14/2014
[204365.820575] 0000000000000286 0000000099d2dd54 ffff8800764c3a58 ffffffff813e93c3
[204365.820577] ffff8800764c3c50 ffff8800776dee00 ffff8800764c3ac8 ffffffff81208b2e
[204365.820579] ffffffff81ccc43f 0000000000000000 ffffffff81e66660 0000000000000206
[204365.820581] Call Trace:
[204365.820587] [<ffffffff813e93c3>] dump_stack+0x63/0x90
[204365.820590] [<ffffffff81208b2e>] dump_header+0x5a/0x1c5
[204365.820592] [<ffffffff81190dd2>] oom_kill_process+0x202/0x3c0
[204365.820594] [<ffffffff811911f5>] out_of_memory+0x215/0x460
[204365.820597] [<ffffffff81197190>] __alloc_pages_nodemask+0x9d0/0xb60
[204365.820600] [<ffffffff811e084c>] alloc_pages_current+0x8c/0x110
[204365.820602] [<ffffffff8118d67b>] __page_cache_alloc+0xab/0xc0
[204365.820603] [<ffffffff8118f8aa>] filemap_fault+0x14a/0x3f0
[204365.820606] [<ffffffff811bc120>] __do_fault+0x50/0xe0
[204365.820609] [<ffffffff811bfb5b>] handle_mm_fault+0xf8b/0x1820
[204365.820611] [<ffffffff8125572b>] ? ep_poll+0x37b/0x3d0
[204365.820614] [<ffffffff8106b537>] __do_page_fault+0x197/0x400
[204365.820616] [<ffffffff8106b7c2>] do_page_fault+0x22/0x30
[204365.820618] [<ffffffff81826678>] page_fault+0x28/0x30
[204365.820620] Mem-Info:
[204365.820622] active_anon:351582 inactive_anon:118804 isolated_anon:0
active_file:48 inactive_file:52 isolated_file:0
unevictable:1149 dirty:0 writeback:0 unstable:0
slab_reclaimable:5357 slab_unreclaimable:8373
mapped:678 shmem:91 pagetables:2659 bounce:0
free:3341 free_pcp:60 free_cma:0
Updated by C K over 8 years ago
Sorry to reopen it again, getting this error too while recording.
I have ~5-10 recordings at one time and measured my disc write speed with
dd if=/dev/zero of=/home/waldmeister/test123 bs=1G count=1 oflag=direct 110 - 140 MB/s
How can i provide more tests?
Updated by C K over 8 years ago
So i really don't think thats HDD speed related. There must be another problem. Maybe timeshift?
Updated by Jaroslav Kysela over 8 years ago
Create a new issue. Provide screenshot of 'Config / Debugging / Memory information entries' when tvh eats too much memory.
Updated by Jaroslav Kysela over 8 years ago
And sync with latest master code again...