Project

General

Profile

Bug #2156

TVHeadend crashes since some builds back

Added by Crazy Fin over 10 years ago. Updated over 10 years ago.

Status:
Fixed
Priority:
Normal
Assignee:
-
Category:
Crashes
Target version:
-
Start date:
2014-06-29
Due date:
% Done:

0%

Estimated time:
Found in version:
3.9.967~gd924c84~precise
Affected Versions:

Description

I tested the latest build 3.9.967~gd924c84~precise and I notice that TVHeadend crashes every 30-60 minute. I can not see any specifically similar in the debug log that gives me a clue of what is happening.

I am not sure how many builds back this started but it was working fine with the builds back from last weeks of May.

Here is a piece of the syslog with debug -all enabld (I had no trace enabled though):

==========================

Jun 29 20:30:56 tvserver tvheadend7315: cwc: Sending ECM (PID 7093) section=0/0, for service "SVT1 HD" (seqno: 1277)
Jun 29 20:30:56 tvserver tvheadend7315: cwc: Received ECM reply (PID 7093) for service "SVT1 HD" even: af.19.90.58.f6.8c.a4.26 odd: 8f.a2.a8.d9.3e.b8.62.58 (seqno: 1277 Req delay: 332 ms)

Jun 29 20:30:58 tvserver tvheadend7315: CRASH: Signal: 6 in PRG: tvheadend (3.9.967~gd924c84~precise) [19dc61acdf5674894de2298eec397423f45001c5] CWD: /
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: Fault address 0x7300001c93 (N/A)
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: Loaded libraries: /lib/x86_64-linux-gnu/libssl.so.1.0.0 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 /lib/x86_64-linux-gnu/libz.so.1 /usr/lib/liburiparser.so.1 /usr/lib/x86_64-linux-gnu/libavahi-common.so.3 /usr/lib/x86_64-linux-gnu/libavahi-client.so.3 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/librt.so.1 /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libdbus-1.so.3 /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libnss_compat.so.2 /lib/x86_64-linux-gnu/libnsl.so.1 /lib/x86_64-linux-gnu/libnss_nis.so.2 /lib/x86_64-linux-gnu/libnss_files.so.2
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: Register dump [23]: fefefefefefefeffffffffffff0000000000000000000008000000000000020600000000004b2fe500000000004b31e0000000000000000f00000000000000120000000000001c930000000000001ca700000000004b30a000007f918d84800000000000000000060000000000000000ffffffffffffffff00007f917bffe8e800007f918bafb4250000000000000206000000000000003300000000000000000000000000000000fffffffe7ffbba170000000000000000
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: STACKTRACE
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: ??:0 0x42f37c
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: ??:0 0x7f918c398cb0
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: gsignal+0x35 (/lib/x86_64-linux-gnu/libc.so.6)
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: abort+0x17b (/lib/x86_64-linux-gnu/libc.so.6)
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: ??:0 0x7f918baf40ee
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: ??:0 0x7f918baf4192
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: ??:0 0x4630ba
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: ??:0 0x463134
Jun 29 20:30:58 tvserver tvheadend7315: CRASH: ??:0 0x40dfe1
Jun 29 20:30:59 tvserver tvheadend30100: charset: 138 entries loaded
Jun 29 20:30:59 tvserver tvheadend30100: linuxdvb: adapter added /dev/dvb/adapter7
Jun 29 20:30:59 tvserver tvheadend30100: linuxdvb: adapter added /dev/dvb/adapter6
Jun 29 20:30:59 tvserver tvheadend30100: linuxdvb: adapter added /dev/dvb/adapter5
Jun 29 20:30:59 tvserver tvheadend30100: linuxdvb: adapter added /dev/dvb/adapter4
Jun 29 20:30:59 tvserver tvheadend30100: linuxdvb: adapter added /dev/dvb/adapter3
Jun 29 20:30:59 tvserver tvheadend30100: linuxdvb: adapter added /dev/dvb/adapter2
Jun 29 20:30:59 tvserver tvheadend30100: linuxdvb: adapter added /dev/dvb/adapter1
Jun 29 20:30:59 tvserver tvheadend30100: linuxdvb: adapter added /dev/dvb/adapter0

==========================

Is there anything more I can debug or trace to help you investigate his crashing?


Files

gdb.txt (1.85 KB) gdb.txt Crazy Fin, 2014-07-11 17:21
tvh.log (415 KB) tvh.log Crazy Fin, 2014-07-11 17:23
tvh.log (1.7 MB) tvh.log Crazy Fin, 2014-07-11 22:35
gdb.txt (1.36 KB) gdb.txt Crazy Fin, 2014-07-11 22:35

History

#1

Updated by Crazy Fin over 10 years ago

I tested 3.9.818~gfda1a03 and that one has not yet crashed after 3 hours of running (2 tuners watching, 3rd tuner recording).

#2

Updated by Crazy Fin over 10 years ago

3.9.900~g5394ce2 crashes

#3

Updated by Crazy Fin over 10 years ago

I see this line in the log just before it crashes:

tvheadend: src/input/mpegts/mpegts_input.c:540: mpegts_input_table_dispatch: Assertion `i == len' failed.

3.9.871~ga70a3f5 crashes as well

#4

Updated by Jozef M over 10 years ago

Don't know if related to my issue, but I can ACK this.
Latest stable version I have is from Jun 22nd - 3.9.692~gf36be10-dirty. Feed to TVH is in my case IPTV via MC. Any version up to couple of commits back is segfaulting, at least once a day. I will be able to provide a debug/gdb outputs hopefully this weekend.

#5

Updated by Crazy Fin over 10 years ago

I am back down to release 3.9.821~g8af2590 which now has been up and running for approx 24 hours without a glitch.

I was having crashes from 3.9.848~g2be0f84 and up to 3.9.967~gd924c84.

I will try and test other release between 3.9.848~g2be0f84 and the current one 3.9.821~g8af2590 that I now have installed and is working fine.

Maybe I will find out between which releases the break causing the crashes is.

#6

Updated by Crazy Fin over 10 years ago

Ok I tested all versions from
3.9.848~g2be0f84
down to
3.9.821~g8af2590 (working fine)
and ALL releaseas between those two does not compile without errors. They all generate the following error when compiling "descrambler.o":

==========================
CC src/descrambler/descrambler.o
src/descrambler/descrambler.c: In function ‘descrambler_descramble’:
src/descrambler/descrambler.c:167:31: error: ‘failed’ may be used uninitialized in this function [-Werror=uninitialized]
src/descrambler/descrambler.c:167:31: error: ‘count’ may be used uninitialized in this function [-Werror=uninitialized]
cc1: all warnings being treated as errors
make: *** [/home/tvserver/tvheadend/build.linux/src/descrambler/descrambler.o] Error 1 ==========================

I know tried to compile the latest commit (Build #5474 - v3.9.976 - precise-amd64) but when I do

"git checkout bb181b06c336ae074fe219a982a2d1da8c07039f"

I get error

"fatal: reference is not a tree: bb181b06c336ae074fe219a982a2d1da8c07039f"

So I will stay on 3.9.821~g8af2590 until the crash bug has been fixed.
Let me know if there is any deugging I can do to help solving this issue.

#7

Updated by Crazy Fin over 10 years ago

I tested now with (in the TVHeadend directory)

"git checkout master"

and then

"git pull"

I was now able to do "./configure" and then "make" and the compilation succeeded without any errors and when I started with

"./build.linux/tvheadend -l tvh.log --debug all"

it started fine and I could see that I now was on the latest build "Build #5478 - v3.9.980 - precise-amd64". Cool!

However, crashes still the same way as all versions from 3.9.822 and up so I am still staying on "3.9.821~g8af2590".

#8

Updated by Crazy Fin over 10 years ago

I tested the latest build "3.9.1050~g388b4d7" yesterday and it still crashes after approx 1 hour of running and the log looks identical as in previous crashes:

"tvheadend: src/input/mpegts/mpegts_input.c:540: mpegts_input_table_dispatch: Assertion `i == len' failed."

in the log right before the crash ("segmentation fault")

Going back to "3.9.821~g8af2590" again and it works fine without crashes. However, this version has some "counter continuity errors" that I do not see in the newer versions that crashes after approx 1 hour.

#9

Updated by Crazy Fin over 10 years ago

Ok, version 3.9.1064~gd81a612 has not crashed (yet) for 5 hours so this release is promising!

"Continuity counter errors" that I see in the version that was my latest working release (3.9.821~g8af2590) are also gone.

I´ll keep on running this version for about 24 hours more before finally confirming that it is "stable" enough compared to the releases 2 months back and up until now.

#10

Updated by Crazy Fin over 10 years ago

Nooooo... crashed again but it 9 hours.

Same message in the debug log just before it crashes:

"tvheadend: src/input/mpegts/mpegts_input.c:540: mpegts_input_table_dispatch: Assertion `i == len' failed."

#11

Updated by Mario D over 10 years ago

I think a 'bt full' from the core dump image could be useful to the developers: [[https://tvheadend.org/projects/tvheadend/wiki/Debugging]]

#12

Updated by Crazy Fin over 10 years ago

I did the following (as per https://tvheadend.org/projects/tvheadend/wiki/Debugging):

1. Enabled coredumps with
ulimit -c unlimited

2. Since I am using Ubuntu 12.0.4 LTS I did
core_uses_pid=1 (as suggested in the debugging info page)

3. Once I had the core dump file I did
gdb tvheadend core
(gdb) set logging on
(gdb) set pagination off
(gdb) bt full

but the only output I get into gdb.txt is

[New LWP 3833]
[New LWP 3840]
[New LWP 3837]
[New LWP 3836]
[New LWP 3841]
[New LWP 3844]
[New LWP 3845]
[New LWP 3851]
[New LWP 3848]
[New LWP 3861]
[New LWP 3849]
[New LWP 3852]
[New LWP 3862]
[New LWP 3855]
[New LWP 3866]
[New LWP 3856]
[New LWP 3817]
[New LWP 3858]
[New LWP 3881]
[New LWP 3859]
[New LWP 3891]
[New LWP 3860]
[New LWP 3864]
[New LWP 3877]
[New LWP 3878]
[New LWP 3880]
[New LWP 3887]
[New LWP 3888]
[New LWP 6837]
[New LWP 6840]
[New LWP 3818]
[New LWP 3819]
[New LWP 3820]
[New LWP 3821]
[New LWP 3822]
[New LWP 3823]
[New LWP 3824]
[New LWP 3825]
[New LWP 3826]
[New LWP 3828]
[New LWP 3885]
[New LWP 3832]
[New LWP 3829]
[New LWP 3876]

warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff05fe000
Core was generated by `./build.linux/tvheadend -c test-conf -l tvh.log --debug all'.
Program terminated with signal 6, Aborted.
#0 0x00007f03d7d3d425 in ?? ()

and the output from

set logging on
(gdb) set pagination off
(gdb) bt full

shows

#0 0x00007f03d7d3d425 in ?? ()
No symbol table info available.
#1 0x00007f03d7d40b8b in ?? ()
No symbol table info available.
#2 0x00000000004ba2a5 in ?? ()
No symbol table info available

and so on up to

#23 0x0000000000000000 in ?? ()
No symbol table info available.

I guess I am doing something wrong with these core dumps?

#13

Updated by Crazy Fin over 10 years ago

Ok I think I have now been able to get a proper core dump file.

When running

gdb tvheadend core

I had instead of course to do

./build.linux/tvheadend core

to get it work properly (since I was in tvheadend directory)

Attached is the tvh.log (debug log) and the gdb.txt file from the core dump.

Let me know if there is some other files that I should attach.

#14

Updated by Crazy Fin over 10 years ago

Here is a new tvh.log and gdb.txt from a crash using the latest commit 3.9.1073

#15

Updated by Crazy Fin over 10 years ago

Ok, I thiiiiink that we can close this bug ticket now. :-)

I am currently testing 3.9.1083 and it does not crash anymore within an hour or so.

I have been able to run TVHeadend for 12-24 hours until it crashes and the crash is now quite different than this one.

I will report the new type of crash in a separate ticket with corresponing debug log and core dump.

#16

Updated by Jaroslav Kysela over 10 years ago

  • Status changed from New to Fixed

Also available in: Atom PDF