Bug #5255
API call "filemoved" crashes TVH
Status:
Fixed
Priority:
Normal
Assignee:
-
Category:
API
Target version:
-
Start date:
2018-10-13
Due date:
% Done:
0%
Estimated time:
Found in version:
4.3-1457~gc45bfdc
Affected Versions:
Description
I'm using a post process script after a recording has finished which converts the .ts file into a .mkv via ffmpeg. It creates the .mkv file in parallel and when the conversion has finished, it calls the TVH API to inform TVH about the filechange:
_TVH_IP="10.0.6.2"
_TVH_PORT="9981"
_API_ENDPOINT="/api/dvr/entry/filemoved"
_BIN_CURL="/usr/bin/curl"
....
success=0
repeat=0
while [ "$repeat" -lt 10 ]; do
if ${_BIN_CURL} -s "http://${_TVH_IP}:${_TVH_PORT}${_API_ENDPOINT}" \
--data-urlencode "src=${TSVIDEO}" \
--data-urlencode "dst=${MKVIDEO}" > /dev/null; then
echo -e "$(date): Notification successful!"
echo -e "$(date): Notification successful!" >> $logfile
success=1
break
else
echo -e "$(date): Notification attempt failed ..."
echo -e "$(date): Notification attempt failed ..." >> $logfile
sleep 30
fi
repeat=$((repeat+1))
done
....
Unfortunately in 80% of all cases TVH crashes right when the API is called the first time. When the TVH server has restarted, the notification usually works right after the server came back. But the next recording which calls the API leads again to the crash.
Logfile entries:
Oct 13 17:26:30 FlairTV tvheadend[31506]: CRASH: Signal: 11 in PRG: /usr/bin/tvheadend (4.3-1457~gc45bfdc) [d4aa275ef2596f88911b7d617b1e87c97ab8e5da] CWD: / Oct 13 17:26:30 FlairTV tvheadend[31506]: CRASH: Fault address (nil) (Address not mapped) Oct 13 17:26:30 FlairTV tvheadend[31506]: CRASH: Loaded libraries: /usr/lib/x86_64-linux-gnu/libdvbcsa.so.1 /lib/x86_64-linux-gnu/libssl.so.1.0.0 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 /lib/x86_64-linux-gnu/libz.so.1 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0 /usr/lib/x86_64-linux-gnu/liburiparser.so.1 /usr/lib/x86_64-linux-gnu/libavahi-common.so.3 /usr/lib/x86_64-linux-gnu/libavahi-client.so.3 /lib/x86_64-linux-gnu/libdbus-1.so.3 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/librt.so.1 /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libsystemd.so.0 /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libgcc_s.so.1 /lib/x86_64-linux-gnu/libselinux.so.1 /lib/x86_64-linux-gnu/liblzma.so.5 /lib/x86_64-linux-gnu/libgcrypt.so.20 /lib/x86_64-linux-gnu/libpcre.so.3 /lib/x86_64-linux-gnu/libgpg-error.so.0 /lib/x86_64-linux-gnu/libnss_compat.so.2 /lib/x86_64-linux-gnu/libnsl.so.1 /lib/x86_64-linux-gnu/libnss_nis.so.2 /lib/x86_64-linux-g Oct 13 17:26:30 FlairTV tvheadend[31506]: CRASH: Register dump [23]: 00007fbe5c00165000000000040000010000000000000001000000000000020600007fbe5c00249c0000000000000000000055cd8149f1390000000000000000000000000000000000007fbe5c00249c000055cd8149e200000055cd8149f1100000000000000000000000000000049c000000000000000000007fbe729f83a800007fbe8ce3358a000000000001028300000000000000330000000000000004000000000000000efffffffe7ffbba130000000000000000 Oct 13 17:26:30 FlairTV tvheadend[31506]: CRASH: STACKTRACE Oct 13 17:26:30 FlairTV tvheadend[31506]: CRASH: ??:0 0x55cd7d0cf6ed 0x55cd7cec0000 Oct 13 17:26:30 FlairTV tvheadend[31506]: CRASH: ??:0 0x7fbe8da02390 0x7fbe8d9f1000 Oct 13 17:26:30 FlairTV tvheadend[31506]: CRASH: ??:0 0x7fbe8ce3358a 0x7fbe8cd94000 Oct 13 17:26:30 FlairTV tvheadend[31506]: CRASH: ??:0 0x55cd7d119fa2 0x55cd7cec0000 Oct 13 17:26:30 FlairTV tvheadend[31506]: CRASH: ??:0 0x55cd7d0fe04f 0x55cd7cec0000 Oct 13 17:26:30 FlairTV tvheadend[31506]: CRASH: ??:0 0x55cd7d12e29f 0x55cd7cec0000 Oct 13 17:26:31 FlairTV tvheadend[31506]: CRASH: ??:0 0x55cd7d09e893 0x55cd7cec0000 Oct 13 17:26:31 FlairTV tvheadend[31506]: CRASH: ??:0 0x55cd7d09fd04 0x55cd7cec0000 Oct 13 17:26:31 FlairTV tvheadend[31506]: CRASH: ??:0 0x55cd7d09ee03 0x55cd7cec0000 Oct 13 17:26:31 FlairTV tvheadend[31506]: CRASH: ??:0 0x55cd7d0a009b 0x55cd7cec0000 Oct 13 17:26:31 FlairTV tvheadend[31506]: CRASH: ??:0 0x55cd7d0a0388 0x55cd7cec0000 Oct 13 17:26:31 FlairTV tvheadend[31506]: CRASH: ??:0 0x55cd7d096e2d 0x55cd7cec0000 Oct 13 17:26:31 FlairTV tvheadend[31506]: CRASH: ??:0 0x55cd7d091f75 0x55cd7cec0000 Oct 13 17:26:31 FlairTV tvheadend[31506]: CRASH: ??:0 0x7fbe8d9f86ba 0x7fbe8d9f1000 Oct 13 17:26:31 FlairTV tvheadend[31506]: CRASH: clone+0x6d (/lib/x86_64-linux-gnu/libc.so.6) Oct 13 17:26:31 FlairTV kernel: [6230486.457888] tvh:tcp-start[31822]: segfault at 0 ip 00007fbe8ce3358a sp 00007fbe729f83a8 error 4 in libc-2.23.so[7fbe8cd94000+1c0000] Oct 13 17:26:32 FlairTV systemd[1]: tvheadend.service: Main process exited, code=killed, status=11/SEGV Oct 13 17:26:32 FlairTV systemd[1]: tvheadend.service: Unit entered failed state. Oct 13 17:26:32 FlairTV systemd[1]: tvheadend.service: Failed with result 'signal'. Oct 13 17:27:01 FlairTV CRON[31852]: (hts) CMD (/home/hts/convert.sh >/dev/null 2>&1) Oct 13 17:27:26 FlairTV systemd[1]: tvheadend.service: Service hold-off time over, scheduling restart. Oct 13 17:27:26 FlairTV systemd[1]: Stopped Tvheadend - a TV streaming server and DVR. Oct 13 17:27:26 FlairTV systemd[1]: Starting Tvheadend - a TV streaming server and DVR... Oct 13 17:27:26 FlairTV systemd[1]: Started Tvheadend - a TV streaming server and DVR. Oct 13 17:27:26 FlairTV tvheadend[31866]: main: Log started Oct 13 17:27:26 FlairTV tvheadend[31866]: http: Starting HTTP server 0.0.0.0:9981 Oct 13 17:27:26 FlairTV tvheadend[31866]: htsp: Starting HTSP server 0.0.0.0:9982 Oct 13 17:27:26 FlairTV tvheadend[31866]: config: loaded Oct 13 17:27:26 FlairTV tvheadend[31866]: config: scanfile (re)initialization with path <none>
If there is any workaround or alternative method to inform TVH about the file change, I'd be happy to use this until the issue is fixed. This has a huge impact to my installation as we're mostly doing recordings and watch them later.
History
Updated by Marco Jakobs about 6 years ago
Updated by Here is the sample of an API call which I've decoded by Wireshark (that was a short test which also crashed TVH once):
POST /api/dvr/entry/filemoved HTTP/1.1 Host: 127.0.0.1:9981 Authorization: Basic xxxxxxxxxxxxxxxxxxxxxxxxxxxx== User-Agent: curl/7.47.0 Accept: */* Content-Length: 395 Content-Type: application/x-www-form-urlencoded src=%2FTVH%2FTVH%2FPVR%2FBj__rn-Freitags-Heimath__ppchen-Duell-WDR-Aachen_WDR-Bielefeld_WDR-Dortmund_WDR-Duisburg_WDR-D__sseldorf_WDR-Essen_WDR-K__ln_WDR-M__nster_WDR-Siegen_WDR-Wuppertal.ts&dst=%2FTVH%2FTVH%2FPVR%2FBj__rn-Freitags-Heimath__ppchen-Duell-WDR-Aachen_WDR-Bielefeld_WDR-Dortmund_WDR-Duisburg_WDR-D__sseldorf_WDR-Essen_WDR-K__ln_WDR-M__nster_WDR-Siegen_WDR-Wuppertal-201810131820.mkv
Updated by Mark Clarkstone about 6 years ago
Updated by Marco Jakobs wrote:
...
Please take a look at "Incorrect (not useable) crash reports" on this page debugging, and provide the correct lines. 
Updated by Marco Jakobs about 6 years ago
OK, I try to find time to dive into this tomorrow, make a Snapshot, find the debug version and reproduce the problem.
Updated by
Updated by Marco Jakobs about 6 years ago
I've fired the debug version, here's the crash with tvheadend-dbg_4.3-1466~g0b8d903~xenial_amd64.deb:
Oct 13 20:02:37 FlairTV tvheadend[1424]: spawn: Sat Oct 13 20:02:37 CEST 2018: *** Creating conversion task for /TVH/TVH/PVR/Tagesschau-Das-Erste.ts *** Oct 13 20:03:01 FlairTV CRON[4854]: (hts) CMD (/home/hts/convert.sh >/dev/null 2>&1) Oct 13 20:03:03 FlairTV tvheadend[1424]: CRASH: Signal: 11 in PRG: /usr/bin/tvheadend (4.3-1466~g0b8d903) [7891bcd886f926b67400822dced4335e255fa352] CWD: / Oct 13 20:03:03 FlairTV tvheadend[1424]: CRASH: Fault address (nil) (Address not mapped) Oct 13 20:03:03 FlairTV tvheadend[1424]: CRASH: Loaded libraries: /usr/lib/x86_64-linux-gnu/libdvbcsa.so.1 /lib/x86_64-linux-gnu/libssl.so.1.0.0 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 /lib/x86_64-linux-gnu/libz.so.1 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0 /usr/lib/x86_64-linux-gnu/liburiparser.so.1 /usr/lib/x86_64-linux-gnu/libavahi-common.so.3 /usr/lib/x86_64-linux-gnu/libavahi-client.so.3 /lib/x86_64-linux-gnu/libdbus-1.so.3 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/librt.so.1 /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libsystemd.so.0 /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libgcc_s.so.1 /lib/x86_64-linux-gnu/libselinux.so.1 /lib/x86_64-linux-gnu/liblzma.so.5 /lib/x86_64-linux-gnu/libgcrypt.so.20 /lib/x86_64-linux-gnu/libpcre.so.3 /lib/x86_64-linux-gnu/libgpg-error.so.0 /lib/x86_64-linux-gnu/libnss_compat.so.2 /lib/x86_64-linux-gnu/libnsl.so.1 /lib/x86_64-linux-gnu/libnss_nis.so.2 /lib/x86_64-linux-g Oct 13 20:03:03 FlairTV tvheadend[1424]: CRASH: Register dump [23]: 00007f5d40000e2000000000040000010000000000000001000000000000020600007f5d400026fc000000000000000000007f5d4c000d090000000000000000000000000000000000007f5d400026fc00007f5d900201b000007f5d4c000ce0000000000000000000000000000006fc000000000000006600007f5da65f136800007f5df09a658a000000000001028700000000000000330000000000000004000000000000000efffffffe7ffbba130000000000000000 Oct 13 20:03:03 FlairTV tvheadend[1424]: CRASH: STACKTRACE Oct 13 20:03:04 FlairTV tvheadend[1424]: CRASH: /project/repo/checkout/src/trap.c:176 0x55eac09d986d 0x55eac07ca000 Oct 13 20:03:04 FlairTV tvheadend[1424]: CRASH: ??:0 0x7f5df1575390 0x7f5df1564000 Oct 13 20:03:04 FlairTV tvheadend[1424]: CRASH: ??:0 0x7f5df09a658a 0x7f5df0907000 Oct 13 20:03:05 FlairTV tvheadend[1424]: CRASH: /project/repo/checkout/src/dvr/dvr_db.c:4801 0x55eac0a24192 0x55eac07ca000 Oct 13 20:03:05 FlairTV tvheadend[1424]: CRASH: /project/repo/checkout/src/api/api_dvr.c:554 0x55eac0a081cf 0x55eac07ca000 Oct 13 20:03:05 FlairTV tvheadend[1424]: CRASH: /project/repo/checkout/src/webui/webui_api.c:43 0x55eac0a3848f 0x55eac07ca000 Oct 13 20:03:05 FlairTV tvheadend[1424]: CRASH: /project/repo/checkout/src/http.c:1194 0x55eac09a8a13 0x55eac07ca000 Oct 13 20:03:06 FlairTV tvheadend[1424]: CRASH: /project/repo/checkout/src/http.c:1331 0x55eac09a9e84 0x55eac07ca000 Oct 13 20:03:06 FlairTV tvheadend[1424]: CRASH: /project/repo/checkout/src/http.c:1486 0x55eac09a8f83 0x55eac07ca000 Oct 13 20:03:06 FlairTV tvheadend[1424]: CRASH: /project/repo/checkout/src/http.c:1971 0x55eac09aa21b 0x55eac07ca000 Oct 13 20:03:06 FlairTV tvheadend[1424]: CRASH: /project/repo/checkout/src/http.c:2022 0x55eac09aa508 0x55eac07ca000 Oct 13 20:03:07 FlairTV tvheadend[1424]: CRASH: /project/repo/checkout/src/tcp.c:717 0x55eac09a0fad 0x55eac07ca000 Oct 13 20:03:07 FlairTV tvheadend[1424]: CRASH: /project/repo/checkout/src/wrappers.c:181 0x55eac099c0f5 0x55eac07ca000 Oct 13 20:03:07 FlairTV tvheadend[1424]: CRASH: ??:0 0x7f5df156b6ba 0x7f5df1564000 Oct 13 20:03:07 FlairTV kernel: [ 3003.275665] show_signal_msg: 12 callbacks suppressed Oct 13 20:03:07 FlairTV kernel: [ 3003.275672] tvh:tcp-start[4875]: segfault at 0 ip 00007f5df09a658a sp 00007f5da65f1368 error 4 in libc-2.23.so[7f5df0907000+1c0000] Oct 13 20:03:07 FlairTV systemd[1]: tvheadend.service: Main process exited, code=killed, status=11/SEGV Oct 13 20:03:07 FlairTV systemd[1]: tvheadend.service: Unit entered failed state. Oct 13 20:03:07 FlairTV systemd[1]: tvheadend.service: Failed with result 'signal'. Oct 13 20:03:34 FlairTV systemd[1]: Started Session 80 of user root. Oct 13 20:03:36 FlairTV systemd[1]: Started Session 81 of user root.
Updated by Marco Jakobs about 6 years ago
Joe User wrote:
I don't see any bad or missing data in my API call (see the Wireshark output above), also after a the restart the very same call works (until one of the following calls crashes it again) ... so it seems not to be related to the format.
Updated by Marco Jakobs about 6 years ago
Joe User wrote:
try with option "--data" instead of "data--urlencode"
OK, I've changed it and will watch and try ...
Updated by Joe User about 6 years ago
Not sure if it will help, but the "data" is parsed differently.
From src/http.c:
/* Parse content-type */
v = http_arg_get(&hc->hc_args, "Content-Type");
if(v != NULL) {
char *argv[2];
int n = http_tokenize(v, argv, 2, ';');
if(n == 0) {
http_error(hc, HTTP_STATUS_BAD_REQUEST);
return 0;
}
if(!strcmp(argv[0], "application/x-www-form-urlencoded"))
http_parse_args(&hc->hc_req_args, hc->hc_post_data);
}
Updated by Joe User about 6 years ago
Also, how are you posting the data after a crash - I guess from command line?
And is there something consistent in the filenames when it crashes - special characters?
Updated by
Updated by Marco Jakobs about 6 years ago
Jaroslav Kysela wrote:
Fixed in v4.3-1468-g3f74523d2 .
Awesome! :-))
Thanks a ton!