Tvheadend

It's opentv grabber, isnt? --trace opentv

Here is the working log from commit hash 328647710c7a816352b55cc36ffdeeeab96fa5a7.
It populates around 10700 epg entries with 99% having extra text (so all as expected)
https://www.dropbox.com/s/7pe9uf7wizrsal4/epg_commit_hash_3286477.log?dl=0

Here is the log from latest.
It populates around 8000 epg entries with 99% of these having no extra text.
https://www.dropbox.com/s/mdake88hgwzmclq/epg_latest.log?dl=0

The only thing that changes is the tvh binary.

Thanks

Could you test this?

diff --git a/src/epggrab/module/opentv.c b/src/epggrab/module/opentv.c
index 936ebb468..d221ee4f5 100644
--- a/src/epggrab/module/opentv.c
+++ b/src/epggrab/module/opentv.c
@@ -916,7 +916,7 @@ static int _opentv_prov_load_one ( const char *id, htsmsg_t *m )
   if (htsmsg_get_u32(m, "tsid", &tsid)) return -1;
   if (htsmsg_get_u32(m, "sid", &sid)) return -1;
   if (htsmsg_get_u32(m, "bouquetid", &bouquetid)) return -1;
-  titles_time = htsmsg_get_u32_or_default(m, "titles_time", 30);
+  titles_time = htsmsg_get_u32_or_default(m, "titles_time", 60);
   summaries_time = htsmsg_get_u32_or_default(m, "summaries_time", 240);

Thanks - its a bit better but still not correct. It has filled in some missing extra text (especially most channels around today but not so much in the future).

Hope this helps identifying the issue, let me know if I can do anything more to help.

Retest with v4.3-1560-g559491630, please.

Thanks Jaroslav. All ~10000 epg updates are now there but still most of the extra text is missing.
I will take another clean trace log shortly.

Thanks again.

Here is the log from latest with fix. As I said it seems have all the epg entries now but still the extra text missing.
https://www.dropbox.com/s/38j42bgzu6ytcl1/epg_latest_with_fix.log?dl=0

v4.3-1565-gf62816bda - it's a big large untested rewrite, report any issues.

Sorry it doesn't work, still no extra text and trying to take a trace log seems to crash it.
I will keep trying and product a log.

It must be one of the trace debug statements, let me take a look.

I am unable to find it.
Here is the stacktrace.

Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: opentv-skyuk: tableid A3 extraid 000014da5c032080 completed
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: opentv-skyuk: incomplete 0 complete 0 total 6
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 254 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 257 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 259 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 46 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 48 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 154 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 254 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 257 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 259 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 46 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 48 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 154 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: opentv: find by eid 254 = (nil)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: CRASH: Signal: 6 in PRG: /usr/bin/tvheadend (4.3-1571~g6eb97e1c9) [440a93ba1ba54aed7084282a78998beea5dfe367] CWD: /
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: CRASH: Fault address 0x3e800007795 (N/A)
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: CRASH: Loaded libraries: linux-vdso.so.1 /usr/lib/x86_64-linux-gnu/libdvbcsa.so.1 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 /lib/x86_64-linux-gnu/libz.so.1 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0 /usr/lib/x86_64-linux-gnu/liburiparser.so.1 /usr/lib/x86_64-linux-gnu/libavahi-common.so.3 /usr/lib/x86_64-linux-gnu/libavahi-client.so.3 /usr/lib/x86_64-linux-gnu/libva.so.2 /lib/x86_64-linux-gnu/libdbus-1.so.3 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/librt.so.1 /lib/x86_64-linux-gnu/libmvec.so.1 /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/lib/x86_64-linux-gnu/libva-drm.so.2 /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libsystemd.so.0 /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libgcc_s.so.1 /usr/lib/x86_64-linux-gnu/libdrm.so.2 /lib/x86_64-linux-gnu/liblzma.so.5 /usr/lib/x86_64-linux-gnu/liblz4.so.1 /lib/x86_64-linux-gnu/libgcrypt.so.20 /lib/x86_64-linux-gnu/libgpg-err
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: CRASH: Register dump [23]: 000000000000000000007f23610663600000000000000008000000000000024600007f23610665d0000000000000100000000000000000000000000000000030000000000000000200007f236106636000007f23610666d000007f23610665d00000000000000000000000000000000000007f2366238e9700007f236106636000007f2366238e970000000000000246002b00000000003300000000000000000000000000000000fffffffe7ffbba130000000000000000
Nov 27 09:18:24 server tvheadend³⁰⁶¹³: CRASH: STACKTRACE

https://tvheadend.org/projects/tvheadend/wiki/Debugging#Incorrect-not-useable-crash-reports

thanks - sorry for the delay.

ov 28 18:12:31 server tvheadend²²⁸³²: opentv: find by eid 445 = (nil)
Nov 28 18:12:31 server tvheadend²²⁸³²: opentv: find by eid 656 = (nil)
Nov 28 18:12:31 server tvheadend²²⁸³²: opentv: opentv-skyuk: completed pid 64 table 000000A8 / 000000fc
Nov 28 18:12:31 server tvheadend²²⁸³²: opentv: opentv-skyuk: completed pid 64 table 000000A8 / 000000fc
Nov 28 18:12:31 server tvheadend²²⁸³²: opentv: opentv-skyuk: completed pid 53 table 000000A0 / 000000fc
Nov 28 18:12:31 server tvheadend²²⁸³²: opentv: opentv-skyuk: completed pid 53 table 000000A0 / 000000fc
Nov 28 18:12:31 server tvheadend²²⁸³²: opentv: opentv-skyuk: completed pid 64 table 000000A8 / 000000fc
Nov 28 18:12:31 server tvheadend²²⁸³²: opentv: find by eid 545 = 0x561402c071c0
Nov 28 18:12:31 server tvheadend²²⁸³²: opentv: summary 'New York cops Gene Hackman and Roy Scheider face up to drug lord Fernando Rey as they struggle to break his narcotics operation. Strong language/violence/flashing images. (1971)(100 mins) Also in HD'
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: Signal: 11 in PRG: /usr/bin/tvheadend (4.3-1593~gc075732a1) [f477d6222aae6d5f88394deef5ec9b8eb7438d90] CWD: /tmp
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: Fault address 0xa1 (Address not mapped)
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: Loaded libraries: linux-vdso.so.1 /usr/lib/x86_64-linux-gnu/libdvbcsa.so.1 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 /lib/x86_64-linux-gnu/libz.so.1 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0 /usr/lib/x86_64-linux-gnu/liburiparser.so.1 /usr/lib/x86_64-linux-gnu/libavahi-common.so.3 /usr/lib/x86_64-linux-gnu/libavahi-client.so.3 /usr/lib/x86_64-linux-gnu/libva.so.2 /lib/x86_64-linux-gnu/libdbus-1.so.3 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/librt.so.1 /lib/x86_64-linux-gnu/libmvec.so.1 /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/lib/x86_64-linux-gnu/libva-drm.so.2 /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libsystemd.so.0 /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libgcc_s.so.1 /usr/lib/x86_64-linux-gnu/libdrm.so.2 /lib/x86_64-linux-gnu/liblzma.so.5 /usr/lib/x86_64-linux-gnu/liblz4.so.1 /lib/x86_64-linux-gnu/libgcrypt.so.20 /lib/x86_64-linux-gnu/libgpg-err
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: Register dump [23]: 00000000000000c600000000000000c600007f27a01d077000000000000000000000000000000000000000000000002100000000000000c600000000000000c6000000000000011000007f279c000950fffffffffffffb88000000000000011000000000000000a1000000000000001000007f279c0008d000007f27f05c0b6000007f27f57ed2070000000000010297002b0000000000330000000000000004000000000000000efffffffe7ffbba1300000000000000a1
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: STACKTRACE
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: /tmp/tvheadend/src/trap.c:176 0x5614003bf4fd 0x5614001a0000
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: ??:0 0x7f27f68b5890 0x7f27f68a3000
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: __libc_malloc+0x197 (/lib/x86_64-linux-gnu/libc.so.6)
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: ??:0 0x7f27f585517b 0x7f27f5756000
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: regexec+0xba (/lib/x86_64-linux-gnu/libc.so.6)
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: /tmp/tvheadend/src/epggrab/module/eitpatternlist.c:139 0x56140046dfd1 0x5614001a0000
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: /tmp/tvheadend/src/epggrab/module/opentv.c:422 0x561400472541 0x5614001a0000
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: /tmp/tvheadend/src/epggrab/module/opentv.c:502 0x561400472fd2 0x5614001a0000
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: /tmp/tvheadend/src/epggrab/module/opentv.c:565 0x5614004734da 0x5614001a0000
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: /tmp/tvheadend/src/epggrab.c:179 0x561400399ef6 0x5614001a0000
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: /tmp/tvheadend/src/tvh_thread.c:93 0x561400380318 0x5614001a0000
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: ??:0 0x7f27f68aa6db 0x7f27f68a3000
Nov 28 18:12:31 server tvheadend²²⁸³²: CRASH: clone+0x3f (/lib/x86_64-linux-gnu/libc.so.6)
Nov 28 18:12:31 server kernel: [41890.331810] tvh:epgdata²²⁹⁰⁰: segfault at a1 ip 00007f27f57ed207 sp 00007f27f05c0b60 error 4 in libc-2.27.so[7f27f5756000+1e7000]

Does valgrind show more? https://tvheadend.org/projects/tvheadend/wiki/Debugging#Memory-leaks-or-corruption

I think the valgrind log will help.

trace log - https://www.dropbox.com/s/eoy5o4wy8cq6aiy/trace?dl=0
valgrind log - https://www.dropbox.com/s/l5nv2wwauz3904n/valgrind?dl=0

Retest with v4.3-1595-gfd2209001, please.

Seems to be working great now, thanks for this.