Tvheadend

At an 1 month old (oldest I have) log I saw a value of 20sec "break" - that was even slow in comparison to the expected (other RPi Tvh users) values of about 1-2sec.

Hmm. It's on my laptop:

2015-10-06 13:05:41.950 [   INFO] scanfile: ATSC - loaded 2 regions with 14 networks
2015-10-06 13:05:41.950 [   INFO] mpegts: Loading input/dvb/networks
2015-10-06 13:05:43.785 [   INFO] mpegts: ... done
2015-10-06 13:05:43.871 [   INFO] linuxdvb: adapter added /dev/dvb/adapter1

The <cfg>/input/dvb/networks files (all) are parsed and networks, muxes and services are created in that time.

iff --git a/src/input/mpegts/mpegts_network_dvb.c b/src/input/mpegts/mpegts_network_dvb.c
index 47007c3..f65ba8b 100644
--- a/src/input/mpegts/mpegts_network_dvb.c
+++ b/src/input/mpegts/mpegts_network_dvb.c
@@ -658,6 +658,7 @@ void dvb_network_init ( void )
                                     dvb_network_builder);

   /* Load settings */
+  tvhinfo("mpegts", "Loading input/dvb/networks");
   if (!(c = hts_settings_load_r(1, "input/dvb/networks")))
     return;

@@ -673,6 +674,7 @@ void dvb_network_init ( void )
     }
   }
   htsmsg_destroy(c);
+  tvhinfo("mpegts", "... done");
 }

 void dvb_network_done ( void )

13:54:53.513 [ INFO] main: Log started
13:54:53.576 [ INFO] config: loaded
13:54:58.214 [ INFO] scanfile: DVB-T - loaded 43 regions with 729 networks
13:54:58.214 [ INFO] scanfile: DVB-S - loaded 1 regions with 92 networks
13:54:58.214 [ INFO] scanfile: DVB-C - loaded 17 regions with 52 networks
13:54:58.214 [ INFO] scanfile: ATSC - loaded 2 regions with 14 networks
13:54:58.214 [ INFO] mpegts: Loading input/dvb/networks
13:56:05.405 [ INFO] mpegts: ... done

Cached ? (You need to clear page cache to simulate the boot process: "echo 1 > /proc/sys/vm/drop_caches")

a reboot should do it ?

Yeah. I overlooked difference between 54/56 (minutes). I though that the timediff is only 7 seconds. You may compare, if the start time is lower when the files are cached (start tvh multiple times after boot). Just for a curiosity to check the i/o delays at boot.

I had tested this at the mean time,

looks like it works "quick - 20sec" if it is cached (new profile is a tvh profile from an x64 htpc)
my old very slow (1-2min) profile has ~14000 files at tvh/input/dvb/networks/id the other one from my x64 (20-30sec) has only ~6000 but both profiles have same network, same channels and same config O_o

Maybe it is purely overwhelmed by the number of files that the i/o is breaking down (some cache is full?)?
I don't know if Tvh has to open every file at input/dvb/... or if it is just on demand? The pure read speed should not be a problem, but to look into 14000 files is a problem ;)
If it has to open every file (design fault?) it would explain it. Is there no sort of cache which prevents to load the settings at every start up ? For example throw the settings at runtime into a settings.xmlz (or at shut-down) and let the ~1mb xmlz been read instead of the ~k files at startup ?

cached

new profile
20:06:14.049 [ INFO] main: Log started
20:06:14.099 [ INFO] config: loaded
20:06:15.040 [ INFO] scanfile: DVB-T - loaded 43 regions with 729 networks
20:06:15.040 [ INFO] scanfile: DVB-S - loaded 1 regions with 92 networks
20:06:15.040 [ INFO] scanfile: DVB-C - loaded 17 regions with 52 networks
20:06:15.040 [ INFO] scanfile: ATSC - loaded 2 regions with 14 networks
20:06:28.769 [ INFO] dvr: Creating new configuration ''

20:09:32.826 [ INFO] main: Log started
20:09:32.878 [ INFO] config: loaded
20:09:33.785 [ INFO] scanfile: DVB-T - loaded 43 regions with 729 networks
20:09:33.785 [ INFO] scanfile: DVB-S - loaded 1 regions with 92 networks
20:09:33.785 [ INFO] scanfile: DVB-C - loaded 17 regions with 52 networks
20:09:33.785 [ INFO] scanfile: ATSC - loaded 2 regions with 14 networks
20:09:47.196 [ INFO] dvr: Creating new configuration ''

20:15:55.918 [ INFO] main: Log started
20:15:55.965 [ INFO] config: loaded
20:15:56.878 [ INFO] scanfile: DVB-T - loaded 43 regions with 729 networks
20:15:56.878 [ INFO] scanfile: DVB-S - loaded 1 regions with 92 networks
20:15:56.878 [ INFO] scanfile: DVB-C - loaded 17 regions with 52 networks
20:15:56.878 [ INFO] scanfile: ATSC - loaded 2 regions with 14 networks
20:16:10.481 [ INFO] dvr: Creating new configuration ''

old profile
20:19:20.629 [ INFO] main: Log started
20:19:20.680 [ INFO] config: loaded
20:19:22.269 [ INFO] scanfile: DVB-T - loaded 43 regions with 729 networks
20:19:22.270 [ INFO] scanfile: DVB-S - loaded 1 regions with 92 networks
20:19:22.270 [ INFO] scanfile: DVB-C - loaded 17 regions with 52 networks
20:19:22.270 [ INFO] scanfile: ATSC - loaded 2 regions with 14 networks
20:19:39.562 [ INFO] dvr: Creating new configuration ''

20:33:57.306 [ INFO] main: Log started
20:33:57.355 [ INFO] config: loaded
20:33:58.276 [ INFO] scanfile: DVB-T - loaded 43 regions with 729 networks
20:33:58.276 [ INFO] scanfile: DVB-S - loaded 1 regions with 92 networks
20:33:58.276 [ INFO] scanfile: DVB-C - loaded 17 regions with 52 networks
20:33:58.276 [ INFO] scanfile: ATSC - loaded 2 regions with 14 networks
20:34:12.061 [ INFO] dvr: Creating new configuration ''

21:04:53.319 [ INFO] main: Log started
21:04:53.366 [ INFO] config: loaded
21:04:54.283 [ INFO] scanfile: DVB-T - loaded 43 regions with 729 networks
21:04:54.284 [ INFO] scanfile: DVB-S - loaded 1 regions with 92 networks
21:04:54.284 [ INFO] scanfile: DVB-C - loaded 17 regions with 52 networks
21:04:54.284 [ INFO] scanfile: ATSC - loaded 2 regions with 14 networks
21:05:08.165 [ INFO] dvr: Creating new configuration ''

Maybe some kind of xml file? I get a ~4mb file which could be gziped (fastest) to 300kb. Compression + file size shouldn't be a problem for every platform.

In the mean time tx for your quick help (as always) :)

C vH wrote:

Maybe some kind of xml file? I get a ~4mb file which could be gziped (fastest) to 300kb. Compression + file size shouldn't be a problem for every platform.

In the mean time tx for your quick help (as always) :)

What about using a database like mysql or postgresql for all the storage? this could allow - in a case of a re-installation - to keep your channel nummbers :-)

sqlite/libsqlite would do it also (no need for heavy mysql server), would fix the i/o problem also including simple import/backup possibility also sounds like easy implementation

downside, it would break the config again :/ -> easy downgrade isn't possible

There are two changes in latest master to reduce disk I/O in the start phase:

1) EPG data can be gziped now (it's on by default - global config)
2) mux config / service config data are merged together and gziped, too (forced - updated with the new data save)

It should reduce significantly the loading times when the disk I/O is an issue.

Note: You should rescan all networks/muxes to trigger the new save functions. Otherwise, the old configs are not touched.

Just for the record: Savings at my installation was approx. 10 MBytes.

Feeds: Astra 19.2° E plus a local DVB-C cable provider.

I think this can be closed after the gzip config change???

Joe User wrote:

I think this can be closed after the gzip config change???

And cookie files already (auto)cleared after 7 days. I think that is not longer a problem.
(c73e4248a9072be57d5ede3a510773ba06bebc09)