Bug #1376
closed
Around 40% CPU at start on Pi for minutes (dunno yet if it stops)
Added by Eric Valette over 12 years ago. Updated over 12 years ago.
0%
Description
I have build tvheaedend git today on PI and I have problem playing TV on XBMC. It turns out that tvheadend consumes up to 45% cpu and 35%+ always at the beginning for long period of time probably parsing eit, or updating its epg database. This prevent streaming to local XBMC. I suspect this is due to eit parsing, and epg database updating. I add there is nothing connected to it (no XBMC, no web interface, nothing). Is there a way to prevent eit background parsing or epg updating? Staring in foreground and debug mode, we see the eit parsing on all channels, epg updating but it consumes memory for too long.
top - 15:41:24 up 32 min, 5 users, load average: 0.21, 0.29, 0.33
Tasks: 73 total, 1 running, 72 sleeping, 0 stopped, 0 zombie
%Cpu(s): 10.2 us, 23.4 sy, 0.0 ni, 66.0 id, 0.0 wa, 0.4 hi, 0.0 si, 0.0 st
KiB Mem: 124176 total, 98040 used, 26136 free, 11536 buffers
KiB Swap: 974844 total, 0 used, 974844 free, 39320 cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2184 hts 20 0 99396 3784 1820 S 34.5 3.0 4:05.79 tvheadend Tasks: 63 total, 1 running, 62 sleeping, 0 stopped, 0 zombieFiles
Updated by Adam Sutton over 12 years ago
- Status changed from New to Need feedback
- Assignee deleted (
Adam Sutton) - Target version deleted (
3.4)
Please provide proper version info for tracking purposes.
What EPG modules have you enabled? Or more importantly do you expect to receive from?
Can you disable all the OTA modules to check whether it is indeed EIT related, there will definitely be a reasonable load for certain providers (due to coding), but bog standard EIT should not be that high.
Also worth checking 3.2 to see how that performs, probably the same, but given recent development it would be good to rule this out.
Adam
Updated by Eric Valette over 12 years ago
Adam Sutton wrote:
Please provide proper version info for tracking purposes.
If I knew the meaning of 3.2, 3.3, 3.4, it would be simpler. What are you asking for? the git version?
What EPG modules have you enabled? Or more importantly do you expect to receive from?
I removed all OTA modules but still have two threads that both consumes around 30%
2196 hts 20 0 150m 6192 1184 S 13.9 5.0 6:40.34 tvheadend
2195 hts 20 0 150m 6192 1184 S 13.6 5.0 6:19.82 tvheadend
Can you disable all the OTA modules to check whether it is indeed EIT related, there will definitely be a reasonable load for certain providers (due to coding), but bog standard EIT should not be that high.
See above.
Updated by Eric Valette over 12 years ago
cat epggrab/config
{
"channel_rename": 0,
"channel_renumber": 0,
"channel_reicon": 0,
"interval": 43200
}
Updated by Eric Valette over 12 years ago
git version: d761985f3b6c0002e5d4f39c6ded3d00a0d57ed9
Updated by Adam Sutton over 12 years ago
- Found in version changed from git today to d761985f3b6c0002e5d4f39c6ded3d00a0d57ed9
Yes the git commit'ish is by far the best way of reporting issues against anything other than official Releases. Since git master etc... represents a moving target. I'll try and put together a wiki article to more clearly detail the new TVH release process.
Ok, sounds like your problem is not EIT related then. So lets work on that basis for now.
1. How many adapters/tuners do you have?
2. Can you try this with 3.2 to rule out possible problems with recent work.
Adam
P.S.
Ta for the commit, I'd already typed above before you updated.
Updated by Eric Valette over 12 years ago
Ok, sounds like your problem is not EIT related then. So lets work on that basis for now.
I'm not so sure as we see the message "begin processing periodically/end processing" with a 18s interval, 1min, 20s and a soon as it finishes another starts
BTW: when creating threads, a debug message with the tid and the main function whould help as I know the culprit with a top -H
1. How many adapters/tuners do you have?
Two.
2. Can you try this with 3.2 to rule out possible problems with recent work.
If I go backward, I use my own branch ;-)
Updated by Adam Sutton over 12 years ago
You said you had disabled all OTA modules (as your config suggests), in which case you should not see those messages. As all OTA EPG processing (EIT etc...) will be disabled. It might be worth attaching a debug log.
Two adapters indeed makes sense, the 2 threads are most probably the DVB/DVR input threads and this may well relate to the recent work. The EPG processing (even for multiple adapters) is performed from a single thread. (Actually that might not be true, since Andreas has been changing stuff there too).
If you cannot try another version to rule out changes, then its unlikely any progress will be made and most likely the issue will be closed in time.
Adam
Updated by Eric Valette over 12 years ago
Adam Sutton wrote:
You said you had disabled all OTA modules (as your config suggests), in which case you should not see those messages. As all OTA EPG processing (EIT etc...) will be disabled. It might be worth attaching a debug log.
Here it is:
If you cannot try another version to rule out changes, then its unlikely any progress will be made and most likely the issue will be closed in time.
Well it should be easy to reproduce and debug as well...
Updated by Eric Valette over 12 years ago
I realized the log contains two run: one with OTA eit DVB grabing enabled (first one) and one without (second one).
Updated by Adam Sutton over 12 years ago
It does look like the EIT module has been enabled. I will double check that there is not a mistake in the config processing which results in this module being forcefully enabled.
Sure I can reproduce and test it when I have time and someone donates the relevant hardware. Would you like a mailing address to send yours?
I wouldn't be at all surprised if some of the recent development work on full mux RX would cause problems on some systems (including the pi, due to USB performance). This is to be expected, eventually there will be an override to avoid this mode for problem devices, but that in itself does not constitute a bug.
Adam
Updated by Eric Valette over 12 years ago
Adam Sutton wrote:
It does look like the EIT module has been enabled. I will double check that there is not a mistake in the config processing which results in this module being forcefully enabled.
I wouldn't be at all surprised if some of the recent development work on full mux RX would cause problems on some systems (including the pi, due to USB performance). This is to be expected, eventually there will be an override to avoid this mode for problem devices, but that in itself does not constitute a bug.
That part is easy to check I can force to non full mux mode myself...
Updated by Eric Valette over 12 years ago
Clearly forcing to non full mux mode is a big gain. We have a transponder in france where there are 3 HD muxes leading to around 20 MB but in peak mode it could reach up to 45 Mb.
Updated by Scott Ware over 12 years ago
Probably a similar issue but since revision '9f0de04a' on 25/10/12 (dvb: Add support for grabbing entire mux directly via HTTP), CPU usage on my dual-core AMD server went from between 2-8% to an average of 30% when idle so I suspect your suspicions regarding full mux RX and USB TV tuners are correct. I have a TBS dual tuner PCI card and a PCTV USB tuner. I will try unplugging the USB tuner and see if the high CPU usage goes away.
Updated by Eric Valette over 12 years ago
Yes I'm back at 3% CPU but I stilldo not understand the eit messages from the log...
Updated by Scott Ware over 12 years ago
I have tried the latest commit (3cd63363) and set all of my adapters to 'Disable full mux rx' and restarted tvheadend but still have high CPU usage.
Updated by Adam Sutton over 12 years ago
Eric,
Nor do I, I couldn't repeat it on my laptop but I will try again on my main test system.
Scott,
Might be worth popping along to #hts and discussing there.
adam
Updated by Eric Valette over 12 years ago
Scott Ware wrote:
I have tried the latest commit (3cd63363) and set all of my adapters to 'Disable full mux rx' and restarted tvheadend but still have high CPU usage.
Thanks for the fix adam.
For the record, I did not use the commit myself, I hacked the file with an #if 0 around the code that test full mux support. Will not be able to access the pi before friday to report. Been really lucky with the full chain lately (bug in XBMC, in the rasberry pi firmware and in tvheadend to prevent me to play h264!!!!).
Updated by Adam Sutton over 12 years ago
- Status changed from Need feedback to Invalid
No and that commit does not work ;)
It appears that the decision to use full mux rx happens before my configuration is loaded, so I need to re-think the way this happens.
That being said, I'm closing this bug as its not actually a fault per se, high CPU load is an expected behaviour given the use of full mux rx.
Adam
Updated by Eric Valette over 12 years ago
Adam Sutton wrote:
That being said, I'm closing this bug as its not actually a fault per se, high CPU load is an expected behaviour given the use of full mux rx.
I think this will just prevent helping people debugging their HTPC setup. I have atom+ion setup where some deinterlace mode works only when nothings else eats CPU. This would break my config and probabbly many others one. If the bug is closed and said mistaken, people will not even read it.
If consuming 40% CPU idle is not a bug then there are other bugs that will be much more difficult to fix...
Updated by Adam Sutton over 12 years ago
The report relates to pi (though I know there are other comments) on which the poor performance is an understood issue. We all know that trying to use the USB heavily on the pi causes problems, but this does not represent a bug in TVH.
Personally I have run TVH on dual DVB-S (PCIe) using Andreas' mods without issue, and thats on an AMD E350, the increased CPU load was negligible.
Yes we have changed the default mode of operation and that might in itself be something that needs further discussion based on the input we get over the 3.4 development period. We might decide that the default mode of operation be filtered for USB tuners for example (where bandwidth can often be more directly correlated to CPU load).
But that doesn't make this a bug.
Adam
Updated by Eric Valette over 12 years ago
Adam Sutton wrote:
Personally I have run TVH on dual DVB-S (PCIe) using Andreas' mods without issue, and thats on an AMD E350, the increased CPU load was negligible.
Comparing PCIE and USB2 is a .... What is the percentage of people having PCIE card in an HTPC? Tried to put one in a zotac?
Yes we have changed the default mode of operation and that might in itself be something that needs further discussion based on the input we get over the 3.4 development period. We might decide that the default mode of operation be filtered for USB tuners for example (where bandwidth can often be more directly correlated to CPU load).
But that doesn't make this a bug.
For me it does and clsing such a report before the configuration option exist is not helping your users...
Updated by Adam Sutton over 12 years ago
In my experience the split is mostly towards PCI, I accept that for smaller boxes (like the zotac etc..) USB is the only option. But this also represents (in my experience) a smaller segment of the market. I'm not trying to say that we should disregard this (as I have already pointed out it will be under consideration) and I might be wrong about my numbers as I'm only going on those users I have directly communicated with.
With regards to a configuration option, there is already an option to disable the full mux rx. My first commit was indeed broken as I had misunderstood the loading sequence, however I fixed that this morning along with another full mux rx related problem.
This configuration option will probably change (before 3.4) into a 3-way option enable/disable/auto. The only thing then is to determine what is best selection for auto and at hte moment I'm thinking it might be that only enable for PCI, though it will depend on feedback from user experience with USB/FW/etc.. For now the default will stay as is because it will make the issue (if one exists) more prominent during the early stages of the development cycle.
Adam