Heavy CPU load
Added by bas t about 12 years ago
Do you have any OTA EPG services enabled? Is the CPU load constantly higher or just for brief periods?
@addamsutton:
I am using the epg my cable provider sends along with the signal. That is not the origin of the problem.
The heavy cpu load is while recording a show and is constant.
I'll be glad to help debugging this, I'll need directions though.
Replies (66)
RE: Heavy CPU load - Added by bas t about 12 years ago
running in place, afbaf935d6 is still affected with the heavy cpu load.
I get the feel;ing that the new config script is somehow responsible for this.
Going to check.
RE: Heavy CPU load - Added by Alex . about 12 years ago
I can't seem to build 6d260fc8b7 or 841ff8eb49.
It ends in errors at build.linux/bundle.c.
make: *** [build.linux/bundle.o] Error 1
However, I can build afbaf935d6 and confirm that afbaf935d6 has the same problems with high load.
841ff8eb49 is the one with the new script.
RE: Heavy CPU load - Added by bas t about 12 years ago
commit 9e53683ca0 is not affected (just before the script)
RE: Heavy CPU load - Added by bas t about 12 years ago
a259025d31 is the first after the script that will build ok on my machine.
It suffers fropm the heavy cpuload.
So I think the new script and/or the makefile (which was tidied up) are responsible for the trouble
RE: Heavy CPU load - Added by Alex . about 12 years ago
bas t wrote:
commit 9e53683ca0 is not affected (just before the script)
Ok, and since it is hard to see how the zlib thing can make a difference (making it optional), it is probably in 841ff8eb49.
RE: Heavy CPU load - Added by Alex . about 12 years ago
bas t wrote:
a259025d31 is the first after the script that will build ok on my machine.
It suffers fropm the heavy cpuload.
So I think the new script and/or the makefile (which was tidied up) are responsible for the trouble
Yeah, indeed, the problem occurs after that.
I will look through the code, but i am affraid i might not be of any help there.....
RE: Heavy CPU load - Added by bas t about 12 years ago
Does ffdecsa use zlib?
If it does, a259025d31 or 6d260fc8b7 may be the ones we are after
RE: Heavy CPU load - Added by bas t about 12 years ago
Alex. _ wrote:
Ok, and since it is hard to see how the zlib thing can make a difference (making it optional), it is probably in 841ff8eb49.
Missed that, you're right
RE: Heavy CPU load - Added by Adam Sutton about 12 years ago
Zlib is only used for the UI and some config files (if you enable file bundling as well). It should have ZERO impact on the main code.
Also I think looking at the configure script is way off track same goes for Makefile. Unless there is a clear diff in the CC flags being passed that mean I've cocked up some optimisation etc...
But I'm open to being proven wrong!
Adam
RE: Heavy CPU load - Added by Alex . about 12 years ago
Adam Sutton wrote:
Also I think looking at the configure script is way off track same goes for Makefile. Unless there is a clear diff in the CC flags being passed that mean I've cocked up some optimisation etc...
Some slight differences, but nothing that is apparently strange to me:
New: .config.ml
# Automatically generated by configure - DO NOT EDIT!
ROOTDIR ?= .
BUILDDIR ?= ./build.linux
OSENV ?= posix
ARCH ?= x86_64
ifeq ($(origin CC),default)
CC = cc
endif
PYTHON ?= python
CFLAGS +=
LDFLAGS +=
prefix = /opt/tvheadend
bindir = ${prefix}/bin
mandir = ${datadir}/man
datadir = ${prefix}/share
libdir = ${prefix}/lib
LDFLAGS += -lssl -lcrypto
CFLAGS +=
LDFLAGS += -lz
CFLAGS +=
LDFLAGS += -lavahi-common -lavahi-client
CFLAGS += -D_REENTRANT
CONFIG_CWC = yes
CONFIG_V4L = yes
CONFIG_LINUXDVB = yes
CONFIG_DVBSCAN = yes
CONFIG_AVAHI = yes
CONFIG_ZLIB = yes
CONFIG_BUNDLE = no
CONFIG_EXECINFO = yes
CONFIG_MMX = yes
CONFIG_SSE2 = yes
CONFIG_PYGZIP = yes
CONFIG_SSL = yes
Old .config.mk:
# Automatically generated by configure - DO NOT EDIT!
ROOTDIR ?= .
BUILDDIR ?= ./build.linux
OSENV ?= posix
CFLAGS +=
LDFLAGS +=
LDFLAGS += -lssl -lcrypto
CFLAGS +=
LDFLAGS += -lz
CFLAGS +=
LDFLAGS += -lavahi-common -lavahi-client
CFLAGS += -D_REENTRANT
CONFIG_CWC = yes
CONFIG_V4L = yes
CONFIG_LINUXDVB = yes
CONFIG_DVBSCAN = yes
CONFIG_AVAHI = yes
CONFIG_ZLIB = yes
CONFIG_EXECINFO = yes
CONFIG_MMX = yes
CONFIG_SSE2 = yes
CONFIG_SSL = yes
RE: Heavy CPU load - Added by bas t about 12 years ago
@addamsutton:
Ruling out the zlib stuff, only 5 files have changed between 9e53683ca0 and a259025d31
That would be: Makefile, configure, support/configure.inc, src/filebundle.c, src/main.c
So where do you suggest to look?
RE: Heavy CPU load - Added by bas t about 12 years ago
Just a wild thought:
Could it be that this got messed up?
(from https://github.com/tvheadend/tvheadend/pull/54)
"tvheadend is 'cheating' a bit (in a very clever way) by selecting optimal settings at runtime (which the original FFdeCSA did at build time)"
RE: Heavy CPU load - Added by Alex . about 12 years ago
Hi,
I rechecked and I am now also not sure about 9e53683ca0 (Where "Updated so that we only need a single binary to cover 3 possible use cases" )
It does seem to give high cpu loads.
Also, I can select that branch, however, if I then do: git reset --hard HEAD
i get:
cd src-9e53683ca0/
src-9e53683ca0 # git reset --hard HEAD
HEAD is now at 00fa2f4 Add some additional support to configure.inc added for new feature check in configure that has now been abandoned.
Which seems a different thing. Also , if I install that one it give the high cpu load.
So, the last working one I now seem to get for sure is b350d44c4d .
( but please verify on your system, since I am confused about the numbering right now).
Alex.
RE: Heavy CPU load - Added by bas t about 12 years ago
I am sure that commit 9e53683ca0 works without the heavy cpu load on my machine
You could check again with a freshly cloned repo.
git reset --hard HEAD should give you the right commit as head (9e53683ca0 + some info)
RE: Heavy CPU load - Added by Alex . about 12 years ago
bas t wrote:
Just a wild thought:
Could it be that this got messed up?
(from https://github.com/tvheadend/tvheadend/pull/54)
"tvheadend is 'cheating' a bit (in a very clever way) by selecting optimal settings at runtime (which the original FFdeCSA did at build time)"
Mmm, I see what it says, but I am not sure if I can understand why this made the change at that specific point in the commits?
Are you suggesting that actually FFdecsa is no longer used after that specific point?
RE: Heavy CPU load - Added by bas t about 12 years ago
No, I am suggesting that the optimal settings are no longer made at runtime.
If not using optimal settings, it is very easy to understand that the cpu load goes way up...
RE: Heavy CPU load - Added by Alex . about 12 years ago
bas t wrote:
No, I am suggesting that the optimal settings are no longer made at runtime.
If not using optimal settings, it is very easy to understand that the cpu load goes way up...
Ok, i understand.
I tried to find something specific by looking in the code but can't seem to find any specific lines that adres the optimalisation.
Every mentioning of FFdeCSA or CWC seem very general, for example in the make-file.
RE: Heavy CPU load - Added by bas t about 12 years ago
I take the wild thought back
While starting tvheadend, the log shows: CSA: Using SSE2 128bit parallel descrambling
which is indeed the optimal setting for my system
EDIT:
I'm not thinking straight, I'm running the unaffected commit.
Tomorrow I'll see what happens when I run an affected commit
RE: Heavy CPU load - Added by Alex . about 12 years ago
One other thing surprises me :
The increase in CPU load is very high ( like 5 or even 6 times as high as it should be, from 10% or less to 60% or more on my system) , so, this really shouldn't escape one's attention.
However: no-one else is mentioning this, which makes me think we might just have some 'bad luck' combination of stuff running...
So (questions to everyone):
- Is anyone else experiencing this high cpu load? ( Or, might this be something very specific to our two systems that (by chance) causes this on our systems?)
- Does anyone run the newest version of tvheadend and experiences a low CPU load when watching encrypted channels? (useful for us to know, since then it might be very specific to our setup)
- What is your cpu load on a GIT tvheadend system from after mid August when watching encrypted channels?
Thanks,
Alex.
PS:
To give some details about my system:
Gentoo AMD 64
dvbhdhomerun tuners
OSCAM as a card server
RE: Heavy CPU load - Added by Alex . about 12 years ago
EDIT:
I'm not thinking straight, I'm running the unaffected commit.
Tomorrow I'll see what happens when I run an affected commit
Lol, yeah, might be after all this time trying ;-)
However, I will not be able to try much tomorrow ( mondays, mondays....)
RE: Heavy CPU load - Added by bas t about 12 years ago
bas t wrote:
While starting tvheadend, the log shows: CSA: Using SSE2 128bit parallel descrambling
which is indeed the optimal setting for my system
The log still shows "CSA: Using SSE2 128bit parallel descrambling" while running a259025d31
Since I'm not a skilled coder, I'm out of options right now.
RE: Heavy CPU load - Added by bas t about 12 years ago
I gave libdvbcsa a chance. Patched master for it (see https://github.com/bas-t/tvheadend/commits/dvbcsa)
compiled libdvbcsa and tvheadend and fired it up.
As expected, the result wasn't as good as compiling from 9e53683ca0, but still remarkable.
Using the unpatched master branch, cpuload was still over 3 times up! So using libdvbcsa gives an excelent performance,
compared with unpatched master.
I'm not suggesting to drop ffdecsa in favour of libdvbcsa, I was just asking myself: what would happen if I use lidvbcsa?
Now I know.
RE: Heavy CPU load - Added by Adam Sutton about 12 years ago
Yeah the PR has mostly been ignored so far as we've had no input since John and I joined the project.
Plus its not something either of us are really very familiar with.
I'm still surprised there should be a significant change in basic performance for encrypted channels since 2.12/3.0 as nothing has been changed (that I know of) that would affect it.
Adam
RE: Heavy CPU load - Added by bas t about 12 years ago
Adam Sutton wrote:
nothing has been changed (that I know of) that would affect it.
Well, that's a big bummer. The build-in ffdecsa (plus not using mysql) is the major advantage of using tvheadend over mythtv + sasc-ng.
Sadly I only can tell in whitch commits it happened. Not which part of the commits.
As it is, tvheadend's master branch is pretty much useless to me, since allmost all channels are encrypted where I live.
All future releases would be useless too, untill someone drops in and fixes it.