Tvheadend

I have no real idea, but I hate to see an unanswered question...

I think you need to (logically) separate the front end from the back.

XBMC depends on its own patched, internal ffmpeg, so shouldn't need either libav or ffmpeg on the system AFAIK. That said, I've seen issues related to an unofficial/unsupported version of XBMC being in some distros' repos, versions which use external ffmpeg and thus cause issues. You may be falling foul of that.

The backend... yes, you'd inevitably need something for transcoding, but I don't know how much dependency tvheadend actually has on the ffmpeg/libav library to do its core functions other than this. In essence, unless it's dynamically depending on those libraries for the .ts or .mkv code, it's simply opening a file and either dumping the contents in (stream) or remuxing the contents in (matroska).

I don't think there's a huge issue with either libav or ffmpeg - both inevitably feed off each other, plus they're constantly being tweaked, so I'd expect bugs to only be an issue if you're on an old version and can't update and/or if you're using bleeding-edge features (e.g. the H.265 debate that's raging over on the XBMC forum).

And, of course, you can always have both on your system if you know what you're doing... if you need to compile XYZ against ffmpeg but have libav installed, it's perfectly possible to install ffmpeg somewhere non-standard and then point your XYZ build towards this. If you build statically then you should no longer be dependent on the libav libraries at run time.

You might need to do some homework on makefiles and build options, see what tvheadend has been/could be built with, and take it from there.

TVHeadend generally use direct pid content transfer to client, without any transcoding,
exept case when transcoding is enabled.