Tvheadend

Evan JS wrote:

Thank you both for the replies.

For my install tvheadend and xbmc are on the same box so getting tvheadend to record to a ts when live tv is initiated and tell xbmc which file to then play would work. I can see the implementation is more complex when tvheadend is remote as it has to stream the file content.

Eric, any idea how I could achieve that for a a co-located install?

There is an instant record button at the bottom-right of the screen. Then depending on the recording naming option you configured you will get the file name inyour recording location.

-- eric

Adam Sutton wrote:

Eric,

Actually because of the reported issue #1267, this will only work if you use direct file access (or NFS/SMB) and not via the HTTP links used by TVH.

Granted. You have another bug lying in the fact that each time you reinstall, you loose your record track because the software used to only know what he recorded himself instead of looking at the directory content...

This has nothing to do with TS v MKV, its a problem with streaming via HTTP (which is why recordings will eventually also be available via HTSP). I'm sure there is a solution, I just need to read the HTTP spec to see what can be achieved.

As you explained yourself, many user indeed use a NAS that provide file access via SAMBA so this is a perfectly valid solution until its correctly implemented. If you read the second post of the submitter you will also see that he has of course file access as FE/BE are on the same machine. This is what many user told me as the two main reasons to use my old TS branch (the other being able to stream TS to of the shelves devices).

Nor do any of the discussed approaches preclude the use of MKV as an end recording format, ultimately its just the last link in the chain. Though I admit there could be some inefficiencies related to doing this, but for most people they'd never notice and they can always change the preferred format.

"end recording format" YES "suitable for time shifting" NO. Then imagine you include time shifting and the end user play records you will not have two formats... You can reinvent the well and select another format if you want...

1. That's off topic and its not a bug. There is a reason why we don't simply look at the files, for one its only possible to do this properly if we use MKV. Which is something Andreas was considering doing and I argued against. I have never lost my recordings through an upgrade. There may be an issue if you uninstall and then reinstall from scratch, but that's also not a bug as its not the intended way to use the system. But I've not looked into this.

2. You've just regurgitated what I had already said. I don't (and never have) dispute that use of TS via direct file access will not provide a better means of scanning in progress recordings. I was just pointing out the limitations of this in the way it integrates etc.. It's a genuine failing of TVH, possibly brought about by poor support in HTTP, I'm still investigating. However ultimately the solution (for XBMC) will be recordings via HTSP which will allow much better 2 way feedback.

3. Again, you've just regurgitated my own statements. I have already said MKV is NOT suitable for timeshift and NO ONE on the project has ever said it was. Just because a users presses play on a recording (that's in progress) doesn't mean they will actually be playing the final recorded file. This is one reason why recording playback will ultimately be integrated into HTSP so that we can have seamless integration of live tv, recordings and time shift.

For in progress recordings there may be certain storage inefficiencies at the expense of providing better storage efficiency for other use cases like time shift. But that may or may not be the case and it may be optional, where the trade off would be storage efficiency (space) v time to availability of final file (i.e do we have to wait for remux to new container).

No one is suggesting we re-invent any wheels. We currently have 2 viable formats for which all the code exists, TS and HTSP. Both have pro's and con's and as I've already said the ultimate solution will depend on both technical merit and also developer availability.

At the moment we're still focused on other tasks so time shift has only had a limited amount of discussion but we definitely have some good ideas about how it should be done, though there are differing opinions within the team. Nothing much will happen on this though in the next few weeks as I'm taking a break and focussing on stabilising 3.2.

Regards
Adam

Adam Sutton wrote:

1. That's off topic and its not a bug. There is a reason why we don't simply look at the files, for one its only possible to do this properly if we use MKV.

???? Do not understand! Could you elaborate...

2. You've just regurgitated what I had already said.

One.

However ultimately the solution (for XBMC) will be recordings via HTSP which will allow much better 2 way feedback.

I disagree. Why would you record on the frontend??? This ties you to XBMC???

3. Again, you've just regurgitated my own statements.

Two.

Please stay polite even if you start realizing you just touched the crust of the technical problems you will have to face. You started by saying MKV or TS for recording doe not matter and you seem now to realize that for adding traditional feature like time-shifting it will.

No one is suggesting we re-invent any wheels. We currently have 2 viable formats for which all the code exists, TS and HTSP.

Mixing a protocol and a pseudo container here. You might deliver TS via several protocols including multicast, RTP, http, HLS, DASH...

1. Without the metadata in the files (i.e. MKV) it's not a proper system. You can encode in filenames but its just not the way to do it. But as I said I've never lost anything during an upgrade, so if you think there is a problem report it and I'll look.

2. Two.

3. I would NEVER and have NEVER suggested recording in the frontend. Please explain how on earth you came to that conclusion.

4. Four.

I am trying very hard to be polite, but you repeated what I'd previously said so I pointed that out. I really struggle to follow your arguments sometimes as you regularly go off on tangents arguing about irrelevant things, but I'm doing my best to keep things balanced. Your making false statements, based on your pre-conceptions of my opinions and a possibly your very narrow view of things.

MKV v TS does NOT matter for recording, timeshift and recording are NOT the same thing. There can and will be overlap, this does NOT necessarily make them intimately tied. In a very simplistic implementation I don't doubt (nor have I ever) that recording TS straight to disk would be the easiest solution, however this has limitations in the more general case that TVH has to deal with (i.e. where multiple clients can have multiple views on the the streams). Many of these topics I have explored and while I'm not an expert even I can see that simply dumping TS to disk (in a linear file) is not a viable solution.

5. Incorrect. HTSP is indeed a protocol, but it also includes its own internal container format (HTSP muxpkt) for streaming. I was implicitly referring to this muxpkt container format.

And before you start of on another rant about why would we use HTSP muxpkt's, I'm just pointing out there are 2 alternatives that make sense for TVH, I am NOT expressing my opinion on which is better, that would depend on further analysis.

I do in fact have a personal opinion on which is the better approach, but since you've never bothered to ask, you might as well carry on your pointless arguments based on your false pre-conceptions.

Adam

Adam Sutton wrote:

1. Without the metadata in the files (i.e. MKV) it's not a proper system.

I do not catch this. Vdr for example recors in TS, adds the timestamps for iframe to ease trick mode it also records eit/epg info as a text file. Regarding HTSP, I can only point to the tvheadend wiki wiki <https://www.lonelycoder.com/redmine/projects/tvheadend/wiki/Htsp> and ask you think most user will understand when reading your sentence.

Anyway I will stop this thread as you are obviously not able to continue the discussion on a non emotional basis.

Eric Valette - wrote:

Anyway I will stop this thread as you are obviously not able to continue the discussion on a non emotional basis.

To be fair, it may be a language thing, nearly all your posts in this thread come across as angry and confrontational

btw, I have no thoughts on the format wars, I only got involved in the thread as you were stating your opinion as fact that I didn't agree with!

Your preference to have your TV play your media directly, I would guess, is probably a very minority interest, a niche within a niche, if you will.

Cheers, Des.

Hi,

I'm very new to TVH and the whole PVR world. I've been looking for a few months now to find a good client server solutions for my new house. Bought a Satellite dish and that's why things changed for me.

I find it very healthy to read such discussions on the product roadmap and architecture. Found this thread the trigger to come into this community.

Adam, I think you have a lot of commitment and faith into this project to put the effort and patience you've shown in this thread. For that I thank you! Hope you have some other good guys in the dev team to back you up sometimes...

Couple of reasons why I find this solution unique and for which I believe has the potencial to grow tremendously:

1 - It's linux, and like or not, Linux is and will continue to be elected OS plattform from which we can create stripped down versions of the OS, tunned to run software that can perform greatly in minimal hardware. OpenElec is a great example.

2 - It's a project purely dedicated on PVR backend, unlike the looks of Media Center, Mediaportal or MythTV where each tries to create their own blended BE/FE solution. Let's be realistic, for an open source project, our work should be contained and sustainable keeping the product at expected quality level. MythTv evolves very slowly because of the maintenance complexity of the whole BE/FE ecosystem. Lesson should be learned there.

3 - The PVR capabilities, configurations (admin is web based) and APIs are not too complex unlike other solutions like MythTV, and provides means of the PVR client developments to keep up with it continuously. XBMC is leading software for media center frontEnd, and the marriage with TVH has happened. This relation will keep this project running for a long time. That's why I'm here!

Thread opinion:
Direct access to the file (TS or MKV doesn't matther) should not be way to handle on client side for the timeshifting or any other functionality. From a software architecture point of view (I'm a software engineer so I feel confortable saying this), the client should not know how the PVR server stores and handles video streams, or either should have to access simultaneously the PVR api as well as a network filesystem (unless the PVR tells him to) at the same time. Would expose the server in a way that future re-design of architecture could compromise client compatibility and also make client development more complex and maintainance an overwelming task. it's much more sustainabled to have a standalone API (doesn't matter if it's HTTP or HTSP) through which you can access every PVR client action.

By the way I have two NMT and would be super to have the DLNA working with TVH. I know it doesn't exist and it's perfectly understandable by the reasons I mentioned above.

Rod

Thread opinion:
Direct access to the file (TS or MKV doesn't matther) should not be way to handle on client side for the timeshifting or any other functionality. From a software architecture point of view (I'm a software engineer so I feel confortable saying this), the client should not know how the PVR server stores and handles video streams, or either should have to access simultaneously the PVR api as well as a network filesystem (unless the PVR tells him to) at the same time. Would expose the server in a way that future re-design of architecture could compromise client compatibility and also make client development more complex and maintainance an overwelming task.

Now that time shifting API has been introduced in XBMC, pvr-addon tvheadend is one of the target that does not support time shifting. The time shifting example was just a hint to storage format because as adam recognize, not all format would be suitable for time shifting. And if one is used for time shifting and the user push records, conversion is possible but not that much satisfactory.

it's much more sustainabled to have a standalone API (doesn't matter if it's HTTP or HTSP) through which you can access every PVR client action.

Here again I disagree : I want to be able to access the live stream from retails devices. For that http is OK, DLNA also NOT HTSP.

By the way I have two NMT and would be super to have the DLNA working with TVH. I know it doesn't exist and it's perfectly understandable by the reasons I mentioned above.

It does for recordings. For live, as long as you have http URL, it could but the problem is finding a free DMS that suports either dynamic object creation (DLNA createObject) or MP3 playlist.

"Here again I disagree : I want to be able to access the live stream from retails devices. For that http is OK, DLNA also NOT HTSP."

I also want that. When I started looking, was looking exactly for Live tv (not recordings) from any PVR on to my DLNA NMTs. Have one on my bedroom and anoter one on the living room.
I tried it just for kicks with MediaPortal and the way the live recording is made is one thing, but when the filestream is actually published on the DLNA server is another.
But the fact is that no PVR has yet officially released such a feature (please correct me if I'm wrong). That should make you think why... Live TV has different software origins and concepts than just the streaming of files. From a Live TV client development POV I would like not to have too much differences between the Netflix plugin or the TVH. And in Netflix case you won't be able to use DLNA or any Samba or NFS file system access.

One other thing that should make you think: we'removing continously to SOA (Service oriented architecture) systems. If you're a developer you should already know why. If not, google it and you'll understand.

Eri, I understand your point of view, maybe technically is something pretty easy to achieve here (I would like it a lot), but the question that the TVH team will have to ask themselves is: do we really want to open this door?

Rod

Rodrigo,

Thanks for the comments. I definitely agree that by splitting the FE/BE and allowing TVH to focus purely on the BE does have several significant benefits. We also have a good position with XBMC-PVR since Lars (the PVR maintainer) is a TVH user (and notionally on the dev team).

I also agree that "how" the backend (TVH) manages the timeshifting should ideally be hidden from the end user.

There is a very definite reason for this in the current TVH time-shifting implementation plan. The first pass will only timeshift on the subscription level, it won't allow for recording what's in the buffer etc.. and it will buffer in HTSP muxpkt format (re-combined TS packets into frames, etc..). This is being done for several reasons, but mainly because its by far the simplest means of getting something up and running quickly.

However later on this will need to move up to the service level, which will allow the data to be shared between subscribers to a given service. At this point recording the buffer will be possible and because we probably want to be able to still allow the raw TS pass-thru recording option (will need to change what data is actually buffered). This will require the data to be buffered as HTSP TS format (basically TS packets wrapped with light internal header).

Unfortunately the rather poor (don't tell Lars I said that) implementation of the PVR timeshift API in XBMC does mean that what will be possible with TVH in Frodo will be limited. However the changes to support a more full operation are fairly straightforward (I think). So we'll get there eventually.

I have already demonstrated a working live pause feature. I have started working on the other timeshift features, however I've been ill and busy (in equal measure) and so have not done much on the code since I refactored it (and thus broke it ). But I hope to do some more on it this week.

As for DLNA, I think I've given my opinions on the matter, so don't see much point in re-hashing that

Adam

Rodrigo Boavida wrote:

But the fact is that no PVR has yet officially released such a feature (please correct me if I'm wrong). That should make you think why...

First DLNA support may be implemented outside of tvheadend and I do not yet see the advantages of bundling the two features. For implementing them in a DMS, you need stable TV http streaming URL plus eventually the mime type. Then the last specification of DLNA end 2009 have added a bunch of things for live/IP TV. Just the code is not yet there. Finally, the DLNA CVP Europe is exactly about redistribution of live stream to retail devices (that supports the standard).

Eri, I understand your point of view, maybe technically is something pretty easy to achieve here (I would like it a lot), but the question that the TVH team will have to ask themselves is: do we really want to open this door?

I think you misunderstood my point. DLNA is standardized and available in many retail devices. HTTP streaming also (TS is still also netter decoded) I want to be able to use one of thoses two protocols to stream live or recorded content from tvheadend. period. I'm not asking for DLNA integration into tvheadend. For me its a bad idea.

The only thing you need is a post recording script to eventually force the DMS to rescan content. It it not always necessary as they may use inotify or sys/fs notify. Then having HTTP links with mime type or even better M3U playlist for live stream is sufficient for live, the requirements are more on the DLNA server side.

Eric,

this seems to tie in with my current understanding of the situation based on your previous comments. Namely that TVH need not involve itself in the whole DLNA argument.

Other than that it would need to provide the few basic resources needed to feed an external DLNA server, namely:

- M3U playlist
- HTTP streaming interface (currently possible in MKV or TS)

A quick look at the code suggests it is possible to get a channel playlist, although I don't believe it currently exposes the fact that both a TS and/or MKV version are available. But I imagine that would not be a large amount of work to update.

Adam

Adam Sutton wrote:

I also agree that "how" the backend (TVH) manages the timeshifting should ideally be hidden from the end user.

So do I.

Unfortunately the rather poor (don't tell Lars I said that) implementation of the PVR timeshift API in XBMC does mean that what will be possible with TVH in Frodo will be limited. However the changes to support a more full operation are fairly straightforward (I think). So we'll get there eventually.

Well, I did not understand why they did put this feature so late in the frodo dev cycle. I guess this has to do with sponsoring!! They put something rather minimalistic/unachieved API and API discussion ended by "too late for frodo"...

Regarding DLNA, I think we do both agree that exposing recordings and live streams via DLNA is fine and that its not the job of tvheadend. We could discuss a bit what would help by defining the data a DMS can expects for publishing resources for live content.

The problems I have with the timeshift API are that they implemented something that makes sense for many of the existing backends (which is where the drive came from) that provide a more file orientated interface. So they have simple commands like read(), seek() etc... that can be used to cover the whole pause,skip,ff,rwd system as would happen with any other file based playback (and these backends also tend to operate on a file based mechanism for the live TV as well).

The problem for TVH is that it provides a streaming service and we don't want to have a streaming service for live TV and a file based service for time shift (at least not from an external API point of view). There are several reasons for this and while it can be "bodged" to work like this it won't work well.

The annoying thing is that actually internally XBMC already has all the relevant hooks etc.. to do things properly and since PVR (and TVH) have their own demuxers within XBMC this can in fact all be handled with a few simple changes to implement these relevant hooks in the PVR demuxer and pass-thru accordingly to supporting addons. What's more annoying is that had they done things along these lines in the first place I think quite a few of the nasty PVR specific hacks in the DVDPlayer core could actually have been completely omitted and left a much cleaner code base (which I think will happen in the post Frodo cleanup).

But this is partly my fault for not keeping on top of the changes that were being made, since I wasn't sure at the time when we'd get around to doing timeshift. It was only in the last few weeks I started playing and realising the issues.

Adam

Adam Sutton wrote:

The problems I have with the timeshift API are that they implemented something that makes sense for many of the existing backends (which is where the drive came from) that provide a more file orientated interface. So they have simple commands like read(), seek() etc... that can be used to cover the whole pause,skip,ff,rwd system as would happen with any other file based playback (and these backends also tend to operate on a file based mechanism for the live TV as well).

I thinks the file oriented API comes from XBMC VFS itself as most implementers did say that the backend use "always record on disk/buffers" even when live streaming (if you have PVR you have a disk) and then always streams recordings... You can see part of the discussion on github pvr addon

Adam Sutton wrote:

Other than that it would need to provide the few basic resources needed to feed an external DLNA server, namely:

- M3U playlist
- HTTP streaming interface (currently possible in MKV or TS)

A quick look at the code suggests it is possible to get a channel playlist, although I don't believe it currently exposes the fact that both a TS and/or MKV version are available. But I imagine that would not be a large amount of work to update.

In theory, the best way to really feed a DMS for live streams is to implement the client side of createObject/destroyObject and the UPnP discovery protocol but not all DMS support it as it's an optional feature. This itself could be done first by an external dedicated program providing all the needed information are exposed and accessible (get playlist, get mimetype for streams, get streams name and streams URL).

The other way is to make the DMS find the resources itself by creating the playlist with relevant URL as a file in the recordings directory...

Every DMS that I know of or tried ais based on directory scanning for find new files and publishing them, even if it's on demand. This is time and processor consuming process, even more when there are movie info and cover updates in the middle. If I turn on XMBC and I want to see channel A I doubt this would be a fast process.
Are we talking about a specific and customizable DMS that wouldn't do the scanning and any other stuff when it publishes the new media?

Sorry if I missed the response on other post...

Rod

Rodrigo Boavida wrote:

Every DMS that I know of or tried ais based on directory scanning for find new files and publishing them, even if it's on demand. This is time and processor consuming process, even more when there are movie info and cover updates in the middle.

Scanning a full new disk is indeed a long process, that's why DLNA server can and should do caching of FS hierarchy to only process new files exactly like rsync does...

If I turn on XMBC and I want to see channel A I doubt this would be a fast process.

Live channels URL can be fixed by channel name even if the detail to access them may chnage (e.g transponder, IP address, ...). This means a MP3U play list would almost never change and be immediately available.

Are we talking about a specific and customizable DMS that wouldn't do the scanning and any other stuff when it publishes the new media?

Like commercial DMS do...

"In theory, the best way to really feed a DMS for live streams is to implement the client side of createObject/destroyObject and the UPnP discovery protocol but not all DMS support it as it's an optional feature. This itself could be done first by an external dedicated program providing all the needed information are exposed and accessible (get playlist, get mimetype for streams, get streams name and streams URL)."

RB - Who would be the client? The TVH? or the XMBC? Both have different architecture implications. One common implication is the additional server software put in between the moment I press the button and the moment I actually see the channel. Besides refreshing the playlist, the DMS will also refresh internal database and correspondent memory indexes associated with the files, as well as parse the file and its properties to categorize it.

"The other way is to make the DMS find the resources itself by creating the playlist with relevant URL as a file in the recordings directory..."

RB - Who would be responsible for doing this? Again you're talking about more software to blend the PVR and the DMS as a single Live TV solution.

"Like commercial DMS do..."

RB - the word "commercial" sounds very wrong associated with TVH or XMBC. First of all, I don't believe people will ever want to pay and install software just to resolve TVH time shifting or any other feature. Secondly I very much like to see which kind of Linux commercial DMS exists that would support what you are suggesting that could be done. Could you give an example?

I can see you are very convict this could be the way to do it. Don't want to convince you otherwise, believe me, I would be one of "your" best customers since I have two popcorn hour and was desperate to put some live TV on these clients.
DMS will always take some (needless) memory and processor usage on the servers. I turned off my DMS from my QNAP that has 3 TB and usual re-indexation was taking CPU to 100%. Now I'm very happy with my raspberry pi xmbc client accessing the files through NFS, not having to wait for file publishs, and storing media info locally from Internet (loading media info from server side is horrible when you want big mosaico HD views!). Besides that, DLNA doesn't allow to load subtitles from file while file share does. Now I'm giving some boring speach, sorry! :)
But I'm still really not convinced that we would need a more complex client and server side solution just to fix the Time shifting or any other missing feature! There should be more elegant and SOA way to do it using merely TVH and XBMC.

I might as well tel yo what I want to achieve:

1 Mini PC with TVH connected to the DVB-S USB card and also serving as living room XBMC client
2 Rasp Pi with Openelec for other two TVs. My TVH should be accessed through PVR client. Now I haven't tested this yet...only tested with XBMC with the Mini Pc nd a wireless connected windows laptop which performed very well. Need to be sure after installing Openelec and getting the Rasp codecs that the rasp will be the right client for the job.

If anyone sees any problem with this configuration would appreciate your comments!

Tnks
Rod