Tvheadend

Martin,

you're not entirely correct about series recording. Andrea's fork currently does not have support, my own, which includes a complete EPG system re-write does include such support (where upstream info is available).

The available support is very much dependant on what is available in the upstream data source, e.g. basic EIT doesn't have enough info, most XMLTV scripts don't have enough info (those that use dd_progid are now supported for series linking). Most of the more useful features only currently exist for UK sources (since that's where I am) but the model is now in place to accept the data when/if it becomes available.

Andreas has agreed to change the way the TVH project is run, to open it up to the community, and so I am expecting to merge my code at some point in the not too distant future. But I'm currently waiting for Andreas to actually hand-over his git repo (he's away from home at the moment).

Exactly how that all fits into what you're trying to do I'm not sure but I thought it was worth making a note of what's going on.

Adam

Hi.

Although I'm aware of that you are working on improving tvheadend, I completely missed that you are taking care of series recording too.

Actually, I'm only trying to find a solution for the "more_recent" rule. But like you've said you need valid epg data. And frontend interfaces to configure it.

Implementing the frontend stuff can be done, I'll check your branch to see what you have.

But the valid epg data seem to be a big problem. EIT is crap and I don't know of any xmltv service providing the needed information.

So for me the only solution is to build a custom scraper for this. But instead of populating the whole epg database with the needed info, it would be a lot more efficient to scrap and decide at the moment, the autorec logic has found a matching epg event.

Anyway I'll check your work, it would make more sense to help you providing better epg data.

Cheers,
Martin

Hi Martin,

to some extent series recording is a by-product of a better structure in the EPG database, however it was also one of my primary reasons for getting into the whole EPG code in the first place. Unfortunately series linking is dependant on upstream data, though I fallback to a standard title based autorec should someone request series link for something that does have the required info.

You are indeed correct that most upstream providers are crap. Standard EIT does not carry the required info, though there are some special descriptors (they are part of an addition to the EIT standard, see www.etsi.org/deliver/etsi_ts/.../ts_102323v010301p.pdf). These provide a system for adding CRIDs (unique'ish IDs for episodes/series). I'm not sure how wide spread the use of CRIDs is but I know that Freesat in the UK use them for their series link info.

Also many of the other EPG OTA standards OpenTV (used by Sky, and supported in my branch), MHEG5, MediaHighway also have the ability to include series link info. Again I don't know which are used where. OpenTV is a bit noddy in the way it does series linking (things are service locked, i.e. Episode X shown on both service A and B will most likely have 2 different series IDs, but I'm working to cope with that).

As for XMLTV, there are some providers (generally commercial ones, eg schedules direct) that support the dd_progid episode numbering namespace. This allows for more definite unique episode IDs as well as series info. And so can be considered similar to some of the other things.

In the UK there is an EPG aggregation service known as Atlas, this currently is (indirectly) providing the UK XMLTV feed. However they have a JSON based API for access the underlying database which is really well structured (much of the inspiration for my new EPG structure comes from them). At the moment the JSON API is NOT free (even though 90% of the data is already available in a flatten form for XMLTV) however metabroadcast (the company that run the system) are working to change this. So for UK users at least there could be (via my PyEPG script) access to some excellent EPG data.

Obviously not much good for yourself and everyone else outside the UK, but what I'm hoping is that this sort of thing will get better over time and by at least including the required support in TVH (in terms of a good DB structure) it'll be trivial to get the data in when its available.

Anyway, long winded rant over back to your specific point.

I'm not quite sure what the best way to handle your situation is. Personally for most XBMC users (which is a significant percentage of TVH users) I would say the best option is for things you want to keep rename them (auto or manual) to use the naming scheme used by XBMC and import into the XBMC database. But there may need to be some help to deal with this.

Exactly where this should happen I'm not currently sure, e.g. should we build a "scraper" system into TVH? Or do it as a post processing step externally? One of the big issues, as you point out is that often the necessary data (bare minimum is usually series number / episode number) to do a decent scrape simply doesn't exist.

But I'm open to suggestions

Adam

Hi Adam,

currently I'm using a post processor script which does some scraping after the recording in order to rename the file for XBMC.

This is just to have XMBC displaying nice information.

But this brought me to the idea to extend the concept for autorecoding rules in order to decide if a autorecording rule should really lead into a scheduled recording. With this, EPG data inside TVH needs only to be time, title and channel as it is now. Works for crap data.

The pre autorecording processor script would take this data (time,title,channel) and can do whatever is needed. exit status 0 would let the scheduler create the recording.

For sure, a better EPG with series information would be the goal. I guess you are on the right track to enhance EPG structure and provide external interfaces to feed the database.

My "works for me" approach is just to avoid populating thousands of EPG objects with rich data where only a couple of them are used by recording rules.

Anyway, I'll try some commercial EPG services to see what data they provide.

Martin

Hi Martin,

Can I just double check I've got things right (sorry to repeat what you've already said):

Your goal is to create a simple autorec rule (based on things you've noted) that will cause a (web) scrape to occur which will determine whether to proceed with the recording?

If so it's definitely doable, do you have a working prototype? Or is this more of a feature request?

If the later, then I'd suggest that it will probably have to be the case that the autorec rule will still setup the recording (regardless) and spawn the script (in a thread, to stop it clogging things up, since autorec decisions run in critical path code) with enough information that if a negative result is returned by the script it can later delete the recording.

Adam

P.S.
I have plans to make improvements to the DVR code (so this could fit into that), but I'm waiting until I've stabilised my EPG work first.

Hi Adam!

Sorry for the late reply....

Yep, you got it. And your're right to have the script forked. In this case, it's more an event notification mechanism (calling an external program) saying that an autorecording has been scheduled. The scraping part is only my implementation of this kind of interface and has nothing to do with TVH.

I haven't started working on this, just thought the idea would be useful.

Martin

Martin,

I think its not a bad idea, and does provide the ability to "externally" add more flexibility to TVH. We might consider adding such "hooks" to other areas as well. But I don't know about that yet.

I think it would however we worth submitting this as a feature request, it'll probably be considered low priority (especially while we try and sort out all the general stuff needed to get the project back on track) but its worth noting it down.

Of course if you fancy helping out and providing a PR then we'd be very grateful and it would no doubt have greater chance of inclusion in shorter time frame

Adam