Tvheadend

You're right on a couple of accounts:

1. C5 is indeed crap at getting their data right!
2. TVH will disregard any attempt to re-record an already recorded episode (by its reckoning), this is something I need to sort. But you can always unlink the recordings from the TVH history.

However its nothing to do with descriptions, it was at an earlier point in time but this caused lots of problems for the very reasons you mentioned.

If you're using Freesat/view/opentv for EPG then it should be using the broadcast series link info just like a normal STB. However there are some known issues with opentv (since the protocol is proprietary and the rev-eng isn't perfect). But C5, as noted above, does seem to be pretty shite with its data and there are a few known issues (though not this one) with C5 data.

If you use XMLTV proper series linking will be disabled and will be replaced by pseudo series linking (i.e. title matching) and no attempt to block duplicates will exist.

Adam

Thanks, Adam - that makes it clearer. Given some other issues I've had with recordings dying early, I'm going to stick with the manual approach for the moment since that way I know I can force a re-record on a timeshifted channel if it's something I consider to be potentially injurious to my health to not record.

I'm looking into the Atlas stuff as well, so I'll see if that gives anything different.

Cheers... Ian

Yeah I'm definitely hoping to update PyEPG when Atlas Deer finally arrives. If the service has overcome some of the issues with v3 then I might switch over from Freesat since I have all the support in TVH to properly take advantage of the power of Atlas

Adam

Adam Sutton wrote:

If you use XMLTV proper series linking will be disabled and will be replaced by pseudo series linking (i.e. title matching) and no attempt to block duplicates will exist.

Why is that? Series information in EIT is many times just crap. XMLTV provides an easy (and structured with xmlt_ns) way to add series information to epg events. For example, I managed to populate xmltv fragements with proper information to be loaded using xmltv.sock. They show up fine in the EPG view of TVH - and also in the recording filenames which can be scraped by xbmc.

But now, it seems that I'm unable to make use of series linking, since it's not available when creating an recording rule.

Cheers,
Martin

Martin,

Previously I did a pseudo-series link using title matching and using description hashing for unique checks. The former was actually a bug in the code (it wasn't checking the series link identifiers) and the later caused a horrible mess in many many setups.

TVH will use series link identifiers if and when they are available. For the UK, we're lucky that our OTA/EIT data uses the CRID system for proper resource (series, episode) identification. It's still not always perfect, but it works 95% of the time (100% on the channels I use).

XMLTV provides no such system, yes there is some info there that in addition to extra scripts can be used to create such a system, and indeed others have added XMLTV extensions (dd_progid, for example) that does provide some semblance of what is required for series linking.

I have also seen some other scripts (a swedish one I think) that includes a URL (that I think points to thetvdb or something like that) which could be used for episode ID (but not sure it has series link info, but clearly could).

If you want to add an XMLTV extension that would provide such info, maybe we can discuss it and I can include support in TVH for processing the fields accordingly.

Adam

Adam, could you please point me to some info on the current structure of CRID urls used in your EIT data?

As far as I understand, the uniquely identify a specific show, but I'm not sure if they really identify a unique event (i.e. linked to a broadcast date) or "just" a specific episode. Later would already be the xmltv_ns system. Maybe I get it wrong, but why isn't xmltv_ns sufficient to do series linking? From my understanding, it would be enough to know which specific episode is already recorded, to avoid recording it again.

Sure it would be easy to introduce a <episode-num system="crid">CRID RESOURCE LOCATOR</episode-num> xmltv extension. But currenty I've no glue what's inside a CRID.

Many thanks,
Martin

Martin,

CRIDs are specified in TS 102 323, They are a URI of the form crid://PROVIDER/ID, where PROVIDER would be the broadcaster (or something similar) and ID would be a unique episode or series ID (which depends on the CRID flags in the EIT descriptors). #

They do not uniquely identify broadcasts, no need for that, they uniquely identify episodes and provides linkages between broadcasts (series). They are used by STBs to provide things like series linking capability and dup detection. CRIDs are used by Freesat/view (UK S/T FTA), they are used elsewhere but I'm not sure how widespread adoption is.

TVH internally maintains unique URIs for episodes, seasons, brands and series links. These can come in multiple flavours depending on the data source. XMLTV only currently has one such scheme parsed in TVH (dd_progid, defined by schedules direct). I know some providers give an episode URL that would definitely be used for episode ID, but not sure they provide a series one. In fact I added some extra IDs to my own XMLTV generator, but I think I removed the parsing of those fields from TVH.

Use of the xmltv_ns is not good enough for series linking, since you still need to do title matching, and that's prone to failure. Plus you can already do that manually. Many series in the UK (not sure about other parts of the world) have titles that vary from one show to another in a series and there are some instances where different shows share the same title.

Adam

Thanks, I'm such a fool....

I really missed the fact, that series linking is not linking episodes within a series but, as you said, linking broadcasts belonging to the same series.

Since you parse dd_progid, do you do something with the ID itself (other things than just using it as part of the ddprogid:// url)?. Further, I don't understand that dd_progid doesn't contain any season information. If you just use the ID as unique reference to a series, I could easily add a fake ID to my xmltv import.

But how does the recorder know whether the specific episode is recorded or not, since it doesn't know to which season an episode belongs. This would be only possible if the episode number of dd_progid is absolute.

Martin

Martin,

TVH actually has 3 types of linkage:

Brand - Which links episodes (generally Brand is analogous to Show, i.e. The Simpsons, The X-Files, etc..), almost NOTHING supports this but its there for a) future proofing and b) because I do have a UK source that does provide this info.

Season - Which links episodes. This is pretty straightforward and is what most people think or as a Season/Series, i.e. The Simsons Season 1 (2,3 etc...). These two levels will be familiar to anyone that buys DVD box sets of TV shows and/or uses XBMCs TV shows library.

Series Link - Which links broadcasts. This is a broadcast/DVB centric model, most of the time a Series Link is analagous to a Season. However the model used by the broadcasters a) does not require this (its occasionally possible to have random thinks linked, maybe for some seasonal specials week or something) and b) often links a specific set of broadcasts (i.e. there might be one series link specifically for the prime time airing and a completely separate link for the repeats).

For the most part, because its not explicitly listed as a Season, things are mapped into Series Link's.

With regards to the specific processing of dd_progid, the values break down into 3 parts, TTX*.Y*.

TT = Show type, it can either be SH/EP or MV I think
X* = Season reference
Y* = Episode reference

For MV/EP the full reference string is used as the ID of the episode. SH does not provide an episode ref, so you won't get dup detect on those. Typically though these are things that don't repeat like news programmes I think.
For EP the ref up to the . is used as season reference.

And having just typed that I think there is a bug in the code I think that last statement should be true for EP and SH. so I need to look into that.

Maybe if you have an idea of how you will get your references and what they will look like we can consider whether this is approach is in fact the best one to use. Personally I don't like dd_progid, however its better than nothing so it would be bad form of me not to process it.

Adam

Adam Sutton wrote:

With regards to the specific processing of dd_progid, the values break down into 3 parts, TTX*.Y*.

TT = Show type, it can either be SH/EP or MV I think
X* = Season reference
Y* = Episode reference

IMHO, the X* part isn't a season reference but a brand reference. I don't like dd_progid either, since it seems (found no real proof or specification) that episode reference is an absolute number.

But combined, dd_progid and xmltv_ns could be sufficient.

I see that you overwrite epg_episode_num_t.e_num with the episode reference from dd_progid, which would be wrong, if my assumption above is correct.

I'll try playing around with dd_progid and xmltv_ns - anyway using a more powerful numbering system inside xmltv would be much better.

Cheers,
Martin

Martin,

you might indeed be correct that X* is better described as a brand link. However my rule (for now) is that unless I know otherwise I consider things to be broadcast linkage (series link) as its a safer way to link things. But I'm open to suggestion on that.

With regard to adding your own custom system, how are you planning to do it? And personally I wouldn't use these existing approaches (unless your desperate for some level of compat). I would recommend that putting something like a <uri> field in the programme tag, with addition <branduri>, <seasonuri>, <serieslinkuri> etc.. would be a better solution (its something I've suggested, sort of, on XMLTV forums, but never followed up on as I don't like XMLTV :p).

I'm more than happy to add support for such fields in the TVH XMLTV parser as I need to do this anyway as I believe at least 1 script provides a <uri> (or <url> field based on tvdb.com URLs.

As for the bit about overwriting episode number that doesn't sound right. But yeah I do seem to be doing that, I think that's a bug (it should possibly only set that value if nothing is currently set from say xmltv_ns), feel free to report it.

Adam

Adam,

I didn't report anything yet, since I'm trying to get series linking to work initially first.

By now it seems to work! I've enhanced the xmltv config with additional dd_progid information (collected by title matching ;)), inserted in the xml dom before the xmltv_ns. And as I can say from the recording schedule, the proper list of shows is scheduled.

Big thanks for that and your help.

Martin

Martin,

How are you getting the extra info? Where are you getting it from etc...? Still not sure whether dd_progid is the best solution to the problem, i.e. I think we might be able to add some additional/custom fields that are more applicable.

Adam

Before feeding the xmltv data into tvheadend I use a self written parser script which modifies the data.

Since I only have a few series I record on a regulary basis, I simply enhanced this parser script with a configured regexp matching on the title and the proper dd_progid, which is manually entered.

Sorry that I don't have any better news on this.

Martin