Tvheadend

Oh that was a misstake, can be closed

This is absolutelly fantastic idea!
Makes is so easy to find out what is wrong with scheduling and then adapt schedule to available TV cards.
+1!

It is a small change and i think that it would make tvheadend better, but still noone is interested in implementing it...

Although I like the idea, I think it's slightly more complicated than it first seems. For example, I have a setup with DVB-T, DVB-T2, and DVB-S2. Some channels are broadcast on all three, some on two of them, and some on only one of them. Most muxes broadcast multiple channels. So perhaps one tuner could record "Tele5" and ZDFInfo" without requiring a second tuner.

So, if I have three programmes to be scheduled they may be able to be all recorded on one tuner, or they may need all three tuners. The tvheadend only allocates tuners right at recording time, and can even failover between tuners if something is wrong, such as fails to tune or the tuner is offline. Whereas some other pvrs pre-allocate tuners at scheduling time.

An alternative idea that a different pvr has is an ical on its recording page (coupled with a filter). So you can subscribe in your calendar application for upcoming recordings and see them all side-by-side in your calendar and then click on them in the app to go to the details page to edit/cancel the subscription. You can also subscribe for completed recordings, etc.

I would suspect that the ical for upcoming recordings is easier to implement since the implementation would be similar to the existing api calls.

@Em Smith

Let me try to do some brainstorming.

Will it not be helpful to split the problem into 2 issues?

1) Detect if the timers exceed the tuner capacity and inform the user when it happens.

2) How to solve the problem of the timers exceeding the available tuners?

How can problem 1) be approached?

(I am assuming that the failover you mentioned in reply #6 is meant for tuners of the same type. Or is tvheadend already able to move a timer from a channel on DVB-S to a channel on DVB-T?)

In fact, I am wondering, whether the problem does not boil down to check if at any given time, the number of needed muxes (not channels, as a mux carries multiple channels) does not exceed the available tuners for that mux. This does not take into account that for example "Tele5" might be available on DVB-S and DVB-T, as they are considered to be different channels since they come from different muxes. But it does tell the user, that it is not possible to record all the timers the way they are.

How can 2 be approached?

a) Manually by the user

Once the user knows what timers conflict, he either has to disable some timers or on a mixed setup move a recording to a different input way, for example from DVB-S to DVB-T.

b) Automatic

This will probably involve additional setups:
- tvheadend must know that "Tele5@S2", "Tele5@T2" and Tele5@T" are equivalent channels on 3 different input ways.
- define priorities for the input ways: for example prefer "Tele5@S2" over "Tele5@T2" over "Tele5@T".
- define priorities for the timers
Have tvheadend distribute the timers according to the priorities.

Of course, this whole reply assumes that tvheadend has knowledge about the number of available tuners for every input way. This might be difficult for setups also involving Sat>ip or similar. What about a setting for each input way where the user can tell tvheadend the number of tuners to consider for the timer conflict detection? This way, the users with a fixed number of tuners will know whether their timers are ok; and the users with the number of tuners varying over time can at least get an idea of the tuners required to record the available timers.

Hoping to have been helpful.

Cheers

b) is already implemented, just assign more services to the channel and define your priorities in the tuner settings to prefer some input against another

@Ludi K. K. your explanation is very good and exactly what i mean

But the get reliable tuner capacities, it should be possible, to add tuners only for streaming and only for recording, so that you can have always a tuner for watching live tv, and maybe use it as a spare tuner, if it is really necessary for recording :D

Also: I have the problem that i have 2 DVB-S Cards in the pc and a network sat>ip tuner with 4 DVB-S Cards. The network tuner has the risk of data losts and i see often artifacts in the recordings of that and i don't want it in my recording but it is not so bad for live streaming. Yes i know, there is the record priority and streaming priority, but if i watch the same mux, it take it for recording to, or if i idle scan this mux.

The kingsway for the recordings could also be, that if a recording conflicts with another, that tvheadend looks for the next time the video is in tv and record this instead. But this should only be an opt-in in the settings.

Tonight I sat down to watch a recording only to find the first hour hadn't recorded due to a conflict, so I have sympathy.

Re: scheduling

Since tvh has options like "auto-re-record on error", this means that you could look at your guide in the morning and have no conflicts, only to have a programme in the morning fail (such as due to bad weather breaking up the signal) and be re-scheduled for the afternoon and cause a conflict with your other recordings that you thought were ok. So, the only real way to solve it is to do the significantly more complicated scheduling.

In my country, I'm fortunate that most programmes are repeated ad nauseum so it's easy to record most programmes later.

There is an alternative free PVR called mythtv that does much of the scheduling you mention such as reserving tuners for live tv, priority for different tuners or channels, and more options for pretty much anything you can think of such as preferring to record a programme in HD (or SD), or with subtitles, or first/last showing of a programme, or boosting priority for sports shown on a Saturday night in HD on a sports channel, and very good conflict resolution such as recording a programme later due to conflicts.

But it is far more heavy-weight in terms of resources and databases. Basically it has to do (IIRC) nine passes over the data to prioritise recordings and schedule later recordings. This it has to reschedule every time you watch tv (since the tuner is busy), and again every five minutes due to EPG updates. So, this can't run on low-spec hardware and probably wouldn't fit in nicely with the tvh model since it causes a busy cpu from a few seconds to sometimes several minutes. But, it does give amazing scheduling results. It doesn't support sat>ip though.

So, there are trade-offs of machine spec and complexity. I expect that tvh will eventually implement more complicated scheduling since I think automatic scheduling is the only way to go.

To answer your other question, yes TVH can schedule between different tuner types since you can "merge same channel name" on the mapping option.

I hadn't realised a tuner couldn't be reserved for live tv. If you set the rec priority of all other tunes to a high value and the sat>ip tuners to a low value then this might help, but not something I've tried.

To clarify why "full" scheduling is time consuming for some regions, I'll give an example of recording The Simpsons.

In the next week, there are 250 episodes shown here on 19 "different" channels I receive (due to receiving different tv regions).

But this is only actually 12 unique episodes across one SD channel (and one +1 hour timeshift channel) and the same two channels in HD.

So for a full schedule, mythtv has to (soft-)schedule all 250 episodes, calculate their priority (based on time, date, channel, HD, etc), check which are duplicates based on rules against all other recordings and against previous recordings. Then determine which ones should actually be scheduled and can give the best results on conflict. If you multiply this by, say 200 recording rules then six tuners you can see scheduling quickly gets large to get results that seem obvious to us humans.

I was not aware that tvheadend already supports failover across tuner types; thanks Jaroslav Kysela for the piece of information. This probably explains why there is not a conflict detection in tvheadend yet: because of the failover across tuner types, it is not sufficient anymore to count the muxes for each tuner type to determine timer conflicts.

@Mytril Goldhand Goldhand

If you define your recording as an auto-rec, I suppose that you get the behaviour you are asking for in message #10. (Please, anybody correct me, if I am wrong.)

@Ed Gr Smith

Apart the trad-off between light-weight and complexity, there is another trad-off to keep in mind: ease of use vs complexity. IMHO, it is important for tvheadend to also remain easy to use. What is for example the point of having a scheduling with whistles and bells when only few people are going to use it because it is to complicated to set up.

For me, with a mixed DVB-T/DVB-S setup, the "problem" I hit last night is a programme started recording on DVB-S, then an hour later, it was overriden by another programme that could only be recorded on DVB-S and was higher priority. So the first programme was cancelled from DVB-S and then (correctly) failed over to continue recording on DVB-T, with the second programme (correctly) recording on DVB-S.

So existing behaviour is:
9pm->10pm: prog1 started recording on DVB-S
10pm prog 2 needed to start, but channel was only available on DVB-S
10pm prog1 was automatically cancelled, prog2 starts on DVB-S
10pm prog2 rescheduled on DVB-T and started, losing four seconds of the show.

An interim solution could be at the start of a recording to do a look-ahead for a couple of hours to determine the best tuner to allocate so prog1 would be allocated to DVB-T up-front instead of DVB-S. I'd have to look at my recording schedules to see if it would actually solve anything.

Slightly off-topic reply bits below:

Yes, almost all my rules are manually created autorec rules to match something I'd probably like to watch. If it missed a showing, then it normally records later in the week on the repeat channels.

The trick is to work out why you watch something, then create a rule to match similar programmes in the future.

For example, "I like Mission Impossible films", so rather than look at the tv guide every day to see if it is on, you set an autorec to record when it's next on. Except, you then determine that you actually only like the film because it's Tom Cruise. So, alter the rule to pick up all Tom Cruise films, or action films, or action films made in the late 1990s. Apply that logic to a few more rules and suddenly you get things recorded that you like without having to tediously search through tv guides.

Or, as another example, if you have young children/grandchildren, "record all animated films, limit to 10 recordings" (which for my country is an autorec regexp of "(animated|animation|anime)" and content type of movie) and you have something for them to potentially watch.

I agree that you don't want too much complexity, but sometimes you need it, even it is hidden away as an option. For example, if you're recording a ten-part thriller series and the final showing of the final episode isn't recorded due to a conflict, you might be upset.

@Em Smith

Thanks for the detailed example of the fail over splitting the record in several parts (two parts in your example) recorded on different Input was.

As interim solution, you propose a look ahead for a couple of hours at the start of a recording. That would probably mean to do the look ahead at each recording start. At this point, it might be better to do a conflict catch when a new timer is added, assuming a light weight conflict catch is possible.

I would suggest to start with a conflict check without taking failovers into account; that would at least solve part of the problem, as the user as the opportunity to manually interviene before the conflict breaks a recording.

What about the following approach:

1) Since a record on DVB-T cannot for example conflict with a record on DVB-S, the available input ways get checked separately.

2) How to check an input way for conflicts:
- Build a cronological list containing all start and stop times of the records for the given input way.
- Go through the list:
-- Each start time that needs an unused frequency creates an n-tuple of the form (frequency, timer-id using the frequency.
-- Each start time that needs a frequency that is already in use, appends its id to the already existing n-tuple for that frequency.
-- Each stop time removes the timer-id from the corresponding n-tuple or the n-tuple gets deleted if it was the only timer.
- When the number of n-tuples is greater than the number of tuners, there is a conflict involving the timers whose timer-ids are in the n-tuples available. All those timers might get marked as being part of conflict 1. (I say conflict 1, because at a later point, there might be another conflict involving different timers, which then has to get marked differently than conflict one.)

Please, be aware that the above is only an idea and might be missing some points. For a working algorythme, you might look at the VDR code, which has a working conflict check, however without conflict handling; it only announces the conflicts.

Finally, a few words about your off-topic bits: You will rapidly get a big amount of timers, if you start looking for actors, producers or other key words... How do you handle that big amount of timers and corresponding conflicts? By keeping the numbers of rules small?

For my timers, I actually use a combination of mythtv and tvheadend. I want to migrate it all to tvh but there's still some functionality missing that I need for the way I use pvr.

Since you ask, in mythtv for the next couple of weeks I have just under a thousand potential timers using 130 recording rules with zero conflicts. Mythtv has amazing conflict resolution since it will record programmes later if necessary, or move from a preferred tuner to a less-preferred tuner to ensure there are no conflicts.

For my timers, a large number of those potential timers are not allocated a tuner due to being repeats of already recorded programmes I've watched (mythtv doesn't re-record things you've watched unless you want it to), or no tuner due to being recorded earlier or later to avoid conflicts, or due to "don't record more than 10 of this programme", etc.

For example an episode I watch is on 21 times but is allocated a tuner on only one day on one dvb-t input, the rest are marked as duplicates. Another programme has 158 potential timers but zero allocated tuners since I only want maximum of 10 recordings, but, if I watch a recording then the next one will be scheduled.

Every few days I look at what is potentially recording and hit "never record" if I don't want it. So, I will schedule to record certain types of film if they have a 7/10 rating or higher or have certain keywords, but many of them don't sound interesting so I mark them not to be recorded. For example, Monday I have ten programmes currently scheduled but only want one to actually record.

To me, this is easier than looking through tv guides for a hundred channels trying to find something to record, and you also pick up some surprisingly interesting programmes that you've never heard of and you rarely miss recording something that would be of interest. Many of the rules are lower priority so that if there is something I explicitly schedule then that it recorded if there is a conflict rather than something generic.

I think my requirements slightly diverge from yours because you want to know about conflicts, whereas I want tvheadend to resolve conflicts.

For example, yesterday I was testing some change and had three test recordings. I restarted tvheadend and it could only restart recording two of them due to it allocating tuners in a different order. Clearly that's not ideal since I had no conflict, restarted and then had a conflict. This is because dvb-t can conflict with dvb-t2 and dvb-s/s2.

For me, dvb-t2 is a superset of dvb-t (all channels in dvb-t are also on my dvb-t2 tuner, though that might not be true for many countries). Some channels are identical in dvb-s2, some are in dvb-t/t2 only, some in dvb-s2 only.

For example, news channel is on dvb-t and dvb-s, with news HD in t2, and s2 showing the same programme. So if I want to record "news" it could schedule it on any of them, but perhaps I have "prefer HD" enabled in the settings so it will try to schedule it on s2 or t2.

One of my movie channels is on t2 only. Another is on s2 only. So if my "news" was scheduled as "news HD" on t2 then I might not be able to record my movie since it is on a different t2 multiplex. Whereas if it had been scheduled on to s2 or non-HD version recorded on dvb-t then I could've recorded the movie.

I think we have to build up a list of frequencies, maybe sort all recordings by priority,starttime,name, allocate tuners, if we can't then mark as a conflict. But, it really needs multiple passes to get things right. So, news is allocated as "news HD" unless it is conflicted in which case try to allocate as "news HD" on other tuner, else allocate as "News".

I think it's a lot of work to determine a set of cases that needs to be solved before even working out how to solve it. Also we need to take in to account that every time you watch a programme you might be getting EIT updates indicating a programme has overrun or a new showing of the programme is now on next week.

Em Smith wrote:

For me, with a mixed DVB-T/DVB-S setup, the "problem" I hit last night is a programme started recording on DVB-S, then an hour later, it was overriden by another programme that could only be recorded on DVB-S and was higher priority. So the first programme was cancelled from DVB-S and then (correctly) failed over to continue recording on DVB-T, with the second programme (correctly) recording on DVB-S.

So existing behaviour is:
9pm->10pm: prog1 started recording on DVB-S
10pm prog 2 needed to start, but channel was only available on DVB-S
10pm prog1 was automatically cancelled, prog2 starts on DVB-S
10pm prog2 rescheduled on DVB-T and started, losing four seconds of the show.

You can assign lower priority to DVB-S tuner (limited resources) or highter priority to DVB-T tuners, so tvh will use DVB-T tuner as first then DVB-S.

I think part of my problem is that not all my channel names are merged so tvh doesn't always know it can use another input.

For example DVB-T uses "Channel 5 HD" whereas DVB-S2 uses "Channel 5HD" (without the space), or "Trutv +1" vs "Trutv+1", and I've not yet manually merged them.

I tried priorities a while ago but I had some problem where it (correctly) couldn't access the high priority tuners due to them being busy and so tvh failed-over trying each of them, but would never then try the lower priority tuners. So, it went tuner 4, 5, 4, 5, never trying 0-3. Which is correct (I said use high priority tuners), but also wrong (high priority tuners unavailable so should try low priority).

Em Smith wrote:

I tried priorities a while ago but I had some problem where it (correctly) couldn't access the high priority tuners due to them being busy and so tvh failed-over trying each of them, but would never then try the lower priority tuners. So, it went tuner 4, 5, 4, 5, never trying 0-3. Which is correct (I said use high priority tuners), but also wrong (high priority tuners unavailable so should try low priority).

You can use '--trace service' to see which instances (service tuner list) is used and which priority / weight is applied. TVH should try to use all allowed tuners.

You're right. Perhaps on my previous attempt the channel hadn't been merged due to whitespace differences on some of my channel names on dvb-t vs dvb-s.

I re-tested it and it is working fine now. So I recorded programmes that were only available on satellite, then started recording another programme that was on both and it tried (in my case the higher priority) satellite and then failed over to aerial.

If I changed priority for dvb-t to be higher than dvb-s then it correctly tried aerial first.

The log always logged a subscription with "weight: 500" regardless of the priority I assigned to the tuner, so I guess that's the programme priority of recording.

Weight is for the subscription, priority is for the input selection. The weight is applied when no free tuners are available (so subscriptions with lower weight will be closed).

Em Smith wrote:

I think my requirements slightly diverge from yours because you want to know about conflicts, whereas I want tvheadend to resolve conflicts.

You are right: I prefer to know beforehand if there are conflicts, even if it does not take into account tuners that might be occupied by people watching Live TV. For example, when I record a TV Show that I intend to keep, I would like every episode to be recorded on the same version of a channel, not on some failover channel.

On the other hand, you seem rather to be looking at conflict resolution at recording start.

Your point of interest, might however not exclude mine, provided a correct configuration of the recording rules and a conflict detection that can be configured to not only run at the start of a recording.

For example, yesterday I was testing some change and had three test recordings. I restarted tvheadend and it could only restart recording two of them due to it allocating tuners in a different order. Clearly that's not ideal since I had no conflict, restarted and then had a conflict. This is because dvb-t can conflict with dvb-t2 and dvb-s/s2.

Ok; that was surprising. How did you detect the conflict situation in tvheadend?

Also we need to take in to account that every time you watch a programme you might be getting EIT updates indicating a programme has overrun or a new showing of the programme is now on next week.

I suppose that EIT updates are only of importance concerning conflicts, if they perform modifications in the timerlist.

Jaroslav Kysela wrote:

Weight is for the subscription, priority is for the input selection. The weight is applied when no free tuners are available (so subscriptions with lower weight will be closed).

Ok; that's good to know.

I looked through the VDR epg plugin code and the screenshots and it's interesting and I can see why if you're used to that behaviour it's something you'd like. The vdr code also takes in to account decryption which I had forgotten about (since my channels are not encrypted). I took a quick look at the tvh subscription code and don't understand it enough to implement the logic you want, but maybe someone else can.

The conflict I had was logged in syslog as "service instance is bad, reason: Tuning failed" every ten seconds or so. Inside tvheadend GUI itself, I forget if it's notified. I could tell because the filesize wasn't increasing. I don't think there was an explicit indication.

Em Smith wrote:

The conflict I had was logged in syslog as "service instance is bad, reason: Tuning failed" every ten seconds or so. Inside tvheadend GUI itself, I forget if it's notified. I could tell because the filesize wasn't increasing. I don't think there was an explicit indication.

If you don't have free adapters, the 'No free adapter' should be logged. 'Tuning failed' is a different error.

Em Smith wrote:

I looked through the VDR epg plugin code and the screenshots and it's interesting and I can see why if you're used to that behaviour it's something you'd like.

Thanks for looking into it.

Em Smith wrote:

I took a quick look at the tvh subscription code and don't understand it enough to implement the logic you want, but maybe someone else can.

You might perhaps give some pointers here at what you looked at, in case somebody wants to pick up the ball.

On the other hand, I can understand that the conflict check might be harder to implement in tvheadend, because of features like the merging of different versions of a channel, which are not yet available in the VDR.

Sure I can document, but bear in mind I only took a brief look and so my understanding can easily be incorrect.

In tvh, the recording will look through the epg for a match. Other recordings for the same episode appear to be excluded, presumably because the UK has around 15 channels broadcasting almost the same programmes (99% of the time) at the same time so it makes sense to exclude these obvious duplicates rather than cluttering up the Upcoming tab. The consequence of this is that you can have a non-merged channel as the "master" for the recording, so instead of recording on "5 HD" it could pick "5" (or 5+1 timeshift channel) instead and the "prefer hd" setting in the profile can't be honoured since it's cancelled without being scheduled.

At recording time, we cancel any other dups based on user's dup criteria, probably because uk OTA doesn't always broadcast detailed episode info so we can then cancel them based on "dup same description." This means we don't schedule duplicate recordings on "ITV (North)", "ITV (South)", "ITV (East)", etc. and this then ensures we exclude duplicates that were created from some other autorec rule.

So the (master) timer has an associated channel which has its associated services (e.g., some UK HD channels don't broadcast news and instead broadcast a programme consisting of a poster telling you to tune to SD, but some people merge HD/SD regardless hence two services for one virtual channel; or merge dvb-s2 and dvb-t2 versions of same channel).

We need to go through the recording profile to determine what service to use since the recording profile can say "prefer HD". So we iterate through the services to find the one to use. But these are services only on the specific channel we are using. For example, if you have 15 channels broadcasting the same programme, we only try that specific channel and don't merge services from other channels with same broadcast. The implication here is that if your dvb-t and dvb-s aren't merged channels then it will only schedule on the channel it picked and won't pick up the channel on the other interface.

If we can't find a match for the profile then we choose one. For example, "prefer HD" but no HD available then it will choose SD.

I only briefly looked through the rest. It seemed that on failure we look through the list again but have a code set on the service so we ignore the one we just used in preference to other services; but I didn't fully investigate.

I also didn't fully look in to whether preference is given to a service already in use; so if I record "channel 4" but "channel 5" on dvb-t shares the same mux as "channel 4" then whether it would prefer that service rather than "channel 5" on dvb-s if all other criteria were equal.

There's quite a lot of other logic for basically ensuring connections are shared correctly.

I was mainly looking to get a gist of whether it was something easy to do because sometimes you can grab some information that you need and process it. But my guess (based on limited analysis) was that it wasn't going to be easy without doing more analysis and taking some notes on how it all interacts. Also decryption would be something I'd have to investigate since I've not looked in to that at all since I believe they limit the number of channels you can simultaneously decode which then affects whether something is conflicted. Overall, the tvh scheduling was simpler than I expected due to it not merging services at recording time.