Tvheadend

Can you try entering the "^New: Arrow$" in to the EPG window?

The ^ doesn't do anything. With or without it, typing New: finds a bunch of programmes, all starting with New:, whilst 'New: ar' finds just New: Around The World By Train... and 'New: arr' finds nothing.

All unwanted upcoming recordings (there's another Strain episode now) have a comment that says:

Auto recording: Created from EPG query

Either that's lying or TVH is keeping some old autorecs hidden somewhere that don't show up in the list. So I just deleted (^new\:\s)the strain and created another one for New: The Strain with an empty comment. We'll see what that does.

I'd update all the comment fields to be unique, disable/re-enable them all, then you'd know exactly which rule was creating it.

Alternatively, you could possibly search for the autorec. Maybe your dvr/log files are in /var/config/tvheadend/dvr/log?

cd /var/config/tvheadend/dvr/log
sudo grep -l "The Strain" * | sudo xargs grep autorec

You'd then see the autorec ids. These are files in ../autorec. I'd expect them to all be the same autorec.

So, you can then look in the autorec directory to see exactly what is causing the match.

Maybe your dvr/log files are in /var/config/tvheadend/dvr/log?

Well /var/lib/tvheadend/config/dvr/log anyway.

# cd /var/lib/tvheadend/config/dvr/log/
# grep -l "The Strain" * | xargs grep autorec
3940e299b517efaecfe72b0dd96f72ad:    "autorec": "",
6600417b0dd21dd26605632c6e673231:    "autorec": "",

I'm not sure what that tells me.

There seems to be a new problem that may be related.

I had an autorec set up to record the BBC news at 6 o'clock every day, which appears to have stopped working. I just created a replacement by clicking on today's episode at 1745 and then modifying the start after to 1630 and the start before to 2000. It's picking up today's and tomorrow's news but none of the BBC News at Six episodes during the week.

As I understand it, an autorec with a regex of 'BBC News' should match 'BBC News at Six'. So is this another case where TVH is not telling the truth and isn't actually using the regex at all? It's got some hidden series link?

How can I make autorecs that actually use the regex it displays to me to select the programs to record?

edit: My countryfile recording seems to have stopped working too!

If the entries were created by an autorec rule then I'd expect it to have an autorec id inside the double quotation marks, such as

"autorec": "3200417b0dd21dd26605632c6e6732aa"

That autorec id would then match an entry in the autorec directory which specifies details of the recording rule.

Without an autorec id, I'd imagine it was manually scheduled. Perhaps post the contents of the file.

Re: News.
Yes a substring regex will match a longer one. So "BBC News" will match anything that contains that string. If you anchor it with a caret (^) then that string must be at the start, so "^BBC" only matches things that start with "BBC". And if you anchor it at the end with a dollar ($) then it must end with it. So "^BBC$" means it must start with BBC and end with BBC, i.e., the string must be exactly the word "BBC" with nothing else.

I don't know what's wrong with your installation and why so many things are suddenly not recording.

I'd possibly shutdown tvheadend, move the "epgdb.v2" file aside (which contains your whole EPG) such as "mv epgdb.v2 epgdb.v2.old", and restart and rescan for EPG.

If the entries were created by an autorec rule then I'd expect it to have an autorec id inside the double quotation marks, such as

"autorec": "3200417b0dd21dd26605632c6e6732aa"

OK, but I certainly haven't recorded these programmes manually.

Perhaps post the contents of the file.

You mean the log entry/file? Here's 3940e299b517efaecfe72b0dd96f72ad

{
        "enabled": false,
        "start": 1552524900,
        "start_extra": 0,
        "stop": 1552528200,
        "stop_extra": 0,
        "channel": "75d76db8d13d3e041fd2068bf94f65a9",
        "channelname": "Paramount Network",
        "title": {
                "eng": "The Strain" 
        },
        "subtitle": {
                "eng": "Eph and Nora fight to save Captain Redfern's life before coming face-to-face with 
the horrifying effects of the virus. (S1 Ep 3)" 
        },
        "description": {
                "eng": "Eph and Nora fight to save Captain Redfern's life before coming face-to-face with 
the horrifying effects of the virus. (S1 Ep 3)" 
        },
        "pri": 2,
        "retention": 0,
        "removal": 0,
        "playposition": 0,
        "playcount": 0,
        "config_name": "c8748fb66bc1e566107c88bb6167f2a6",
        "creator": "127.0.0.1",
        "errorcode": 204,
        "errors": 0,
        "data_errors": 0,
        "dvb_eid": 0,
        "noresched": false,
        "norerecord": false,
        "fileremoved": 0,
        "autorec": "",
        "timerec": "",
        "parent": "",
        "child": "",
        "content_type": 0,
        "broadcast": 0,
        "comment": "Auto recording: Created from EPG query" 
}

Yes a substring regex will match a longer one

Good; I'm pretty familiar with regex so I'm glad they work as I expect. What I still don't understand is whether they have any effect at all. How do they interact with the serieslink (see code snippet in post #3). I think what I'd like is that the system used my regex to determine what to record and didn't use series links at all. Most important to me is a system that is predictable and since series links are hidden from me, I don't want them influencing behaviour. I think that regex have sufficient power; I don't see what series links bring to the party.

I'd possibly shutdown tvheadend, move the "epgdb.v2" file aside (which contains your whole EPG) such as "mv epgdb.v2 epgdb.v2.old", and restart and rescan for EPG.

I'm happy to restart TVH at a time when it's idle. How do I 'rescan for EPG'? Is that something TVH just does for itself or do I have to do anything?

(For reference, the timestamp for that log is 14th March (couple of weeks ago)).

Yes, I don't like series link for similar reasons, but also because it is more limited than a regex since it records only specific showings such as 9pm and will ignore the 10pm repeat if the 9pm showing fails.

I create my rules manually using autorecs, and manually set the "duplicates" policy to "unique episodes" (in 4.3), but not sure what would work in 4.2 since OTA broadcaster tends to add words like "new" and "hd" which make dup detection difficult, which is partly why SchedulesDirect is good, though I accept many don't like subscriptions.

Tvheadend can scan automatically on startup or by internal cron, but otherwise config->epg->grabber and click "trigger ota epg grabber."

Before shutting down, I'd probably also highlight all autorecs and disable them and then check the upcoming tab, just so you start with a more clean setup.

Well, I restarted and I have four entries for Arrow:

2019-03-30 20:49:24.642 dvr: entry 192b0be50404bf748be49a4926c3d0e7 "Arrow" on "Paramount Network" starting at 2019-04-02 03:34:30, scheduled for recording by "127.0.0.1"

2019-03-30 20:49:24.642 dvr: entry 110015a9986fa140e7b6c918d468af66 "Arrow" on "Paramount Network" starting at 2019-04-02 04:14:30, scheduled for recording by "127.0.0.1"

2019-03-30 20:49:24.642 dvr: entry 25bed9c3b9caea27758f965ab1dd51c3 "Arrow" on "Paramount Network" starting at 2019-04-06 03:34:30, scheduled for recording by "127.0.0.1"

2019-03-30 20:49:24.642 dvr: entry f90709e28c7ea310e2885ab2fa0543d5 "Arrow" on "Paramount Network" starting at 2019-04-06 04:19:30, scheduled for recording by "127.0.0.1"

and no entries for weekday news or for Countryfile. None for The Strain either.

With regards to Arrow and The Strain, I believe the difference is that Arrow was created from the EPG and subsequently edited, so I suspect it has completely ignored my edit and is matching just against the original series link. To me that is clearly a bug and I will raise an issue. The Strain is hand-created and isn't matching. What evidence should I submit?

BBC News and Countyfile I have no idea what is going on

The dvr/log/192b0be50404bf748be49a4926c3d0e7 file should contain an "autorec" entry. This should then match the filename in the dvr/autorec directory. The entry should be the same for your other "Arrow" recordings.

If it is a serieslink, then the autorec file will have "serieslink" in it. It will also have things like "channel", so perhaps your other recordings were against BBC1 but your region has had a recent retune so the actual channel has changed (even though the new channel has the exact same name)?

Where is the actual log stored? i.e. whatever is output in 'Tvheadend log' in the UI?

/var/lib/tvheadend/config # grep -r Arrow dvr

perhaps your other recordings were against BBC1 but your region has had a recent retune

There's been no retune that I'm aware of. And specifically Look East is still recording fine. It is broadcast immediately after BBC News, and before Countryfile. My Humax box is still recording all three just fine, so I think this is a TVH aberration.

Wherever the dvr/log files are (possibly /var/lib/tvheadend/config/dvr/log) you should have files for Arrow called 192b0be50404bf748be49a4926c3d0e7, 110015a9986fa140e7b6c918d468af66, 25bed9c3b9caea27758f965ab1dd51c3, and f90709e28c7ea310e2885ab2fa0543d5 (based on the log you provided earlier).

I don't. That's why I was asking

acer-suse:/var/lib/tvheadend/config # ls dvr/log | grep 192b0be50404bf748be49a4926c3d0e7
acer-suse:/var/lib/tvheadend/config # ls dvr/log | grep 110015a9986fa140e7b6c918d468af66
acer-suse:/var/lib/tvheadend/config # ls dvr/log | grep 25bed9c3b9caea27758f965ab1dd51c3
acer-suse:/var/lib/tvheadend/config # ls dvr/log | grep f90709e28c7ea310e2885ab2fa0543d5
acer-suse:/var/lib/tvheadend/config #

It's a very odd way to keep log information. Particularly since there seems to be nothing that actually records what was actually shown to the user? How does one list the log in time-series order? (in a file so I can grep it etc)

Looking at my news recording setup. I have a hand-crafted autorec:

{
        "enabled": true,
        "title": "BBC News",
        "fulltext": false,
        "channel": "10fcb4e57081bbce451eb03443044f2d",
        "tag": "",
        "btype": 0,
        "content_type": 0,
        "start": "16:30",
        "start_window": "20:00",
        "start_extra": 0,
        "stop_extra": 0,
        "weekdays": [
                1,
                2,
                3,
                4,
                5,
                6,
                7
        ],
        "minduration": 0,
        "maxduration": 0,
        "pri": 2,
        "record": 0,
        "retention": 0,
        "removal": 0,
        "maxcount": 0,
        "maxsched": 0,
        "config_name": "c8748fb66bc1e566107c88bb6167f2a6",
        "brand": "",
        "season": "",
        "serieslink": "crid://fp.bbc.co.uk/b-5ADU8",
        "creator": "127.0.0.1",
        "comment": "6 o'clock news" 
}

It isn't going to record anything except next Sunday at 18:35.

It's a hand-entered autorec with a regex of 'BBC News'. Why is that the only programme it is matching?

That entry has a serieslink in it. I don't know how you got a valid serieslink entry if you manually added the autorec in the autorec tab.

To display a directory in time order add "ls -t" or "ls -rt" for reverse time order.

That entry has a serieslink in it. I don't know how you got a valid serieslink entry if you manually added the autorec in the autorec tab.

You're right.I think I must be confused. I'm giving up; the whole subject/interface is just too complicated. I'll just poke about pushing buttons until it does what I want and accept that I don't really control it and need to watch it carefully just like my Humax.

I think the problems with the BBC are due to the BBC retitling and presumably relinking their programmes. The news at the weekend no longer matches a regex of BBC News for example.

To display a directory in time order add "ls -t" or "ls -rt" for reverse time order.

Thanks for that. I hadn't come across the -t option before. That only allows to list the DVR log entries though. Is there nowhere to see all the log entries?

I think by "see all the log entries" you mean to see the contents of the files?

If so, the easiest way is to "ls -t | xargs less" (use ":n" to go to the next file, ":p" to go to previous file). The command "xargs" take its input from stdin (the pipe) and puts that content on the end of the command. So if its input is "a", "b", "c", then it effectively runs "less a b c".

An alternative, and perhaps more readable, way is to use backquotes (the key next to the 1 key on a normal UK keyboard). So "less `ls -t`", though you have to be a bit more careful since it would run the "less" command even if the "ls" command returned no output, and some commands would then read from stdin instead. (Young people prefer "less $(ls -t)" as equivalent to backquotes).

So, if you wanted to grep perhaps you could use "ls -t | xargs grep -e Arrow -e Strain". And if you wanted to view those files you could "ls -t | xargs grep -e Arrow -e Strain | awk -F: '{print $1}' | xargs less". You could replace "xargs less" with "xargs cat | less" if you don't want to just view all the files in one long list.

The autorecs will work properly, but I think series link is a bit complicated and, as you say, BBC weekend news probably has a different series link to BBC weekday news, even though they have the same title.

I always manually create my autorecs on the autorec tab. It's a bit more tedious than doing it from the EPG tab, but it does mean you have better control.

I think by "see all the log entries" you mean to see the contents of the files?

No, I mean see all the log entries. All the lines that appear in the box at the bottom of the UI entitled 'Tvheadend log'.

BBC weekend news probably has a different series link to BBC weekday news, even though they have the same title.

They do now, but they didn't use to. They have different titles as well, as I say.

I always manually create my autorecs on the autorec tab. It's a bit more tedious than doing it from the EPG tab, but it does mean you have better control.

I agree, but it doesn't need to be like that. But as I say, I've given up.

edit: PS Thanks for all your help

FWIW, the BBC news has now broken recording on my Humax box as well. It seems the lunatics have taken over the asylum. By which I mean the marketdroids have apparently told the technical people to do something stupid. I'm hopeful it is now so broken that they'll have to redesign it before long.

I tried it with SchedulesDirect (costs around 5p/day, 19.00pa) and it works as I'd expect for BBC News at Six. I know you said earlier that you don't like subscription services, but it might be something to consider if you do want consistency since SchedulesDirect gives each programme a "series id" and an "episode id" so it can identify programmes. They give a free 7 day trial to see if it works for you. I think it provides most FTA channels except for a few timeshift channels. (I get no money at all if you sign up and have no involvement with the company).

The normal grabber to use would be "tv_grab_zz_sdjson_sqlite", though I have a variant (replace "zz" with "az") which is more difficult to use but provides a few more esoteric features.