Tvheadend

Yes please, I'd be pleased to try it. Please could you also provide instructions on how to install it, targeted to a person with mediocre tech skills please. Thanks, Peter

Gee that was quick!

Installation should be very straightforward .. happy to discuss and walk you through it, but how to communicate initially?

Not keen on leaving email details in public but I may have sorted something - does 0632 mean anything to you?

Ivan, thanks for posting your solution.

Perhaps you could assist me with a project that I’m working on regarding EPG data.

I’m in Sydney, so I get my EPG the straight from EIT. However, I have been doing some work on getting rating labels (‘PG’, etc) to display with TVH.

https://tvheadend.org/boards/5/topics/50356

The next step that I am working on is for TVH to be able to automatically detect as many parental rating systems globally and seamlessly fill in the rating label for the user thus avoiding a manual setup step.

If you are willing, could you please post a zipped copy of your log file for the EIT now+next as well as the kind of parental rating data that you get from your HMEG-5 feed?

https://tvheadend.org/boards/5/topics/50501

Also, if you have access to the full Auckland EIT EPG then a log of that would also be greatly appreciated.

If you are also willing to share the source code for your module, I could also look at how it could be incorporated with rating labels.

@PeteHodd - I'd be interested in a log file from you too if you are willing.

Yes 0632 is my postcode

I M wrote:

Installation should be very straightforward .. happy to discuss and walk you through it, but how to communicate initially?

You could just communicate via the forum. I'd be happy to eavesdrop on the process if you don't mind.

Ok, so how from here?

DMC, happy to help where I can but I'm no guru with this.

Also, I don't have any EIT data at the moment as I'm all mheg-5, and I'm not in Auckland! That said I am parsing the mheg data to extract episode detail from the description text which is then included in the resultant XML import into TVH. This appears to be working well and no doubt something similar could be done to extract the rating detail.

I can zip up a download of the raw stream data, or the XML output as you wish, but I suspect the following sites would be just as helpful:

https://freeviewnz.tv/parental-control/
https://safeviewing.co.nz/classification/

And here's an example of the XML I'm outputting and how the detail is presently in the description field (as broadcast by the provider):


<programme channel="eden.mheg5" start="20231128123500 +1300" stop="20231128133000 +1300">
<title>Paramedics: On The Front Line</title>
<desc>PG-LC The West Midlands Ambulance Service deal with a 19-year-old man who has had a motorbike crash, and an elderly woman showing signs of a stroke but refusing treatment. S3 E2</desc>
<episode-num system="dd_progid">T90392895</episode-num>
<episode-num system="onscreen">S3 E2</episode-num>
<video>
<present>yes</present>
<quality>HDTV</quality>
</video>
<rating system="Freeview">
<value>G</value>
</rating>
</programme>
<programme channel="eden.mheg5" start="20231128133000 +1300" stop="20231128142500 +1300">


Having had a quick look at other programmes it would appear that if something like "16-VLC" is displayed in the description text, it's not in the 'rating system' data. I'd need to more closely inspect the raw data before determining it that's an issue with the parsing and XML output, or if it's just not there to start with. Of course the EIT data may be quite different for all I know.

WRT to my source, at this time I've not included rating labels (at least not in a way that TVH understands) as it's not really of concern to me, but you're welcome to a copy if you think it might be useful.

Pete Hodd wrote:

Ok, so how from here?

I just tried telephoning the number I'd found for you but it's 'no longer in service' .. if you wanted to use that number with some addition/subtraction to give me another number that would probably work? Just let me know if you're including the area code too

Delta Mike Charlie wrote:

I M wrote:

Installation should be very straightforward .. happy to discuss and walk you through it, but how to communicate initially?

You could just communicate via the forum. I'd be happy to eavesdrop on the process if you don't mind.

That's no problem, but I need to get the code to Pete for a start.

It will be interesting to see how he goes as I've not had anyone else test it. In theory it's just a matter of dropping it into /bin/, restarting TVH and selecting the new internal XMLTV grabber that should appear. I guess we'll see if I'm being overly optimistic!

I M wrote:

DMC, happy to help where I can but I'm no guru with this.

Thanks very much, any information that you can provide will be helpful.

Also, I don't have any EIT data at the moment as I'm all mheg-5, and I'm not in Auckland! That said I am parsing the mheg data to extract episode detail from the description text which is then included in the resultant XML import into TVH. This appears to be working well and no doubt something similar could be done to extract the rating detail.

I can zip up a download of the raw stream data, or the XML output as you wish, but I suspect the following sites would be just as helpful:

Actually, both would be great.

<rating system="Freeview">
<value>G</value>
</rating>

Would you be willing to change your system name from "Freeview" to "Freeview-NZ" or even "FVLB"? There are multiple regions that use the "Freeview" name.

Having had a quick look at other programmes it would appear that if something like "16-VLC" is displayed in the description text, it's not in the 'rating system' data. I'd need to more closely inspect the raw data before determining it that's an issue with the parsing and XML output, or if it's just not there to start with. Of course the EIT data may be quite different for all I know.

The "16-VLC" just means 'restricted to persons 16 years and over' with 'violence may offend', 'language may offend' and 'content may offend'.

The EIT data has a country code, in your case NZL, and then a number representing the age. The receiver then uses the age to display a label. This is why I was keen to see what the now/next looks like.

WRT to my source, at this time I've not included rating labels (at least not in a way that TVH understands) as it's not really of concern to me, but you're welcome to a copy if you think it might be useful.

Thanks. I do not plan to make any changes, however, I can provide feedback should you wish to incorporate rating labels.

I M wrote:

That's no problem, but I need to get the code to Pete for a start.

Naturally. Get Pete working first and get back to me when you have the time. No rush.

It will be interesting to see how he goes as I've not had anyone else test it. In theory it's just a matter of dropping it into /bin/, restarting TVH and selecting the new internal XMLTV grabber that should appear. I guess we'll see if I'm being overly optimistic!

What language did you write your module in?

Hmmmm, not sure what phone number is out in the ether for me. It may be related to a previous employer, or a home landline before I went all mobile. Could you provide the first 3 digits of the number, excluding the area or mobile prefix?

is the native wood pigeon in the address you've found?

Pete Hodd wrote:

is the native wood pigeon in the address you've found?

Yes, and 413 to your other question.

prefix old telecom for its cdma network
taking fax number
digit 1 add 4
digit 2 subtract 3
digit 3 add 1
digit 4 subtract 2
digit 5 subtract 3
digit 6 add 3
digit 7 subtract 3

you'll find me on whatsapp if you wish to message

Delta Mike Charlie wrote:

I can zip up a download of the raw stream data, or the XML output as you wish, but I suspect the following sites would be just as helpful:

Actually, both would be great.

Ok, both attached now.

Would you be willing to change your system name from "Freeview" to "Freeview-NZ" or even "FVLB"? There are multiple regions that use the "Freeview" name.

Sure, I really don't mind, it's very easy to change.

The "16-VLC" just means 'restricted to persons 16 years and over' with 'violence may offend', 'language may offend' and 'content may offend'.

Quite, I was just trying to explain the present behaviour of the code...

The EIT data has a country code, in your case NZL, and then a number representing the age. The receiver then uses the age to display a label. This is why I was keen to see what the now/next looks like.

It may be that Pete is able to output this for you, if that's what he's using presently. I'm loath to mess with my setup too much while I'm still experimenting with this code. Maybe later if no-one else is able to help.

Delta Mike Charlie wrote:

What language did you write your module in?

It's in Python.

Pete Hodd wrote:

prefix old telecom for its cdma network
taking fax number
digit 1 add 4
digit 2 subtract 3
digit 3 add 1
digit 4 subtract 2
digit 5 subtract 3
digit 6 add 3
digit 7 subtract 3

you'll find me on whatsapp if you wish to message

Sorry, don't do any social at all. I've tried the resultant but the 'ph system doesn't like it.

FYI I have two possible numbers for the base, first ends in 4, second in 3, have tried [converting] both. I get a negative number for digit 2 and 5 but have just assumed use of the resultant numeral? Not using the area code to start.

Seems you've found a different listing.

Based on phone number
+1
-1
+4
-4
+3
-5

Are you ok with my cryptic reference to the 'area' code?

I M wrote:

Ok, both attached now.

Thanks for the data, here are some observations:

You can probably scrape some more information if you want to.

Look at the PNG names: They can tell you the raw parental rating. I’m guessing that you are already using this for HD and if it has hearing-impaired options.

The augmented parental rating is prepended to the programme description in for format of <rating><dash><augmentation>.

You can’t just scrape out the first word from the description because a programme without a rating just has the descriptions. There would have to be a test for valid rating prefixes before committing to extracting the rating.

Valid ratings from your sample set seem to be:

G, G-CV
PG, PG-C, PG-CL, PG-L, PG-LC, PG-LS, PG-LSC, PG-S, PG-SC, PG-V, PG-VC, PG-VL, PG-VLC
16, 16-C, 16-CV, 16-L, 16-LC, 16-LS, 16-LSC, 16-VC, 16-VL, 16-VLC, 16-VLS, 16-VLSC
18, 18-VLC
M, M-C, M-CV, M-L, M-LC, M-LS, M-LSC, M-SC, M-V, M-VC, M-VL, M-VLC, M-VLS, M-VLSC

With the base ratings being: G, PG, M, 13, 16, 18, and advisory comments being C, L, S, V.

Perhaps you could look for <base><space> or <base><dash><something><space>

I’d be happy to do some python code for you if that would help.

Would you be willing to change your system name from "Freeview" to "Freeview-NZ" or even "FVLB"? There are multiple regions that use the "Freeview" name.

Sure, I really don't mind, it's very easy to change.

Thanks, I think that "FVLB" would be best.

The EIT data has a country code, in your case NZL, and then a number representing the age. The receiver then uses the age to display a label. This is why I was keen to see what the now/next looks like.

It may be that Pete is able to output this for you, if that's what he's using presently. I'm loath to mess with my setup too much while I'm still experimenting with this code. Maybe later if no-one else is able to help.

@PeteHodd - Once you are done, if you are willing to help I would be grateful.

Delta Mike Charlie wrote:

I’d be happy to do some python code for you if that would help.

Here's a RegeX that I came up with to split the rating from the rest of the description.

((^G|^PG|^16|^M|^18)( |-)([CSLV ]*))(.*)

I may need some fine-tuning. If there is no match then there is no rating and the description remains unchanged.

Pete Hodd wrote:

Are you ok with my cryptic reference to the 'area' code?

Yep, fine with the first three digit code, but in the first try you gave me seven numbers, this is only six ..

I tried with a 4th digit on the area code and got someone but it wasn't you. I'm sure we'll get there, but if maybe you could just use the number ending in 6, and give me all 7 digits? Or we could use a tempmail address or something like that.

Interesting problem I've not come across recently, most other forums have a PM facility of some sort, but not this one it seems...

Update: For those following this saga we've establish contact. Hope to report back on results in the next few days.

Delta Mike Charlie wrote:

Delta Mike Charlie wrote:

I’d be happy to do some python code for you if that would help.

Here's a RegeX that I came up with to split the rating from the rest of the description.

((^G|^PG|^16|^M|^18)( |-)([CSLV ]*))(.*)

I may need some fine-tuning. If there is no match then there is no rating and the description remains unchanged.

Thanks, will check it out when I'm back in the office. Appreciate the input as it's not something I'd get around to myself.

Re the naming, will probably just add -NZ to existing as it's more naturally descriptive for anyone else coming along.

I M wrote:

Thanks, will check it out when I'm back in the office. Appreciate the input as it's not something I'd get around to myself.

Sorry, I got a bit carried away:

import re

def parseRating(description):

    pattern = '((^G|^PG|^16|^M|^18)( |-)([CSLV ]*))(.*)|.*'
    returned = {}

    result = re.split(pattern, description)

    if(result[2] is None):
        returned['short_rating'] = "" 
    else:
        returned['short_rating'] = result[2]

    if(result[1] is None):
        returned['full_rating'] = "" 
    else:
        returned['full_rating'] = result[1].strip()

    if(result[5] is None):
        returned['desc'] = description
    else:
        returned['desc'] = result[5]

    return returned

#Test various inputs.
ret = parseRating("PG one text here")
print(ret)
#{'short_rating': 'PG', 'full_rating': 'PG', 'desc': 'one text here'}

ret = parseRating("M-LSC two text here")
print(ret)
#{'short_rating': 'M', 'full_rating': 'M-LSC', 'desc': 'two text here'}

ret = parseRating("three text here")
print(ret)
#{'short_rating': '', 'full_rating': '', 'desc': 'three text here'}

Delta Mike Charlie wrote:

Sorry, I got a bit carried away:

[...]

Heh, just a little, but thanks for taking the time

There is already a dictionary and some parsing for ratings however I think before going too much further it'd be good to first see how the module works for Pete as it is. It may be some work is needed as a result of that test but if it looks promising and is integrating ok then will see how best to enhance things further.

I'm not near the code at present but imagine what you've done would fit in well and improve the detail that's presently available. I'd need to re-enable the column in the EPG tab to test (removed it in the js code as I wanted a very simple layout), but can prob look at that next week.