Tvheadend

Hi everyone. I am the developer for the zap2it to TVHeadend to Kodi script listed on the top of this topic. I personally like the concept to use Kodi as the cronjob to update tvheadend EPG. It would make the updates OS independent. But, to have it work will require constant updates to keep up with zap2it website, unless the developer somehow integrates the perl script into the python script. That is why I use the perl script from http://zap2xml.awardspace.info/
That site will actively maintain the core script needed to extract the data as zap2it changes. As a note, zap2it changed their website on Jan 4th 2018 and on Jan 8th 2018, the perl script was updated and available. Now, I do realize my solution is for Linux OS although it should also work within cygwin under windows and is fully documented at

https://github.com/rocky4546/script.xmltv.tvheadend/wiki/Guide:-How-to-Setup-XMLTV-for-TVHeadEnd

I just maintain the wrapper scripts which rarely change. Good luck getting it to work and be maintained. It is a great concept.

Or if you switch to the zap2epg kodi addon - you can get all sorts of options for your grabber. It uses the same zap2it data, but is configured from within Kodi. It comes with its own grabber that self configures your channels if your running a recent tvh build.

Here's the release: https://github.com/edit4ever/script.module.zap2epg/releases

Discussion/help here: https://forum.kodi.tv/showthread.php?tid=267022&pid=2689449#pid2689449

Thanks!

• drac2000 * wrote:

Now that zap2xml is not an option anymore

zap2xml was only "not an option" for about 4-5 days. The developer of that program fixed it to work with the changes on the Zap2it site. Do read the release notes for the latest versions, and pay particular attention to the (otherwise undocumented) -8 and -9 options (only use -8 if you have "starred" your favorite channels). The developer says the -9 option is "not recommended" but I think he doesn't fully understands that without it, changes in channel numbers on a listings site could make the guide data stop working in Tvheadend. He was trying to solve one problem (duplicate listings) without understanding that he was creating another (listings that would disappear if a station or cable/satellite service changes channel numbers). At least he did provide the -9 option the next day, I just suspect he still doesn't get why people wouldn't want channel numbers in their channel ID's.

One big remaining problem with the new Zap2it site is that favorites don't work as they did before. In the past, you could add favorite channels from multiple geographic areas to your favorites list, but you can't do that anymore. Well, strictly speaking that's not true, you can still do that, but Zap2it will only display channels from whatever geographic area you last specified, and if you have "starred" favorite channels and click the button to show those, it will only show the "starred" channels from whatever geographic area you are currently viewing the listings for. In the revised zap2xml, you can use the -8 option to select only your "starred" favorites, but it will still only show the favorites from the current geographic area. And also, many cable and satellite services will put the same channel on multiple channels numbers, so if for example the same channel appears as channel 200 and 1200, if you use the -9 option WITHOUT the -8 option you will get the listings for both channels even though they are duplicates. If you favorite (star) one of the channels (doesn't matter which) and use both the -8 and -9 options, you will get the first one (channel 200 in this example) even though you may have "starred" channel 1200, but you won't get duplicate listings.

As for edit4ever's software, I just wish it didn't primarily depend on a Kodi addon for configuration, since some of us don't run Kodi on our Tvheadend backends. Sure, you can muck around in a configuration file and maybe make it work the way you want, but it wasn't designed for that use and AFAIK there is no unified instruction document (outside of posts in this thread) explaining all the options. I appreciate his efforts, and it's always good to have multiple options, but I already know how to use zap2xml so making the leap to the new version didn't take as much mental effort on my part. That said, if edit4ever ever publishes a single guide to setting up his software (without using Kodi) or even just a list of all the configuration options, I may take another look at it. Or if he comes up with a configuration utility that will work on a server (doesn't require the presence of a desktop, because Tvheadend doesn't) then I will definitely take another look. On one level I'd much rather run something written in python than perl, because perl programmers tend to use language quirks unique to perl that makes their programs nearly impossible to understand by anyone not well-versed in perl, whereas python is much more readable - even if you don't know enough to write python code, you can usually figure out what a python script is doing, especially if the programmer left any comments in the code. But from a user's standpoint, I don't care what language it is written in as long as it works, therefore for now my tendency is to stick with the software I already know.

• drac2000 * wrote:

Actually I was about to update my post above. In fact, I am lucky enough to catch few more channels than what tvlistings shows up for my zipcode. I used another postal code to pull those channels and that did it for me.

I have found that the zip code you use can make a big difference in the channel lineup presented, especially if you receive signals from multiple TV markets. If you use Google Maps and look for a location almost equidistant between two cities, even if it's the proverbial one-horse town, you may get channels from both markets, or if you pick a point in the center of a triangle formed by three markets you may get listings for all three! And if you don't get what you want, moving to another zip code five or ten miles closer to the "missing" channels may get you those.

On the other hand, if you are in a place that is served by translator stations, I have found that the listings for translators aren't always as accurate as the listings for the station that the translators are repeating. So if you are only shown the translator stations, and find the listings are screwed up, you may want to move a little closer to town (in zip code only) to get the listings for the main stations.

Finally, I have found that overall, Zap2it has more accurate listings than TV Guide, but that's not universally true. For example, compare the national listings of PBS East and West on Zap2it and TV Guide, particularly on Saturday afternoon during the "This Old House" block. In this case, TV Guide is the one that currently has the correct listings. But for all the other national listings I looked at, Zap2it seemed more accurate.

K Shea wrote:

• drac2000 * wrote:

Now that zap2xml is not an option anymore

zap2xml was only "not an option" for about 4-5 days. ...

K shea, I think you got it wrong. It was not option for "me". I used zap2xml for few years now, and I enjoyed a lot (on different OS including windows) and all credits are for its author.
Now for my needs (Kodi on whatever OS you have), I found that zap2epg (even if it's still in dev) is quite simpler --in my opinion-- and fixes an issue with encoding (special chars in EPG descriptions).

• drac2000 * wrote:

K Shea wrote:

• drac2000 * wrote:

Now that zap2xml is not an option anymore

zap2xml was only "not an option" for about 4-5 days. ...

K shea, I think you got it wrong. It was not option for "me". I used zap2xml for few years now, and I enjoyed a lot (on different OS including windows) and all credits are for its author.
Now for my needs (Kodi on whatever OS you have), I found that zap2epg (even if it's still in dev) is quite simpler --in my opinion-- and fixes an issue with encoding (special chars in EPG descriptions).

Well you are certainly entitled to your opinion, and to use the software that you prefer. All else being relatively equal, I prefer to stick with the software I already know how to use, but maybe you are the type of person that likes to experiment with the latest software, and that's fine too. Anyway, it's good that there are multiple options!

I personally have never noticed an issue with special characters in EPG descriptions, maybe that's what zap2xml's "-e = encode entities (html special characters like accents)" option is for?

K Shea wrote:

I personally have never noticed an issue with special characters in EPG descriptions, maybe that's what zap2xml's "-e = encode entities (html special characters like accents)" option is for?

You are right, I've seen that switch and I used it but I think it was fixing the titles but not the descriptions or details. So maybe I didn't got it well (I used it in a windows context in a long time). But just make my point, I've a lot of respect to all the community and the specially open-source project authors. Thanks to all of them.

drac2000

FYI - ​I have pushed the 0.7.1 update to zap2epg that deals with an error in the episode id number.
This error was causing tvheadend to record all shows on a channel when setting a series recording.

Please update to fix this issue:

https://github.com/edit4ever/script.module.zap2epg/releases

Thanks!​

For those interested - I have posted a basic setup guide wiki as well as information on manually creating/editing the settings.xml file. This is useful for those that just want to run zap2epg.py without running it as a kodi addon.

https://github.com/edit4ever/script.module.zap2epg/wiki

Please let me know if any information needs more detail.

Thanks!

edit4ever ! wrote:

For those interested - I have posted a basic setup guide wiki as well as information on manually creating/editing the settings.xml file. This is useful for those that just want to run zap2epg.py without running it as a kodi addon.

Huh? Doesn't look like it...

https://github.com/edit4ever/script.module.zap2epg/wiki

Please let me know if any information needs more detail.

From the first paragraph: "Configuration is done through the addon in kodi..." and I see nothing that shows how to use it without running it as a kodi addon. What's needed is a non-GUI utility or script that can be run on a server.

K Shea wrote:

Huh? Doesn't look like it...

Please keep in mind...my intention in creating the zap2epg addon was to simplify the setup process for those who are not technically inclined (and new to kodi pvr setups).

That being said, since some people started to ask for a way to use the script outside of a kodi/tvheadend setup, I wanted to let everyone know that I designed the zap2epg.py script to be run on it's own without kodi. However - it does require the settings.xml file for configuration.

Since someone asked how the settings.xml file works I have now put that information in the wiki under the settings.xml section. https://github.com/edit4ever/script.module.zap2epg/wiki/6.-Settings.xml

Basically - just copy the zap2epg.py file to whatever system/directory you want. Create/Copy the settings.xml to the same directory. Configure the settings.xml the way you want based on the new wiki information. And now run the zap2epg.py script. If it generates an xmltv.xml file - bingo! Now just setup a cron to run it once or twice a day and you're good to go.

edit4ever ! wrote:

Since someone asked how the settings.xml file works I have now put that information in the wiki under the settings.xml section. https://github.com/edit4ever/script.module.zap2epg/wiki/6.-Settings.xml

You didn't link to that specific page in your previous post, so that's why I didn't see it. After looking it over, I don't personally see anything that at the present time would induce me to switch from zap2xml, if only because I'm already familiar with that software. The two limitations I see are that it's not real clear how the "desc01", etc. settings are used, but the bigger problem is that if you can receive channels from more than one TV market there is no easy way to download or combine listings from a second area. Your program expects there to be one configuration file, and in that file it appears you can only specify a single lineup. For some people (those that can only receive channels from a single market) this will not be an issue, but for others it's a fairly major limitation.

I do realize that you could probably have multiple copies of your program, each with its own configuration file, in different directories on your system, but that's still not exactly an easy thing to deal with. In contrast, with zaptoxml there are options to specify filenames and you actually don't even need a configuration file, since all options can be specified on the command line, and it does have an option to combine xml files. So, it is much easier to run it more than once to pick up additional lineups.

Different people have different preferences in software and I'm sure yours will appeal to some people, in fact it might be much more appealing to me if I only received channels from a single market area, if only because of my slight dislike of Perl scripts. If you can ever get it to download and combine listings from multiple lineups, then I think it will be much more useful to those of us in those "between market" areas.

K Shea wrote:

Different people have different preferences in software

Agreed. My main focus was to get noobs up and running with a guide in a simple way - especially for those using setups like LibreELEC. I am happy to contunue looking into making it easier/better to use for those with more experience in setting their systems up manually.

Multiple lineups is something I can investigate. Just as an FYI, I have found that if you choose a different zipcode in the "in between" areas, you often can get a lineup that includes both markets. For example, here in San Diego, many but not all of the zipcodes include the LA OTA stations as well as the San Diego stations. So one can check on zap2it to see if there is a zipcode that will pull both markets.

I can also look at setting up either command line arguments for the options - but it would be messy to handle the DESC01 type options that append information to the plot description. Maybe I can make a command line based setup interface if that would be desired.

edit4ever ! wrote:

I can also look at setting up either command line arguments for the options - but it would be messy to handle the DESC01 type options that append information to the plot description. Maybe I can make a command line based setup interface if that would be desired.

Even if you could just do something as simple as

zap2epg.py settings1.xml
zap2epg.py settings2.xml
zap2epg.py settings3.xml
...

That would make it easier, although you'd still need a way to combine the resulting xml listings files, but that can be easily done using sed if I recall correctly (I just saw something about that on a page the other day). That would need to be done between runs of your program, so that subsequent runs after the first don't overwrite previous output files.

Or better yet

zap2epg.py settings1.xml settings2.xml settings3.xml ...

With automatic combination of the resulting output xml files. That way you could keep all the existing options (though I still don't quite understand the need for the DESC01 options) and you wouldn't need multiple directories or multiple instances of your program.

As for the using the zipcode from an "in between" area trick, I know that will work in many areas, but where it kind of falls apart is for free-to-air satellite viewers that can receive channels from multiple non-contiguous areas. It's a bit hard to explain if you aren't into free-to-air satellite, and not many people are, but often you can receive channels that would be offered on a subchannel (one of the .2, .3, .4 etc channels) in some markets, but maybe not in yours. Then, to get listings you need to find a station that actually carries that channel, even if it's nowhere near you, and associate the schedule for that channel with the satellite feed. Or maybe you get both east and west coast feeds of a channel such as public broadcasting, in that case you may want to incorporate the listings for that channel from the generic eastern and pacific time zone listings. I would not spend a lot of time worrying about this because you probably could count the number of free-to-air viewers that use Tvheadend and that read this forum on both hands (maybe even just one hand), but just wanted to explain that there can be situations where someone might want listings from multiple areas that are not contiguous.

K Shea wrote:

...but just wanted to explain that there can be situations where someone might want listings from multiple areas that are not contiguous.

Thanks for clarifying. I actually have an idea for how to do this - but it will be easiest to setup in the kodi interface to make sure that the channel list gets configured, in order to limit the number of downloads.

In your case, do you use kodi at all on any system? If so, I can build the interface for configuration - and then the zap2epg.py and settings.xml file can be copied to any other system you want to run the script on.

edit4ever ! wrote:

K Shea wrote:

...but just wanted to explain that there can be situations where someone might want listings from multiple areas that are not contiguous.

Thanks for clarifying. I actually have an idea for how to do this - but it will be easiest to setup in the kodi interface to make sure that the channel list gets configured, in order to limit the number of downloads.

In your case, do you use kodi at all on any system? If so, I can build the interface for configuration - and then the zap2epg.py and settings.xml file can be copied to any other system you want to run the script on.

I actually do run Kodi, but not on the same system as my Tvheadend backend. And right now I'm happy with the way zap2xml is working, and I don't want to go through the hassle of using a Kodi addon that's not in their "approved" official repository. My strong preference is to let Kodi be Kodi, and let Tvheadend be Tvheadend, and not try to mix the two except via the official Tvheadend PVR addon for Kodi. So if you want to try and add that functionality using the Kodi interface that's fine, and it may help some people, but I personally would not be likely to use it as long as it's not in the official Kodi addon repo. Also, I am still running Kodi Isengard, because newer versions of Kodi have broken some functionality, so chances are that if your addon expects a newer version of Kodi it wouldn't run on my Kodi installation anyway.

As a side note, I realize that at some point I will probably be forced to upgrade Kodi and I really dread that day, just as I dread most major software upgrades, because nowadays they tend to break more things than they fix (Tvheadend being an exception to this; I don't think I've ever yet done a Tvheadend upgrade where the newer version hasn't been better than the one it replaced; even so I seldom upgrade once I have a working Tvheaded installation). But it's definitely not going to happen until after Ubuntu 18.04 has been out for a while, at which point I will probably start over with a clean install of Ubuntu and then install Kodi. But given my aversion to upgrades to working software, and because I have specifically read that newer versions of Kodi have some issues with passthrough audio, it's likely I won't upgrade until just before the long term support period for Ubuntu 14.04 ends next year.

I am getting the following error in the log file for zap2epg.py:

2018/01/28 15:40:35 Exception: main
Traceback (most recent call last):
File "zap2epg.py", line 718, in mainRun
tvhMatchGet()
File "zap2epg.py", line 115, in tvhMatchGet
response = urllib2.urlopen(channels_url)
File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 447, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1228, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 111] Connection refused>

Never mind had to disable tvheadend stuff.

edit4ever Hi...

I really appreciate your kodi addon " zap2epg" But i'm trying to install it on a Odroid-HC2 NAS with openmediavault...or dietpi......my zap2epg work very nicely on

libreelec.....But kodi and me...it's over.....i prefer plex....on docker via my nas....with tvheadend as pvr....lol

I finally install openmediavault....on it....It's a nice little NAS server....

But for the benefit of all.....would it be possible to create a "Docker" for "zap2epg" or having an integrated tvheadend's docker with zap2epg included.... or a

zap2epg for Plex's docker....

I know i ask too much...lol

But the idea is to replace my old Windows Media Center who is eating tons of electrons......lol

My Droid HC2 run on 12 volts with 6 TB RED WD

http://www.hardkernel.com/main/_Files/prdt/2018/201801/201801130635362637.jpg

just like that....for more infos...

http://www.hardkernel.com/main/products/prdt_info.php?g_code=G151505170472

https://www.zdnet.com/article/what-is-docker-and-why-is-it-so-darn-popular/

https://www.docker.com/

Or another possibility is to add or integrating zap2epg grabber to tvheaden's grabber software all this without kodi........

Thanks !!

Dan

Hi edit4ever!

I have been using your program nicely with a couple lineups (in separate directories) with only 5-10 channels. I recently switched my main lineup over to zap2epg, which has 85 channels. In the slist, i have those channel IDs. However, no matter what i do, it only outputs 5 channels to the xmltv file. Would you have any idea why this happens?

Thanks

Jim Sanders dods - happy to try to help...would you be able to provide the zap2peg.log file?

Thanks. I have been playing around with it. It seems to be only certain channels. For my latest one, I have about 70 channels, but only 32 write to XMLTV (those with actual channels number <100). Thanks'

I should add, running from the Kodi addon works for those channels, like Show 11115, but I run it via command line on RasPi server and it does pull it in.

Thanks for the log - it will definitely help. Looks like there are some errors with individual show details - and it seems there is a naming bug in the code that is showing up for those...which I'll have to dig into.

It is only showing 32 channels - so to dig into that - can you provide the channels.json file? I want to compare the channel info and how the system is pulling.

It will probably be another day or so until I can fully dig in - but thanks for providing the info on the issue!

Thank you. Which .json file to provide? there are so many, and not sure how to tell which is for which channel.

Sorry - in your script.module.zap2epg userdata folder - you will find the channels.json and TVHchannels.json files. Those are the two I am looking for.

Thanks!