Tvheadend

To Clarify how 123TV works with Cabernet... 123TV uses multiple providers, some are P2P based. This is all behind the 123TV website. They group channels based on their providers. Some providers are working fine while others are no longer working. Cabernet can tell if the stream is really down based on how it fails, so when a stream is requested, Cabernet will disable the channel, if it determines the channel is one of those that is not expected to ever be up. The important thing to note is to get the channel list, then the EPG, and then run a scan on all the channels to have Cabernet determine which channels are an issue and place them as disabled.

Once the bad channels are disabled, TVH will receive only the list of channels that are not disabled. This is about half the channels on 123TV. You can re-enable the channels in Cabernet and rerun a scan to see if anything has changed. Cabernet will change them back to disabled if again the channel is offline. Cabernet keeps track of each channel status to determine if a channel was ever successful. Once a channel is successful, Cabernet will not disable it even if the channel is not working. Of course, you can still change a channel to disable as needed. I am thinking about adding a schedule event to run a internal scan weekly on 123TV disabled channels in case something changed. It could also be used to run the initial scan instead of using VLC or TVH to run it.

0.9.8.1 added channel scanning scheduled task
This is helpful when trying to initialize what is up and down for 123TV.
Plan to create the daddy live plugin next. Issue tracking that plugin is here
https://github.com/cabernetwork/cabernet/issues/36

0.9.8.2 Added DaddyLive Plugin
- EPG is not provided at this time due to the website not providing any epg data
- Easy way to review the streams are to use VLC. Use the network address : http://ipaddress:6077/daddylive/channel.m3u
- All channels listed seem to be working; however, DaddyLive is going through growing pains (possibly due to USTVGO shutdown). I have seen 404 and timeout errors in the evening on all channels. Cabernet updates have reduced the issue, but it was still occurring. Expect they will be making performance/hardware updates to correct the issue.
- Will be looking into getting EPG data remotely, but wanted to get out the streams since they are working well with much better resolutions.

Small bug found, hold doing the upgrade until a patch is provided...

0.9.8.5 adds a stream fix for the 2 channels that have malformed m3u8 files.
my stats are
- 249 HD channels worked per TVH
- 7 channels failed (may work some time in the future, but down when TVH scanned them)
- The rest are SD channels that are also good per TVH
- Good channels did appear in Kodi as well as the DaddyLive tag
Note: HD means at least a 720 resolution

I upgraded today but still have two questions:

1. I changed the PlutoTV Starting Ch Number from 1 to 2501 but even after rescanning, and then restarting Cabernet the channel numbers don't change.

2. Cabernet shows 123TV is enabled but no channels appear. Do I have to add an instance to the config file, and if so do you have a suggested configuration or any suggestions for settings?

For number 1, I find it an issue with TVH. I normally need to delete the mux and tvguide data. Make sure the channel list for the network is empty, then try to do a scan again. Before that, you can check Cabernet by requesting a channel list like http://ipaddress/PlutoTV/lineup.json and see that the guidenumber is updated.

Yes, both the plugin and its instance need to be enabled for Cabernet to get the channel list. The release notes normally have the configuration. Look like I removed the old releases during releases cleanup (oops). It is the standard way to add an instance.

[123tv_default]
label = 123TV Instance

I also have a technical question for you that's only tangentially related to Cabernet. When recording or streaming from a source, there are occasions where they "go dark" for a time, such as during extended commercial breaks, etc. Tvheadend will by default time out after 20 seconds, and while you can override that in the Tvheadend network settings, that only seems to apply to recordings, not live TV. So how do you handle that? Do you send "dummy" video to Tvheadend to keep it from timing out? And if so, is the code to do that part of Cabernet or do you use some external program that filters the stream and drops in "dummy" video of some kind when the desired stream is missing, just to keep Tvheadend alive?

I ask because I have encountered this issue on local devices that take more than 20 seconds to start outputting a stream (the issue is that when TVH first starts the stream it also starts a script to turn the device on, and that device takes more than 20 seconds to start sending video), and the Tvheadend developers don't seem to care that the timeout override in the network settings does not seem to apply to live streams (at least not on the version of TVH being used). But also TVH is working fine otherwise except for that one thing, so I just wondered if you might know of something that would fix this problem. The way I envision it, this piece of software would start supplying a stream to Tvheadend immediately when Tvheadend accesses it (could be just a black screen, could be a single image or whatever) but at the same time it would try to get the desired stream from the source and as soon as that arrives it would stop sending the "dummy" stream and start sending the actual stream to Tvheadend. Hope that makes sense, and I suspect Cabernet does something very much like that during commercial breaks? Anyway, I figured if anyone might know of such a thing, you would.

G Kazaroth wrote:

For number 1, I find it an issue with TVH. I normally need to delete the mux and tvguide data. Make sure the channel list for the network is empty, then try to do a scan again. Before that, you can check Cabernet by requesting a channel list like http://ipaddress/PlutoTV/lineup.json and see that the guidenumber is updated.

No, it's not getting updated that way. Same thing with 123TV and I started out using a Starting Ch Number of 3801 but it is still numbering starting at 1. Same thing in the channel guide in the GUI, nothing changes.

Yes, both the plugin and its instance need to be enabled for Cabernet to get the channel list. The release notes normally have the configuration. Look like I removed the old releases during releases cleanup (oops). It is the standard way to add an instance.

Thanks, it had been so long since I have added one that I forgot about that!

yes, there is code in Cabernet to send ATSC packets when no video is coming in from the provider as a keep alive. The issue with the play versus record for me is that Kodi is terminating the connection if video does not keep coming. I am not aware of any parameter in Kodi to stop this from happening. You can try it with VLC and see if it also does it there. I have not tried that. This is not associated with initialization and only talking about times when streams stop streaming for a while.

Now there is a difference between TVH initial start and pauses during streaming. TVH has a hard coded timeout on initialization, that overrides the configuration value (I found it in the TVH source). I think its like 10 seconds. We still try to send the ATSC packets, but TVH will terminate, if real video packets are not received at the start. Once video has been verified as received, then TVH uses the value in the configuration, to determine, if it should terminate the stream. In the past, I did ask TVH forum the question on the hardcoded timeout and got the normal response of if you want to become a developer, please take a shot.

I did look into sending a small blank video to TVH to get it out of initialization, but it was very difficult based on all the different stream configurations providers are using and possible dynamics within a channel. The biggest issue was knowing the future. The video must tag the video snippet with a timestamp (DTS/PTS) that matches what will be coming from the provider in the future. Obviously it is hard to determine this and is measured to 93000th of a second.

To Shea, I did the following. Ran http://ipaddress/123tv/lineup.xml and saw A&E started at 1001 (I have the channel prefix set to 1000). I then changed the value to 2000 and saved the changed. Then refreshed the tab with the url http://ipaddress/123tv/lineup.xml in my browser.
The GuideNumber changed from 1001 to 2001. This indicates Cabernet is changing the value as it should. All the client interfaces should be getting the updates, so the issue is not with Cabernet. I personally had to reorder a number of networks, so I know it is working, so long as the cache values in TVH and Kodi are cleared.

G Kazaroth wrote:

To Shea, I did the following. Ran http://ipaddress/123tv/lineup.xml and saw A&E started at 1001 (I have the channel prefix set to 1000). I then changed the value to 2000 and saved the changed. Then refreshed the tab with the url http://ipaddress/123tv/lineup.xml in my browser.
The GuideNumber changed from 1001 to 2001. This indicates Cabernet is changing the value as it should. All the client interfaces should be getting the updates, so the issue is not with Cabernet. I personally had to reorder a number of networks, so I know it is working, so long as the cache values in TVH and Kodi are cleared.

I don't know what to tell you, I have Starting Ch Number set to 3800 (I also tried 500, 1000, 2000, etc.) and it never changes. If I add or remove a suffix (N in this case) that changes but not the channel number. The first couple of channels look like:

<Lineup>
<Program>
<GuideNumber>1N</GuideNumber>
<GuideName>A&E (AE)</GuideName>
<URL>http://_redacted_:5004/123TV/watch/ae&lt;/URL>
<HD>0</HD>
</Program>
<Program>
<GuideNumber>2N</GuideNumber>
<GuideName>ABC</GuideName>
<URL>http://_redacted_:5004/123TV/watch/abc&lt;/URL>
<HD>1</HD>
</Program>

If I remove the EPG Channel Suffix that works

<GuideNumber>1</GuideNumber>

But it simply will not change the channel numbers no matter what value I try. By the way I have to specify the :6077 port to even get that XML file, not sure how you are getting it without using the port number.

Would it make any difference that I have been using essentially the same database since back when Locast was still around?

G Kazaroth wrote:

yes, there is code in Cabernet to send ATSC packets when no video is coming in from the provider as a keep alive. The issue with the play versus record for me is that Kodi is terminating the connection if video does not keep coming. I am not aware of any parameter in Kodi to stop this from happening. You can try it with VLC and see if it also does it there. I have not tried that. This is not associated with initialization and only talking about times when streams stop streaming for a while.

Now there is a difference between TVH initial start and pauses during streaming. TVH has a hard coded timeout on initialization, that overrides the configuration value (I found it in the TVH source). I think its like 10 seconds. We still try to send the ATSC packets, but TVH will terminate, if real video packets are not received at the start. Once video has been verified as received, then TVH uses the value in the configuration, to determine, if it should terminate the stream. In the past, I did ask TVH forum the question on the hardcoded timeout and got the normal response of if you want to become a developer, please take a shot.

I did look into sending a small blank video to TVH to get it out of initialization, but it was very difficult based on all the different stream configurations providers are using and possible dynamics within a channel. The biggest issue was knowing the future. The video must tag the video snippet with a timestamp (DTS/PTS) that matches what will be coming from the provider in the future. Obviously it is hard to determine this and is measured to 93000th of a second.

Okay, thanks. Didn't realize Tvheadend was that persnickety. It is indeed the "hard coded timeout on initialization, that overrides the configuration value" that is the problem. In my experience it is more like 20 seconds. And yeah, I have got the impression that they really don't care if users are inconvenienced by their design choices.

Thanks again, I appreciate the explanation!

You have a very strange thing happening with the channel number. Here is how it works. It takes what appears in the Channel Editor (database) for channel number and adds what appears in the config.ini (ex: epg-prefix = 2000) for that plugin/instance to it. Example: If I have "1" in the database, I get 2001. If I have "1N" in the database, I get 20001N. Basically, it will add or prepend based on whether they are numbers.

So, to have the channel number not change at all can only happen when the config does not have the epg-prefix line for that instance present. This code has not changed in a very long time, so it makes no sense you have the updates in the config.ini while they are not being used.

UPDATE:
I just realized you are using the starting channel number instead of the prefix (i would recommend the prefix). The starting number is implemented when the channel is stored in the database. To have the instance use the starting number, you would need to delete the channels (datamgmt > pick the plugin, select yes and click reset) then you can go to the scheduler and request an update to the channel list for that plugin. Only then would the channel numbers be updated with new values. The prefix basically does the same thing, but between the database and the client, so it is much much easier.

The reason for these implementations is that some people will be upset, if the channel editor does not have the number being set to the client. That was the reason for the starting number. But it is so much easier to have the adjustment made later, when sending the result to the client. The impact of the latter is that the channel editor will have what is in the database and not include the additional increase in channel number that is included when sending to the client. So, people have two choices based on what they want.

G Kazaroth wrote:

UPDATE:
I just realized you are using the starting channel number instead of the prefix (i would recommend the prefix). The starting number is implemented when the channel is stored in the database. To have the instance use the starting number, you would need to delete the channels (datamgmt > pick the plugin, select yes and click reset) then you can go to the scheduler and request an update to the channel list for that plugin. Only then would the channel numbers be updated with new values. The prefix basically does the same thing, but between the database and the client, so it is much much easier.

Using the prefix does work, thanks much!

Thank you for being nice about any issues. I figure it is not quite where it should be, but working on getting closer. Next release is going to be big. It's going to have a new TVGuide plugin mainly to help daddylive with EPG, but should provide additional EPGs for those like me that have an antenna, as well. (I currently use the zap2xml perl script) Fixed a number of issues/bugs and updated the tuner status on the home page to not just say Active; it will now have Starting, Filtered and Streaming states... With the new EPG plugin, an additional provider area will be added on the left side so streaming providers will be separate from EPG providers. The TVGuide plugin is primarily for US based TV channels and expect it not to help the plutotv, xumo or non-us based channels. One step at a time. Finally, I have not done code analytics in a while and preliminary info shows it needs lots of cleanup, so plan is to get that worked.

Sounds great. I also have an antenna and use zap2xml.pl and your category-filter.pl scripts so the TVGuide plugin may reduce some complexity.

I appreciate how Cabernet is getting even more advanced. Fussing with the EPG is where I spend most of my time now. I am trying to see if I can get it working in Kodi using both tvheadend pvr and nextpvr. Nextpvr doesn't play well with VOD. It will play VOD but when it gets to the point of the next program starting it just quits the video. Nextpvr is easier to setup than tvheadend but doesn't seem as robust. Still, I like the way it adds fanart where it can. So I want to use it for non VOD and tvheadend for VOD. I'm not sure I will be able to get it working that way as they seem to conflict. Maybe the channel numbers are stepping on each other. I don't know yet.

I have been working on trying to get better EPG data to fill in some gaps in Xumo. In some cases I prefer how Xumo does the VOD but then the EPG data is not good. Magellan TV Now or Electric Now for example. Sorting through the EPG data provided by Matt Huisman and adapting it to be used with some Xumo channels is something I am working on now. It does get a little tedious though. Sometimes I wonder if fussing with it is worth the hassle but I have been noticing how the sum seems to be much greater than the parts.

I also use Carlos Gomes' TVHlink to integrate various youtube live news channels. He says not to bother with setting up EPG for them but I find it worth the trouble. I would like to devise a robust way of having fake EPG for some of these channels like Africam or some of the music ones. Thus allowing for recording.

https://cgomesu.com/blog/Tvhlink/

I finally setup my hdhomerun extend but it needs a better outdoor antenna.

The loss of epg and playlist from github.com is kind of a bummer but have been managing to work around it. Generally. I used to use a combination of tvheadend with iptv simple and Matt Huisman's merge program. But now it seems like using the playlist and epg he provides with m3u inputs in Cabernet allows for recording. I'm not sure if handles the commercial breaks better than his plugins do but recently in some cases it seems to be fine.

I want to get some kind of PBS going to be able to record a few things like Nova, Frontline, that kind of thing. But most of of those channels seem to be accessable via his plugins or not at all.

Where I am at with EPG seems like such a kludge. Integrating several different services. m3u4u.com, https://i.mjh.nz/ and various grabbers from xmltv. Looking forward to seeing how your epg plugin will work. I notice some of the xumo epg is generic but when I look at the same channel on the web site the data is there. In some cases I try and work around that using xteve but no matter how I do it, it never feels very robust or maintainable.

I recommend the kodi skin arctic horizon 2. It can be difficult to use for troubleshooting but seems to provide the richest experience for fanart and usability when used with nextpvr between cabernet and kodi.

I am curious about the mention of category-filter.pl, I could not find it.

Thanks, loving how it is coming along.

The category_filter.pl is something I did to convert the xmltv.xml from zap2xml to support the tvh content types so colors show up in kodi correctly. The files are located in the repository
https://github.com/rocky4546/script.xmltv.tvheadend
Part of Cabernet incorporates that script so categories are tvh compliant.

The reason VOD works in TVH is that when TVH sees the stream has ended, it will normally request a refresh causing the next program to start in Cabernet. I believe TVH will also start a new file during recordings. Since they are all VOD, it will start at time zero even after some timeout which is configurable. I suspect NextPVR does not know how to handle a stream that has stopped or completed.

zap2xml is gone because, as the author told me, "a lawyer for zap didn't like it." I hope lawyers don't take a dislike to this project as well.

Yes, I would recommend caution when proceeding with EPG gathering as well. In North America at least it's all basically controlled by one company who maintains copyrights and will go after any who redistribute. One needs to stay under the radar. While Schedules Direct managed to secure distribution rights on a subscription model they never managed to develop a workable system to build a lineup for available FTA satellite channels.

You are correct. Plan is to do a number of things to try and keep lawyers away. Obviously, we have learned from trump that you can sue somebody even though it is a lie and many times the person will bend. That would imply doing nothing, which is not an option. So, the plan is to be compliant with the TOS. iptv-org recently got banded from GitHub for redistributing information from epg websites. The TOS clearly states "You are only permitted to access and view the Content for personal, non-commercial purposes in accordance with this Agreement". Redistribution is prohibited unless they specifically authorize it. So, the plan is to
1. Have no data that is redistributed in the plugin
2. Clearly state the plugin falls under the TOS
3. Clearly state the information provided by the plugin is for personal, non-commercial use
4. Separate the plugin from Cabernet so any attacks would only be to the plugin and not to the project as a whole
5. Hope and pray the lawyers have more to do than attack little me just because
6. Keep the project as non-commercial for now. Means things like: no advertising, no subscriptions, etc...

0.9.9.0 new EPG plugin
I do not recommend this unless you want to try it out. Please read release notes before upgrading. Expect more changes due to the size of the changes made. The EPG plugin is in a separate repository of the plugin name and can be downloaded for release 0.1.0. More testing and updates will be needed although the current version does seem stable.

After running the upgrade tests and finding a few bugs, those are all fixed. It looks like 0.9.9.2 with the EPG plugin is stable. Running more tests.

Found during a patch upgrade, the EPG plugin library is deleted (during the upgrade). It can be restored by copying the files back into the EPG plugin folder. Will be working on a fix.

For the external EPG plugin, I have added another plugin folder called plugins_ext at the same level as the current plugins folder to house external plugins. Had to cleanup the plugin in the plugins folder, but it works under 0.9.9.3.

The initial EPG days to pull is set to 2 days, which should stay until you are sure you are ready. The schedule only takes a second, but pulling the program details takes a very long time.

Not seeing how to add plugin.

Seems like it is running now.

If you see any other issues, let me know. I am working to reduce the number of connection refuse exceptions for daddylive right now. will probably be in the 9.9.4 release.

The connection refused looks like a DDOS issue with everyone trying to get the same video segment at the same time. Trying to see if Cabernet slows down for a short, if that reduces the number of lost segments.