Project

General

Profile

DaddyLive, PlutoTV, XUMO, M3U/XMLTV, SamsungTV, Plex, TVGuide interfaces (appliance)

Added by G Kazaroth almost 4 years ago

Cabernet for (Cable Network) version 0.9.12 (4/13/2023)
TVGuide, DaddyLive, PlutoTV, XUMO, M3U/XMLTV, SamsungTV, Plex

Provides a configurable interface from providers to TVHeadend.

  • Direct streaming plugins for DaddyLive, PlutoTV, XUMO
  • EPG supplemented using TVGuide.com data
  • M3U Plugin provides channels for: SamsungTVPlus, Plex, PBS, Stirr, and others
  • From: https://i.mjh.nz/
  • Import your own M3U file

https://github.com/cabernetwork/cabernet/releases
Purpose is to get a service that will take the DaddyLive, PlutoTV or XUMO server stream, clean it and feed it into TVHeadend and other DVRs (Also tested on Emby, JellyFin and Plex). Also it runs on Linux, Windows as a service and Docker. Windows has a installer. Once installed, setup is easy with TVHeadend. Also includes a TVGuide.com plugin to obtain TVGuide data.

1) Setup the automatic IPTV network (recommend new URL is http://[host]:6077/PlutoTV/channels.m3u)

Make sure to set the "Maximum # input streams". PlutoTV is set to a max of 4 and tvheadend uses 2 per tuner during initial screening. Doing a force scan will create the mux and service values. Also, turn View level to Advanced and set the Re-fetch period (mins) to a very large number. TVheadend has a tenancy to cause issues when channels change (Changed Services will not be mapped to channels). The Maximum Timeout is used to wait for a reply during a Forced scan. Recommend keep this low, like 15-20 seconds. Some of the channels may fail, but it is faster than having a high setting and waiting for all channels to scan. Just individually rescan those that failed by setting each mux back to PEND from IDLE.
2) Next you can setup the grabber. I use a URL grabber written in Unix bash and is an extremely small file. It can be found in the github repo at
https://github.com/cabernetwork/cabernet/tree/master/lib/tvheadend/service/Unix called tv_grab_url
Place the grabber file in the same location as the other TVHeadend tv_grab* files, change the permissions to executable and restart TVHeadend. This should allow TVHeadend to pickup the new grabber. While in the grabber list, make sure and disable any OTA grabbers. Stations no longer send this information and will only cause TVHeadend to use a tuner for scanning. Displaying the log window by clicking the three ^ in the bottom right is helpful at this time.

Have the grabber run and populate the EPG data into the EPG Grabber Channels tab. The log should show a quantity of channels were detected.
Pop over to the EPG Grabber tab and disable the OTA grabber cron. Also, update/replace the Internal grabber cron schedule using something like below. The example will pull the TV guide at 6:04am and 5:52pm. Add more if you need. It is recommended to use static cron times.

4 6 * * *
52 17 * * *

3) In the Channel view, select Map all channels.

This will tie the services, EPG and channels together, automatically. After this, re-grab the EPG data. This will populate the EPG tab with shows. For TVH version 4.3, the Number column will auto-populate. For TVH 4.2, you will need to manually add channel numbers.
4) Display the TVGUIDE. This appliance has special features which maps the tvheadend genre, giving colors on tvguides. It also has enhanced guide descriptions and optional additional channel notations if you use many streams. Below is the Kodi tvguide using the pvr.hts plugin.

For Kodi, go to the settings for PVR and turn on the General setting "Use channel numbers from backend".


Replies (960)

RE: PlutoTV, XUMO interface (appliance) - Added by G Kazaroth about 3 years ago

Sean, the logging file will be 0 bytes until you turn it on. The default setting is for Cabernet to log to the syslog file with local file logging disabled. What you are saying is the 6077 port (called webadmin port) is killing all of its threads with exceptions and at some point stops responding. I would recommend increasing the syslog log level to INFO and see what it is saying. An example of an exception is below

Exception in thread Thread-10:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/xxx/cabernet/lib/streams/m3u8_queue.py", line 281, in run
    added += self.add_to_stream_queue(playlist)
  File "/home/xxx/cabernet/lib/streams/m3u8_queue.py", line 330, in add_to_stream_queue
    .format(datetime.datetime.now(), m3u8_segment.program_date_time))
NameError: name 'datetime' is not defined

RE: PlutoTV, XUMO interface (appliance) - Added by Sean Micklem about 3 years ago

G Kazaroth wrote:

I would be interested in having Sean provide his insight on this as to whether always throwing away the first m3u8 set, or use the data_time stamps inside the file to determine where to start. I am good with either way.

I'm fine with throwing away the first m3u8 set, but don't have a strong preference either way as long as it works. The one advantage I can see to throwing away the first m3u8 set is that there is always the possibility that the data_time stamps could be wrong, corrupted, or for a different time zone, if I am understanding correctly. By throwing away everything except the last entry, you avoid that possible issue (even if it is an unlikely one, but I have no idea how accurate those are). The biggest issue I have found with timestamps in Linux systems is that things that rely on them (including some PVR software) can get really flaky around the time of a daylight saving time change, where you gain or lose an hour. Maybe that is just poor programming but I have no idea how those date_time stamps are generated, or interpreted. So if it were me (and if I were a programmer) I'd just avoid that hassle and toss out the old m3u8 sets.

As for my web interface problem, after you mentioned the syslog I took a look in that and saw where it appeared to have run out of memory at one point. The only lines I saw that seemed specific to Cabernet ("locast" in the log) contained things like "2021-09-23 23:07:21,558-WARNING:db Delete request ignored, database is locked" and "2021-09-23 23:07:36,455-WARNING:db Add request ignored, database is locked" but I don't know what would have caused those. Anyway I am trying something to increase available memory and I'll see if the issue still occurs. If it does I'll see if I can figure out how to increase that log level and then report back. Thanks!

RE: PlutoTV, XUMO interface (appliance) - Added by G Kazaroth about 3 years ago

I have tried to reduce memory usage for generating xmltv.xml in the past, but the xml modules do use lots of memory. If you can, you could reduce memory usage by requesting smaller sections of xmltv.xml. Cabernet already processes a plugin/instance separately, so not sure if this will help. Also, pretty printing of xml does use memory. Try disabling that.
Check the size of the files and make sure you do not have something you should not (like Locast items). The sum of all plugin file sizes should equal approximately the same size as the full request.

I ran some memory checks on the xmltv.xml pull and it does not look like it is leaking. Also, added a memory usage enabler on the config settings, which will log at the DEBUG level the memory usage. Here is the result when I ran the xmltv.xml on the M3U plugin test.

2021-09-25 07:22:48,501-DEBUG:utils pid:92097 Top 3 lines
2021-09-25 07:22:48,501-DEBUG:utils #1: dom/expatbuilder.py:778: 2391.9 KiB
2021-09-25 07:22:48,501-DEBUG:utils     _attrsNS[(EMPTY_NAMESPACE, aname)] = a
2021-09-25 07:22:48,502-DEBUG:utils #2: dom/expatbuilder.py:223: 1772.4 KiB
2021-09-25 07:22:48,502-DEBUG:utils     parser.Parse(string, True)
2021-09-25 07:22:48,502-DEBUG:utils #3: dom/expatbuilder.py:744: 1678.0 KiB
2021-09-25 07:22:48,502-DEBUG:utils     node = minidom.Element(qname, uri, prefix, localname)
2021-09-25 07:22:48,502-DEBUG:utils 39 other: 8551.8 KiB
2021-09-25 07:22:48,502-DEBUG:utils Total allocated size: 14394.1 KiB

I will implement the drop all but last video section at start. Finished the M3U plugin; just going through cleanup at this point.

RE: PlutoTV, XUMO interface (appliance) - Added by G Kazaroth about 3 years ago

To Sean, I found a glitch that may be causing the web port issue. If you have a plugin or instance disabled, but still have config settings in the config.ini, then I have found a number of cases that cause it to crash. It's fixed in the next release, but check to see if you have anything disable, but still have settings in the config.ini. To test, go to the links page and click on each link while looking at the log, to see if an exception appears.

Almost finished with the next release, and as usual, it has major changes. It does have the skipping segments at the start by default, but about half of XUMO is VOD-based. This means the last segment is the end of the show, so it will play nothing. I have a setting at the instance level to switch it to VOD when needed. I recorded a 720p VOD show that was 1:37 long in about 8 minutes (translates to around 36Mbps download rate for the 2GB file) with SDT and PTS Resync off. Replayed beautifully too. Biggest thing left is to create the patch, which will move all the plugin settings down to the instance level.

Another interesting fact with the M3U plugin. For samsungtv, it has xumo, plutotv, samsung and other channels. The plutotv channels play only if you uuencode the uri, while the other channels play, if you don't. So, there is a switch in the M3U instance to turn uuencoding on or off. You can tell it's a pluto channel with the name in the uri in the log (and that it won't play).

The M3U plugin will act just like the others. I have an entry in the config.ini file

[m3u_samsumgtv]
label = M3U SamsungTV

Then restart and go into the settings page for the instance; setting the M3U and XMLTV urls. If the XMLTV url is missing, it will try to add blank programs for all the channels that are enabled. Be aware, I have not done anything to the channel editor, so ingest 10,000 channels at your own risk! Lastly, The M3U comes with startup tasks set, but the EPG regular updates are not present. Since each EPG update is different, it made no sense to set it to something. Samsung seems to run at 3 hour intervals, but periodically will do something different.

Some interesting instance setups might include:
- For XUMO, have two instances, one with VOD disabled, playing live channels, and one with VOD enabled, playing channels with older movies and episodes.

RE: PlutoTV, XUMO interface (appliance) - Added by Sean Micklem about 3 years ago

G Kazaroth wrote:

To Sean, I found a glitch that may be causing the web port issue. If you have a plugin or instance disabled, but still have config settings in the config.ini, then I have found a number of cases that cause it to crash. It's fixed in the next release, but check to see if you have anything disable, but still have settings in the config.ini. To test, go to the links page and click on each link while looking at the log, to see if an exception appears.

No, don't have anything disabled, but I haven't seen the issue today so I am fairly certain it was a memory issue.

I recorded a 720p VOD show that was 1:37 long in about 8 minutes with SDT and PTS Resync off. Replayed beautifully too.

I am trying to wrap my head around how that could happen. When you schedule a recording in Tvheadend, as far as I know it records for a specific amount of time. So if you schedule a two hour movie, it will record whatever is on that channel for two hours. I understand that if the stream ends then Tvheadend has to stop recording because there is nothing left to record, but won't that be considered an error by Tvheadend? Guess I'll just have to try it and see how it works. In any case I was wondering if there is any easy way to tell which channels are live, and which are VOD.

If the XMLTV url is missing, it will try to add blank programs for all the channels that are enabled.

Would it be too much trouble to add a setting to not do that on a per-instance basis? I was not planning on using that 10,000 channel .m3u list but I may create a custom .m3u list that is a subset of that list, with a few hand-picked channels. That would be hand-edited, but of course in that case I would not have a XMLTV url, but I also may not want it trying to populate guide data for those channels. I won't know for certain until I see how it operates.

Be aware, I have not done anything to the channel editor, so ingest 10,000 channels at your own risk!

Well if I am having memory issues now, I don't think trying to process 10,000 channels would help that! So no, I'm not trying that.

Some interesting instance setups might include:
- For XUMO, have two instances, one with VOD disabled, playing live channels, and one with VOD enabled, playing channels with older movies and episodes.

This would be one reason it would be good to know if there is a way to tell which XUMO channels are live, and which are VOD.

Thanks again for your continuing work on this!

RE: PlutoTV, XUMO interface (appliance) - Added by G Kazaroth about 3 years ago

I am trying to wrap my head around how that could happen. When you schedule a recording in Tvheadend, as far as I know it records for a specific amount of time. So if you schedule a two hour movie, it will record whatever is on that channel for two hours. I understand that if the stream ends then Tvheadend has to stop recording because there is nothing left to record, but won't that be considered an error by Tvheadend? Guess I'll just have to try it and see how it works. In any case I was wondering if there is any easy way to tell which channels are live, and which are VOD.

The guide for the live ones have the XMLTV tag <live/> while the VOD ones do not. I do not believe the channel editor currently will show which are VOD, but could look into that after this first release. It is similar to saying a channel is HD, but the video quality is set at the program level and not at the channel level. On Locast, I took the first program and that decided what type of video quality it had. I could do the same thing for VOD.

If the XMLTV url is missing, it will try to add blank programs for all the channels that are enabled.

Would it be too much trouble to add a setting to not do that on a per-instance basis? I was not planning on using that 10,000 channel .m3u list but I may create a custom .m3u list that is a subset of that list, with a few hand-picked channels. That would be hand-edited, but of course in that case I would not have a XMLTV url, but I also may not want it trying to populate guide data for those channels. I won't know for certain until I see how it operates.

It does this on a per channel basis and has nothing to do with an instance. Also, it really did not take up much room in the database, so its really not an issue. BTW, there is a setting at the instance level, to disable the EPG portion totally. This was to support two instances of Locast. When enabled, will both not generate any EPG, to the database, for the instance, but also will not use any data in the database even if it exists for the XMLTV. So, I think you are covered.

On the topic of recording VOD, I found something. Using the VLC client or TVH, the video is throttled once about 30 seconds is sent to the client. I suspect the clients want to maintain the PTS real-time capability. There may be a way to increase the buffer size in the client, but have not found it. So, when I recorded with TVH, it was going to take a while due to the throttling. To see what Cabernet could do without throttling, I ran a wget command to act like a client without throttling. The command I used was

wget -q -T 3600 -O movie.ts "http://ipaddress:5004/XUMO/watch/99991709" 

This showed that the python script was not a bottleneck in the recording (with a download rate of 36Mbps) and it ended perfectly. Also, the size of just under 2Gb indicated a quality video.

RE: PlutoTV, XUMO interface (appliance) - Added by G Kazaroth about 3 years ago

0.9.4.0 M3U Plugin
Sorry, it took a while to finish testing the next release. As expected, there are a lot of changes, including adding the M3U plugin.

If you are interesting in testing this, here is what you need to know.
First, I have changed the default state of all plugin and instances to disabled. You may need to turn them back on.
After adding the m3u instance and label to the config.ini, restart cabernet.
The plugin will come up disabled. Enable the plugin and restart...
Now the M3U instance is up. You will need to provide a M3U url. This can be a number of things:
  • A URL or FILE location
  • .m3u8, .m3u, .gz or .zip file extensions are supported. It will automatically uncompress the compress files for you.
  • The File Type setting should be left to autodetect. Set the value when the url provided does not end in an extension.
  • An example of a url is https://i.mjh.nz/SamsungTVPlus/us.m3u8
The XMLTV file is optional.
  • The "XMLTV Enabled" setting will prevent any xmltv data from populating when disabled.
  • Although the M3U file is required, the XMLTV file is not.
  • An example of a url is https://i.mjh.nz/SamsungTVPlus/us.xml.gz
  • The scheduler has an entry for the XMLTV refresh, but basically has no entries. For the URL above, recommend every 3 hours. It provides an 8 hour epg.
Channel Editor
  • You can wait and add the XMLTV info after you edit the channel list. The XMLTV ingest runs very fast, so either way.
    Once everything is setup the way you want, enable the instance and restart Cabernet.

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by C Island about 3 years ago

Tried to test....

I installed latest version 0.9.4.4 and enabled M3U. Then I restarted cabernet a few times.

Under 'Providers' I have a M3U tab but it only has a two options 'enabled' and 'tuner_count'. I do not see where to add a M3U url. Did I miss something?

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by Sean Micklem about 3 years ago

C Island, you need to manually add an instance to your config.ini file, such as:

[m3u_samsumgtv]
label = M3U SamsungTV

Then do

sudo systemctl restart locast.service (assuming you haven't renamed the service to cabernet like I did)

G Kazaroth:

It seems to work great other than that I am seeing one issue. I created a local list from that big 10,000 entry list (mine is MUCH shorter) but what I find is no matter what I do, on some channels it plays one or two seconds of video with no sound (which would not in itself be a problem, except...) then it plays one or two seconds with sound, and then the stream fails completely. If you remember a few days ago I mentioned a channel that starts with "KD", that is one of the ones that have this issue. I'm not sure if this is because the stream is changing resolution or if something else is happening but the direct link plays fine in VLC, it's only when going through Cabernet that it dies out shortly after starting. It does not do this with all channels (I tried adding a Samsung TV link to the list and that worked fine, other than that it was like pulling teeth to get the additional channel to appear in the channel editor). I tried unchecking the VOD box and then those streams played fine, but programming that aired several hours ago! You had said earlier that "On the first m3u8 list it throws away everything except the last entry." I wonder if maybe it's a case of needing to throw away all but the last two or three entries, otherwise there's not enough in the buffer and it dies out? I don't know but right now it's so close, so I hope it's an easy fix.

P.S. Please excuse any typing/syntax errors, I am running on a serious lack of sleep again!

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by G Kazaroth about 3 years ago

0.9.4.5 fixed m3u8 with extra long ts file
Sean, I found a few bugs. Found yours too. Cabernet is timing out trying to assess the list of 300 items coming from the provider. It was a Python slow processing thing. Got it fixed and is now responsive.
Other items:
  • Noticed there was no channel number (which tvh is expecting) I added tvg-id to use for the channel id if it is missing. Did not test through TVH...
  • Noticed the logo was set, but to '', which Cabernet did not like, so added some code to ignore that.

I checked out your channel, and it seems to play fine with VOD unchecked through VLC.

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by Sean Micklem about 3 years ago

G Kazaroth wrote:

I checked out your channel, and it seems to play fine with VOD unchecked through VLC.

I'm glad it does for you, I just wish I could say the same. The newer version made no difference for me. I really wish it had. Most of the SamsungTV channels play fine but a few random ones won't play at all for some reason (not talking about the Pluto ones, but I do have "Decode M3U8 URL" checked anyway). Maybe they are just down at the moment. Thanks for working on this!

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by G Kazaroth about 3 years ago

yea, just woke up a little more and tried again. Its still having the performance issue. let me see if I can fix this.

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by G Kazaroth about 3 years ago

OK, found a sleep command causing all the issues. Glad I woke up! removed that sleep timer and its now running fast, so no timeouts. (cross fingers ... still need breakfast)

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by Sean Micklem about 3 years ago

G Kazaroth wrote:

OK, found a sleep command causing all the issues. Glad I woke up! removed that sleep timer and its now running fast, so no timeouts. (cross fingers ... still need breakfast)

That appears to have solved the issue! Thank you VERY much!

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by Sean Micklem about 3 years ago

Well this is weird, most of the SamsungTV channels will not play at all in VLC and will not scan into Tvheadend, but some will play just fine. Examples of ones that will not play or scan in include 1000 The Movie Hub and 1001 Dr. G: Medical Examiner. Examples of ones that will scan in and play are 1017 Yahoo Finance and 1023 USA Today. Settings I am using under "Player" are internal proxy, and all the check boxes are unchecked except for "Decode M3U8 URL" (I tried unchecking that but it didn't help). In the log I see errors like:

Sep 27 14:35:21 systemname locast[27333]: 2021-09-27 14:35:21,747-INFO:channels M3U: Getting video stream info for USBA3000041ZP
Sep 27 14:35:21 systemname locast[27333]: 2021-09-27 14:35:21,893-INFO:decorators HTTPError in function get_best_stream(), retrying 28268 HTTP Error 400: Bad Request https://dje6yassknq8t.cloudfront.net/v1/master/3722c60a815c199d9c0ef36c5b73da68a62b09d1/MovieHub2-prod/playlist.m3u8?ads.device_did={PSID}&ads.device_dnt={TARGETOPT}&ads.us_privacy=1YNY&ads.app_domain={APP_DOMAIN}&ads.app_name={APP_NAME}
Sep 27 14:35:22 systemname locast[27333]: 2021-09-27 14:35:22,966-INFO:decorators HTTPError in function get_best_stream(), retrying 28268 HTTP Error 400: Bad Request https://dje6yassknq8t.cloudfront.net/v1/master/3722c60a815c199d9c0ef36c5b73da68a62b09d1/MovieHub2-prod/playlist.m3u8?ads.device_did={PSID}&ads.device_dnt={TARGETOPT}&ads.us_privacy=1YNY&ads.app_domain={APP_DOMAIN}&ads.app_name={APP_NAME}
Sep 27 14:35:23 systemname locast[27333]: 2021-09-27 14:35:23,968-WARNING:decorators Multiple HTTP Errors, unable to get url data, skipping get_best_stream() 28268 HTTP Error 400: Bad Request https://dje6yassknq8t.cloudfront.net/v1/master/3722c60a815c199d9c0ef36c5b73da68a62b09d1/MovieHub2-prod/playlist.m3u8?ads.device_did={PSID}&ads.device_dnt={TARGETOPT}&ads.us_privacy=1YNY&ads.app_domain={APP_DOMAIN}&ads.app_name={APP_NAME}
Sep 27 14:35:23 systemname locast[27333]: 2021-09-27 14:35:23,968-WARNING:m3u8_queue Unknown Channel USBA3000041ZP
Sep 27 14:35:24 systemname locast[27333]: 2021-09-27 14:35:24,750-INFO:web_tuner Provider Connection Closed

Sep 27 14:36:32 systemname locast[27333]: 2021-09-27 14:36:32,186-INFO:channels M3U: Getting video stream info for USBB3200004X3
Sep 27 14:36:32 systemname locast[27333]: 2021-09-27 14:36:32,308-INFO:decorators HTTPError in function get_best_stream(), retrying 28351 HTTP Error 400: Bad Request https://d2j25awujlvs9u.cloudfront.net/v1/master/3722c60a815c199d9c0ef36c5b73da68a62b09d1/DrGMedicalExaminer-prod/playlist.m3u8?ads.device_did={PSID}&ads.device_dnt={TARGETOPT}&ads.us_privacy=1YNY&ads.app_domain={APP_DOMAIN}&ads.app_name={APP_NAME}
Sep 27 14:36:33 systemname locast[27333]: 2021-09-27 14:36:33,380-INFO:decorators HTTPError in function get_best_stream(), retrying 28351 HTTP Error 400: Bad Request https://d2j25awujlvs9u.cloudfront.net/v1/master/3722c60a815c199d9c0ef36c5b73da68a62b09d1/DrGMedicalExaminer-prod/playlist.m3u8?ads.device_did={PSID}&ads.device_dnt={TARGETOPT}&ads.us_privacy=1YNY&ads.app_domain={APP_DOMAIN}&ads.app_name={APP_NAME}
Sep 27 14:36:34 systemname locast[27333]: 2021-09-27 14:36:34,382-WARNING:decorators Multiple HTTP Errors, unable to get url data, skipping get_best_stream() 28351 HTTP Error 400: Bad Request https://d2j25awujlvs9u.cloudfront.net/v1/master/3722c60a815c199d9c0ef36c5b73da68a62b09d1/DrGMedicalExaminer-prod/playlist.m3u8?ads.device_did={PSID}&ads.device_dnt={TARGETOPT}&ads.us_privacy=1YNY&ads.app_domain={APP_DOMAIN}&ads.app_name={APP_NAME}
Sep 27 14:36:34 systemname locast[27333]: 2021-09-27 14:36:34,383-WARNING:m3u8_queue Unknown Channel USBB3200004X3
Sep 27 14:36:35 systemname locast[27333]: 2021-09-27 14:36:35,189-INFO:web_tuner Provider Connection Closed

In contrast a good one shows:

Sep 27 14:41:31 systemname locast[27333]: 2021-09-27 14:41:31,859-INFO:channels M3U: Getting video stream info for USAJ4300005PJ
Sep 27 14:41:33 systemname locast[27333]: 2021-09-27 14:41:33,391-INFO:internal_proxy Serving 28735 https://yahoo-samsung.amagi.tv/playlist1080pl_51373.ts (6.006)s (3436828B) ttw:0.15s 0
Sep 27 14:41:41 systemname locast[27333]: 2021-09-27 14:41:41,782-INFO:internal_proxy Connection dropped by end device [Errno 32] Broken pipe 28735
Sep 27 14:41:42 systemname locast[27333]: 2021-09-27 14:41:42,489-INFO:web_tuner Provider Connection Closed

Just wondering, is it possible that the ? character and everything after in those links needs to be stripped off?

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by G Kazaroth about 3 years ago

When I tested, I found the "cloudfront.net" ones with the "{TARGETOPT}" and other variable had to NOT be uudecoded. Based on your log, you are uudecoding the string. Instead it should have a bunch of % characters. Only the PlutoTV ones (that have plutotv in the url) should be decoded.

The one that was good in your example happened to have no uuencoding in the url (no % characters), so it would work either way.

Working on a number of updates. I have a new M3U filter working. It filters on ingest... Tried the following filter successfully "tvg-country=US, group-title=Local". Used it on the 30,000 channel file...
Also finding a number of thumbnail/image issues with channels. When I get it all working, will release another version.

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by Sean Micklem about 3 years ago

I am pretty sure I tried switching off the uudecoding and it made no difference, but I'll try it again.

One other bit of weirdness although you may not be able to do anything about it, it seems that Kodi 18.9 will not play the audio from a LIVE channel that is in high definition (1080) but plays it fine if the channel is not full HD (720), however if you record the channel and then start playing the recording then the sound pops in after a second or so. No idea what causes that; the audio is obviously there, Kodi just won't play it live!

EDIT: I did uncheck "Decode M3U8 URL" (so now none of the checkboxes in the "Player" section are checked), still got this:

Sep 27 16:09:21 systemname locast[27333]: 2021-09-27 16:09:21,669-INFO:channels M3U: Getting video stream info for USBA3000041ZP
Sep 27 16:09:21 systemname locast[27333]: 2021-09-27 16:09:21,769-INFO:decorators HTTPError in function get_best_stream(), retrying 2970 HTTP Error 400: Bad Request https://dje6yassknq8t.cloudfront.net/v1/master/3722c60a815c199d9c0ef36c5b73da68a62b09d1/MovieHub2-prod/playlist.m3u8?ads.device_did={PSID}&ads.device_dnt={TARGETOPT}&ads.us_privacy=1YNY&ads.app_domain={APP_DOMAIN}&ads.app_name={APP_NAME}
Sep 27 16:09:22 systemname locast[27333]: 2021-09-27 16:09:22,850-INFO:decorators HTTPError in function get_best_stream(), retrying 2970 HTTP Error 400: Bad Request https://dje6yassknq8t.cloudfront.net/v1/master/3722c60a815c199d9c0ef36c5b73da68a62b09d1/MovieHub2-prod/playlist.m3u8?ads.device_did={PSID}&ads.device_dnt={TARGETOPT}&ads.us_privacy=1YNY&ads.app_domain={APP_DOMAIN}&ads.app_name={APP_NAME}
Sep 27 16:09:23 systemname locast[27333]: 2021-09-27 16:09:23,852-WARNING:decorators Multiple HTTP Errors, unable to get url data, skipping get_best_stream() 2970 HTTP Error 400: Bad Request https://dje6yassknq8t.cloudfront.net/v1/master/3722c60a815c199d9c0ef36c5b73da68a62b09d1/MovieHub2-prod/playlist.m3u8?ads.device_did={PSID}&ads.device_dnt={TARGETOPT}&ads.us_privacy=1YNY&ads.app_domain={APP_DOMAIN}&ads.app_name={APP_NAME}
Sep 27 16:09:23 systemname locast[27333]: 2021-09-27 16:09:23,852-WARNING:m3u8_queue Unknown Channel USBA3000041ZP
Sep 27 16:09:24 systemname locast[27333]: 2021-09-27 16:09:24,673-INFO:web_tuner Provider Connection Closed

EDIT 2: I restarted Cabernet, only then did they work!

So I'm guessing right now you have to make a separate instance for the channels that require uudecoding, but since those are Pluto channels and since we already have a PlutoTV plugin I guess I'll just disable the Pluto channels in the SamsungTV list for now.

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by G Kazaroth about 3 years ago

Ah, You found one of the bugs I fixed today. The config change was not being applied to scheduled events. Once I get the rest of the thumbnail fixes in, I will release.

As for a separate instance, people claim it is better to use the direct stream than one indirectly provided by another provider. In this case, it would be better to just use the PlutoTV plugin as you have indicated. I did not run into any other issues besides PlutoTV, but did not try all of them.

As for anyway to tell... There is no metadata that would tell you; only the url contains the info. What I did was bring up the channels database using DB Browser app, filter to the samsungtv instance and then search the json column for plutotv. It gave me a list of all the channels that had that.

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by G Kazaroth about 3 years ago

I did the release.

You mention you were having issues with 1080. I played NBC News NOW on XUMO live, which is 1080 at 30fps. It played just fine in Kodi. Going from Cabernet > TVH > Kodi, no issues. You must have a Kodi setup issue. My default Kodi display setting is 1080 at 60fps.

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by G Kazaroth about 3 years ago

small update. Added ability to set the number of segments it plays at the start, when not VOD. I have decided 3 is a good number, but the default is 1. Also shrunk the columns in channel editor.

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by C Island about 3 years ago

I like the idea of the update allowing more segments. I was getting some hesitation on live playback on Kodi. To manually fix it, I found that if I paused, waited a second or two and resumed playback it again it was better so I hope your update will cover that issue.

BTW: With 0.9.4.7 I had multiple python errors/exceptions/tracebacks when I deleted XUMO and re-added it. I have not not set XUMO to 'Play All (VOD)' but I was careful to set max streams to 1 and max timeout to 15 but I still had multiple issues. My wife was watching a Pluto channel when I dropped and added XUMO and it continued to play. However, some of my recordings of Pluto channels (eg 614 DABL) failed during the rescan. A lot of XUMO channels failed or were 'OK - partial' and I am thinking those might have been VOD channels which might have confused TVH as they only got 1 segment. I did not post them due to being from 0.9.4.7 but tell me if you want me to show you the exceptions/tracebacks and I will pull them from the logs and add them here.

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by G Kazaroth about 3 years ago

Done with all the 0.9.4 updates. It seems stable, so I'm going to let it stew for a while. If you find something, especially an exception on 0.9.4.9, let me know. I have not decided what the next upgrade will be; there is plenty to choose.

Having a few channels fail on a rescan is expected. Just reset those from IDLE to PEND and it should be fine. Ads tend to cause most of those issues. OK - Partial just means it did not get enough clean segments; same issue, just rescan those specific channels. Changing the max timeout from 15 to 100 or so can help reduce failed scans, but that causes TVH to take forever, so you have choices. As for playing during a rescan. TVH wants to fully use all tuners for the rescan, but you also want to watch shows. Many cases, the scan wins and the shows fail.

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by Sean Micklem about 3 years ago

G Kazaroth wrote:

I did the release.

You mention you were having issues with 1080. I played NBC News NOW on XUMO live, which is 1080 at 30fps. It played just fine in Kodi. Going from Cabernet > TVH > Kodi, no issues. You must have a Kodi setup issue. My default Kodi display setting is 1080 at 60fps.

NBC News NOW plays fine for me in Kodi, with audio, so it's not strictly a resolution issue. My default Kodi display is the same as yours, so not sure what the issue is. It only happens with a few channels.

One thing that is badly needed, especially now that you can specify a local .m3u file, is a way to force Cabernet to re-read the file, throwing out whatever was there before. I deleted a channel and changed the channel numbers on a few others (that was before I remembered that you can eliminate the channel numbers completely) and it was like pulling teeth to get Cabernet to re-read it, basically I had to delete the instance and start over with a new one, but then some way somehow it brought back the zombie channels and nothing I could do would get rid of them. I could deselect the old ones in the channel editor but if Cabernet thinks a new one is a duplicate (shows it in yellow), then it pretty much ignores it even if the one it's supposedly a duplicate of is disabled. I kept trying to pull myself out of that hole and screwed it up so bad that I had to go back to a backup I made almost a week ago, losing everything I did today. It would be a big help if on the channel editor page there was a trashcan icon next to each channel, so you could absolutely positively delete any channels that shouldn't be there (of course it should give you a warning and confirmation popup before actually doing the delete. But also, when a m3u list is changed, there should be a way to force it to be re-read, and to throw out any channels that are no longer in the m3u file (or that have had changed channel numbers). Hope that makes sense, this is the second night I've had almost no sleep!

Thanks much for all you have done on this. I am going to try to get some sleep now! :)

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by C Island about 3 years ago

I was looking at live vs VOD m3u8 playlists. It seems as though VOD playlists explicitly call out that they are VOD using the EXT-X-PLAYLIST-TYPE tag. Would it be possible for cabernet to automatically treat a stream as VOD and not delete segments from the start if the playlist specifies it is VOD?

Here is an example of a partial XUMO VOD playlist that has the EXT-X-PLAYLIST-TYPE tag set to VOD :

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:6.000000,
1_000.ts
#EXTINF:6.000000,
1_001.ts
#EXTINF:6.000000,
1_002.ts
#EXTINF:6.000000,
1_003.ts
#EXTINF:6.000000,
1_004.ts
#EXTINF:6.000000,
1_005.ts

I also checked rfc8216 and it seems to say to leave the playlist alone if EXT-X-PLAYLIST-TYPE is set to EVENT or VOD. https://datatracker.ietf.org/doc/html/rfc8216#page-24

4.3.3.5.  EXT-X-PLAYLIST-TYPE

   The EXT-X-PLAYLIST-TYPE tag provides mutability information about the
   Media Playlist file.  It applies to the entire Media Playlist file.
   It is OPTIONAL.  Its format is:

   #EXT-X-PLAYLIST-TYPE:<type-enum>

   where type-enum is either EVENT or VOD.

   Section 6.2.1 defines the implications of the EXT-X-PLAYLIST-TYPE
   tag.

   If the EXT-X-PLAYLIST-TYPE value is EVENT, Media Segments can only be
   added to the end of the Media Playlist.  If the EXT-X-PLAYLIST-TYPE
   value is Video On Demand (VOD), the Media Playlist cannot change.

   If the EXT-X-PLAYLIST-TYPE tag is omitted from a Media Playlist, the
   Playlist can be updated according to the rules in Section 6.2.1 with
   no additional restrictions.  For example, a live Playlist
   (Section 6.2.2) MAY be updated to remove Media Segments in the order
   that they appeared.

RE: PlutoTV, XUMO, M3U/XMLTV interface (appliance) - Added by G Kazaroth about 3 years ago

Hope Sean gets some sleep. I went through his list and found one enhancement request, but the rest are already implemented, so no bugs. I ran some tests and it looks like it is all working.

1) Be able to re-read/ingest the channels: This is available as part of Data Mgmt. You can either reset the database to re-read the channels, or remove the edits and channels which will reset the channel list with edits, so your choice. Either way the channel list is re-read on the next ingest request. I use this constantly and it has never had issues.
2) Change to whatever you ingest: When you ingest a new channel-set from any plugin, it will tag all items in the database ingested and REMOVE any that were not in the new ingest. This means any Channel ID that has been removed for the instance will be gone from the database. Any new Channel ID for the instance will be added with default parameters for the edit fields. My test was to ingest the 7000 m3u file and then to ingest a file with 4 of the 7000 listed. It did remove and add correctly. What is a channel id for a m3u file? primary tags: channelID or tvg-id; then tvg-num or tvg-chno. Most times, I believe, we find a tvg-id tag which is used.
3) Adding a trashcan to channel editor: This is an enhancement, which would impact the Channel Database Schema, so it won't be in 0.9.4. This would add another column so a channel could be enabled/disabled and trash/active. You would then be able to restore from the trashcan. The only way to delete from trashcan would be on ingest when the channel is no longer present.

Thank you Island for the info. Our m3u8 module comes with the playlist type, so it was a two line fix in the m3u8_queue file. Will push out an update shortly. If VOD is set, then it will read all segments, no matter what. If it is unchecked, it will read all, if the playlist type is VOD or it will read based on the setting for not VOD.

Also, had a internet outage. It's this type of stuff that helps with testing. I resolve a number of exceptions caused by the internet being down. That will also be in the updates.

(176-200/960)