Tvheadend

Hi

Could someone confirm if I can use this docker image to only add PlutoTV stations to tvheadend (or Jellyfin)?

Also would it be possible to limit the channels to only the ones I am interested in. Currently use this project, https://github.com/evoactivity/PlutoIPTV, to generate a playlist and epg for the PlutoTV channels I want, but there are issues with ad breaks that it looks like your solution will resolve.

Thanks in advance.

G Kazaroth wrote:

How does PTS Filtering work? It determines the PTS of the main show being recorded and anything out of scope (too large or too small) is removed. On normal stations, the PTS continues to grow all day and resets at midnight. So, recording a show early is not an issue since the PTS just keeps going as if all shows are tied together. HOWEVER, these streaming services do things differently. The PTS changes for each show, in many cases. (DABL may not be that way since it is an actual station). The PTS Filter still needs to understand what PTS is for the real show. If you start the show early, then the PTS will think the previous show is the real PTS and when the new show starts, it will think the show should be removed. To keep this from happening, it will be important not to start recording until after the show starts.

To be honest I did not understand much of your technical explanation in this post, but I had PTS Filtering enabled once and somehow it got turned off (I probably unchecked the box for some reason but don;t remember doing it), anyway that would explain why I could not seek within anything I recorded last night. I have turned it back on now so will see what happens next time I try to record something.

But here is the thing, although I am getting very close to the beginning of the programs, it's still cutting a little off. Normally, in my default recording profile, I start recording 1 minute early and end 3 minutes late, this is to accommodate (over-the-air) shows that may jump the gun a little on the starting time, or may drag out a little late. Since your program wants to start a minute late, the two nearly cancel each other out, and the recording starts just seconds after the show starts. Which is fine in most cases, but there is something about it that makes me think that there ought to be a way to ignore and/or rewrite the PTS. I suspect this is what ffmpeg does if you use it on a show that has been previously been recorded. But it also does something else that causes visible glitches in the show itself.

I used ffmpeg to make the things I recorded last night seekable, and it worked, but threw several other errors. Examples were:

PES packet size mismatch
AAC bitstream not in ADTS format and extradata missing
Packet corrupt
Non-monotonous DTS in output stream (these were the most common)

There weren't a lot of these; they only filled about one screen on my display. And I suspect they occurred mainly at the transitions to/from commercial breaks.

I don't understand much about video, especially the technical aspects, but I just wonder if it is somehow possible to just throw away the timestamps that are in the streams and just apply new ones to the video in the order the segments are received. In other words, say we don't care what the PTS is in any part of the stream, we just take the first segment we receive and renumber it to 1, the next one to 2, and so on (or start with a higher number if necessary, maybe one corresponding to the current time although maybe that would be an issue if the program starts before midnight and then goes past it?). I do not know if that's what ffmpeg does, and I don't know how difficult it would be to do that, but if it could be done I'd think it would solve several of these problems, maybe.

Robert Coupee wrote:

Hi

Could someone confirm if I can use this docker image to only add PlutoTV stations to tvheadend (or Jellyfin)?

Yes you can.

Also would it be possible to limit the channels to only the ones I am interested in. Currently use this project, https://github.com/evoactivity/PlutoIPTV, to generate a playlist and epg for the PlutoTV channels I want, but there are issues with ad breaks that it looks like your solution will resolve.

There is a channel editor in the Cabernet web-based interface, and you simply uncheck the channels you don't want and click the save button.

(I should mention that I do not use Docker, but don't think there's any difference in the way that one operates).

We have a number of people that use docker. I do use it every so often just to test it and make sure nothing changed. People use Cabernet with TVHeadend, JellyFin, Emby and Plex. Could be more that is compatible, but I do know those work. I see Sean answered the rest of your questions.

I have a few updates I am tracking. The first is almost ready.
1) Looks like plutotv periodically takes too long to provide the video stream section. The result is a missing 6 second section. I am moving the "get_video" to a separate process, so it can get the streams as fast as possible. This also has a feature where each active channel is a separate process. Basically, you should be able to run as many tuners as Plutotv will allow. Cabernet will just keep running. This is in final testing and should be out today.
2) I am seeing problems with the EPG. Not sure what it is, but in the morning the EPG is very short with "No Information Available" across some channels. I will have to take a look and see what the issue is.

New version has been posted. This should clear up the times when plutotv is not responding quickly, so missing 6 second sections of the video should no longer happen.

G Kazaroth wrote:

New version has been posted. This should clear up the times when plutotv is not responding quickly, so missing 6 second sections of the video should no longer happen.

Unfortunately now it appears anything I record won't play - when I attempt to play it in Kodi, it either immediately returns to the recordings list, or a spinner come up and spins for a while and then returns to the recordings list. Reverting back to 0.9.2.3 allows recordings made with that version to be played.

I check my recording of Ghost; it had an issue as well. A number of video sections were double transmitted. My software is more efficient which can have good and bad. It looks like PlutoTV added a section to transmit, Cabernet did that, then it removed it from its queue, so Cabernet did that. Then the error came when PlutoTV added it back onto the queue (not sure why). For now I have a band-aid to detect and not transmit the duplicate. Need to do more research as to why PlutoTV is doing this.

Also, found the EPG issue including the date aired. Still need to do rigorous testing. Will work on it after church.

Found out the retransmit of the video section has little to do with timing. Recording this morning showed section 1172 came in before the ads started. After the ads finished (about 3 min later...) Section 1172 was again added to the queue by PlutoTV. This means it is a PlutoTV bug and Cabernet needs to take care of the dups. Patch has been applied and version 0.9.2.6 has those changes. Watching some recording now. This morning 2 hours worked without issue.

Hi

So I am trying to get this working for PlutoTV streams. I am running it on a Raspberry Pi with Ubuntu Server 20.04 32 installed. As the docker image is not compatible with the Pi Architecture I am currently running it manually using the command line: python3 tvh_main.py. This runs and I can access the cabernet website. However the "Refresh PlutoTV Channels" Scheduled Task is constantly running and the following is seen on the command line:

2021-09-12 18:58:46,610-ERROR:decorators JSONError in function get_token(): Expecting value: line 1 column 1 (char 0)
2021-09-12 18:58:46,610-ERROR:authenticate Invalid Locast Login Credentials. Disabling instance
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/home/ubuntu/cabernet/lib/schedule/scheduler.py", line 95, in run
self.exec_trigger(trigger)
File "/home/ubuntu/cabernet/lib/schedule/scheduler.py", line 132, in exec_trigger
self.call_trigger(_trigger)
File "/home/ubuntu/cabernet/lib/schedule/scheduler.py", line 152, in call_trigger
call_f()
File "/home/ubuntu/cabernet/lib/plugins/plugin_obj.py", line 106, in refresh_channels
self.refresh_it('Channels', _instance)
File "/home/ubuntu/cabernet/lib/plugins/plugin_obj.py", line 128, in refresh_it
instance.refresh_channels()
File "/home/ubuntu/cabernet/lib/plugins/plugin_instance_obj.py", line 38, in refresh_channels
self.channels.refresh_channels()
File "/home/ubuntu/cabernet/lib/plugins/plugin_channels.py", line 49, in refresh_channels
self.db.save_channel_list(self.plugin_obj.name, self.instance_key, ch_dict, \
File "/home/ubuntu/cabernet/lib/db/db_channels.py", line 172, in save_channel_list
self.add(DB_CHANNELS_TABLE, (
File "/home/ubuntu/cabernet/lib/db/db.py", line 72, in add
cur = self.sql_exec(sqlcmd, _values)
File "/home/ubuntu/cabernet/lib/db/db.py", line 60, in sql_exec
return DB.conn[self.db_name][threading.get_ident()].execute(_sqlcmd, _bindings)
sqlite3.InterfaceError: Error binding parameter 7 - probably unsupported type.

I observed the same behaviour when running on the latest install of Raspberry Pi OS.

Any ideas?

Thanks in advance.

It is probably your config.ini. Since locast is no more, you should have nothing in the config.ini that has anything to do with locast. The way the product works right now is if you want to enable a plugin, you add

[pluginname_instancename]
label = any name you want. it will appear in the config page

So as an example, you can add PlutoTV by adding the following to the config.ini file.

[plutotv_default]
label = PlutoTV Instance

I you still have an issue when you clean up the config.ini, then try to delete the database files in the ./data/db/ folder and see if that works.

Interesting that you were able to cause an exception. I will take a look and see if I can have it update gracefully.

FYI... The latest version is having issues with the tuner subprocesses when you do a Restart from the scheduler. I am working the issue; otherwise, it works fine. If you get a "OSError: [Errno 98] Address already in use", then you have a python process still running.

Robert, you may shed some light on PlutoIPTV. Based on some of the issues, it looks like it somehow grabs 24 or 48 hours of epg from PlutoTV. Every time I do a request (even manually) I do not get more than 12 hours of programming. If it is possible, I would like to make sure we provide it, but I am not sure it can.

ok, I'm done with the fixes. Back to working the PTS/DTS updates.

G Kazaroth wrote:

Robert, you may shed some light on PlutoIPTV. Based on some of the issues, it looks like it somehow grabs 24 or 48 hours of epg from PlutoTV. Every time I do a request (even manually) I do not get more than 12 hours of programming. If it is possible, I would like to make sure we provide it, but I am not sure it can.

I believe PlutoTV only provide 12 hours worth of EPG, but they refresh it every 6/7 hours. So I guess your PlutoTV plugin should by default refresh every 6 hours?

G Kazaroth wrote:

It is probably your config.ini. Since locast is no more, you should have nothing in the config.ini that has anything to do with locast. The way the product works right now is if you want to enable a plugin, you add

[pluginname_instancename]
label = any name you want. it will appear in the config page

So as an example, you can add PlutoTV by adding the following to the config.ini file.

[plutotv_default]
label = PlutoTV Instance

I you still have an issue when you clean up the config.ini, then try to delete the database files in the ./data/db/ folder and see if that works.

Interesting that you were able to cause an exception. I will take a look and see if I can have it update gracefully.

FYI... The latest version is having issues with the tuner subprocesses when you do a Restart from the scheduler. I am working the issue; otherwise, it works fine. If you get a "OSError: [Errno 98] Address already in use", then you have a python process still running.

Thanks for the advice. Did a git fetch and pull and got your latest update. However the error still occurred. It went away if I unticked the "Import channel groups" option under the "PlutoTV" Tab.

The "Refresh PlutoTV Channels" job runs and a channels.m3u file is generated. However the "Channel Editor" -> "PlutoTV" Tab states "NO CHANNEL DATA FOUND" and the following error is seen:

Exception happened during processing of request from ('192.168.1.102', 54506)
Traceback (most recent call last):
File "/usr/lib/python3.8/socketserver.py", line 316, in handle_request_noblock
self.process_request(request, client_address)
File "/usr/lib/python3.8/socketserver.py", line 347, in process_request
self.finish_request(request, client_address)
File "/usr/lib/python3.8/socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/home/ubuntu/cabernet/lib/clients/web_admin.py", line 200, in init
super(CustomWebAdminHttpHandler, self)._init__(*args, **kwargs)
File "/home/ubuntu/cabernet/lib/clients/web_admin.py", line 78, in init
super().__init__(*args)
File "/usr/lib/python3.8/socketserver.py", line 747, in init
self.handle()
File "/usr/lib/python3.8/http/server.py", line 427, in handle
self.handle_one_request()
File "/usr/lib/python3.8/http/server.py", line 415, in handle_one_request
method()
File "/home/ubuntu/cabernet/lib/clients/web_admin.py", line 93, in do_GET
if getrequest.call_url(self, self.content_path):
File "/home/ubuntu/cabernet/lib/common/decorators.py", line 178, in call_url
self.url2func[_name](_webserver, *args, **kwargs)
File "/home/ubuntu/cabernet/lib/clients/channels/channels_form_html.py", line 36, in get_channels_form_html
form = channels_form.get(name, _sort_col, _sort_dir, filter_dict)
File "/home/ubuntu/cabernet/lib/clients/channels/channels_form_html.py", line 79, in get
return ''.join([self.header,self.body])
File "/home/ubuntu/cabernet/lib/clients/channels/channels_form_html.py", line 384, in body
self.form,
File "/home/ubuntu/cabernet/lib/clients/channels/channels_form_html.py", line 272, in form
t = self.table
File "/home/ubuntu/cabernet/lib/clients/channels/channels_form_html.py", line 331, in table
row = ''.join([
TypeError: sequence item 41: expected str instance, list found
---------------------------------------

Any ideas?

BTW: Thanks for all of your work on this tool!

You must be a tester since you find the strangest exceptions. I will fix this one so it does not crash and allows you to fix the Group problem in the editor. "sequence item 41" is cryptic for the Group column. It says you entered a list of values and not a single word in one of the group fields. This caused the exception to occur. If you mouse over the title columns, it has a tool tip. On group it states a single word is required. If you did not enter a value into the group field, let me know. That would mean you have a different list from PlutoTV than I do and we will need to somehow understand how you got a different channel list. Cabernet has a group lookup that translates all the bad group names that Kodi throws up on to simple names. If you have a different list, it is possible other PlutoTV groups appeared that cabernet did not expect.

You will probably need to do a clean channel to fix the problem. There are two ways (both do the same thing). One is to remove the ./data/db/channels.db file and restart the service. The other is to go into the "Data Management" section, Reset Channel Data
- Reset Edits = Yes
- Plugin = PlutoTV
- Click Reset
then go to Scheduler and click on Refresh PlutoTV Channels
The second method does not require a app restart.

I have been around testers long enough that some of their "skill" must have rubbed off!

I think I must have a different channel list: I am based in the UK and PlutoTV has some UK Specific programming.

I have attached the channels.m3u output.

Thanks.

Thanks for the m3u file. It does look like you have a different channel list. I would need to see it with the groups list enabled. The one you sent basically has no groups from PlutoTV. I also noticed that some of the channel names have (+ . '). If you have any issues, we can setup something to filter out special characters from channel names.

Last thing is the channel numbers. There are seem to be a bunch of channel "0" channels. They do all have unique ids, but you may think about giving them unique channel numbers. It is not required.

Couple of notes. If you change the logging from the default WARNING to INFO, you will see the messages about missing groups in the log when the channels are imported from PlutoTV. Currently the Cabernet PlutoTV group converter list is:

    'Classic TV':     groups['CLASSICS'],
    'Comedy':         groups['COMEDY'],
    'Crime':          groups['MYSTERY'],
    'En Español':     groups['SPANISH'],
    'Entertainment':  groups['ENTERTAINMENT'],
    'Explore':        groups['DOCUMENTARIES'],
    'Gaming + Anime': groups['GAMING'],
    'Home + DIY':     groups['DIY'],
    'Kids':           groups['KIDS'],
    'Local':          groups['NEWS'],
    'Movies':         groups['MOVIES'],
    'Music':          groups['MUSIC'],
    'News + Opinion': groups['NEWS'],
    'Pluto TV':       groups['CLASSICS'],
    'Reality':        groups['REALITY'],
    'Sports':         groups['SPORTS'],
    'Vizio':          groups['MOVIES']

Hi

So I do indeed see the following in the logs re: missing groups:

2021-09-13 19:41:42,123-INFO:web_handler [192.168.1.49] "GET /api/schedulehtml HTTP/1.1" 200 -
2021-09-13 19:41:42,143-INFO:web_handler [192.168.1.49] "GET /modules/scheduler/scheduler.css HTTP/1.1" 200 -
2021-09-13 19:41:42,146-INFO:web_handler [192.168.1.49] "GET /modules/scheduler/scheduler.js HTTP/1.1" 200 -
2021-09-13 19:41:42,227-INFO:channels Missing PlutoTV category translation for: Curiosity
2021-09-13 19:41:42,419-INFO:channels Missing PlutoTV category translation for: Curiosity
2021-09-13 19:41:42,812-INFO:channels Missing PlutoTV category translation for: Life + Style
2021-09-13 19:41:42,893-INFO:channels Missing PlutoTV category translation for: Curiosity
2021-09-13 19:41:42,978-INFO:channels Missing PlutoTV category translation for: Motor
2021-09-13 19:41:43,259-INFO:channels Missing PlutoTV category translation for: Life + Style
2021-09-13 19:41:43,349-INFO:channels Missing PlutoTV category translation for: Curiosity
2021-09-13 19:41:43,697-INFO:channels Missing PlutoTV category translation for: Curiosity
2021-09-13 19:41:43,887-INFO:channels Missing PlutoTV category translation for: Drama
2021-09-13 19:41:43,981-INFO:channels Missing PlutoTV category translation for: Curiosity
2021-09-13 19:41:44,311-INFO:channels Missing PlutoTV category translation for: Samsung
2021-09-13 19:41:44,478-INFO:channels Missing PlutoTV category translation for: Curiosity
2021-09-13 19:41:44,570-INFO:channels Missing PlutoTV category translation for: Curiosity
2021-09-13 19:41:44,788-INFO:channels Missing PlutoTV category translation for: Drama
2021-09-13 19:41:44,868-INFO:channels Missing PlutoTV category translation for: Life + Style
2021-09-13 19:41:45,304-INFO:channels Missing PlutoTV category translation for: Samsung
2021-09-13 19:41:45,560-INFO:channels Missing PlutoTV category translation for: Life + Style
2021-09-13 19:41:45,640-INFO:channels Missing PlutoTV category translation for: Motor
2021-09-13 19:41:45,811-INFO:channels Missing PlutoTV category translation for: Drama
2021-09-13 19:41:45,897-INFO:channels Missing PlutoTV category translation for: Drama
2021-09-13 19:41:46,371-INFO:channels Missing PlutoTV category translation for: Drama
2021-09-13 19:41:46,727-INFO:channels Missing PlutoTV category translation for: Life + Style
2021-09-13 19:41:46,808-INFO:channels Missing PlutoTV category translation for: Curiosity
2021-09-13 19:41:46,897-INFO:channels Missing PlutoTV category translation for: Curiosity
2021-09-13 19:41:47,118-INFO:channels Missing PlutoTV category translation for: Paranormal

Regards

Robert

Also attached is the .m3u8 file from the mechanism I currently use to get my channel list. It does include "Groups".

0.9.2.11 Tons of changes with PTS Resync.

First, I removed the previous versions because cabernet was not detecting the upgrade from github. It's caused by going from version ...9 to ...11 (.9 is greater than .11). So, 0.9.2.3 is removed, but the upgrade should appear on the Home Page. The new version corrects the upgrade issue.

Cabernet has a number of fixes from Coupee including adding the UK PlutoTV groups and fixing the exception for group lists.

The PTS Resync is part of internalproxy only. PTS Resync is now available in the configuration list for PlutoTV. You can turn on and off PTS Resync and URL filtering, independently. (as well as PTS Filtering). If you do not like the dead spaces, then keep URL Filtering off. The test I ran was with URL Filtering off (so it recorded the commercials) and PTS Resync on. The result was a pretty clean playback. After one hours of play, it lost about 5 seconds of PTS (so it went forward skip+5sec and back skip less 5sec). Since cabernet should not be dependent on ffmpeg, a version of PTS Resync without ffmpeg will be created in the future.

One of the big changes also includes the cleanup of the internalproxy streaming threads and processes. Each process and thread now runs more efficiently and has better timing to keep from loosing stream sections.

Robert Coupee wrote:

G Kazaroth wrote:

Robert, you may shed some light on PlutoIPTV. Based on some of the issues, it looks like it somehow grabs 24 or 48 hours of epg from PlutoTV. Every time I do a request (even manually) I do not get more than 12 hours of programming. If it is possible, I would like to make sure we provide it, but I am not sure it can.

I believe PlutoTV only provide 12 hours worth of EPG, but they refresh it every 6/7 hours. So I guess your PlutoTV plugin should by default refresh every 6 hours?

The scheduler is one stop shop for scheduling tasks. If you go there and click on the "Refresh PlutoTV EPG" task, it should bring up the triggers, which should say:

At startup
Every 110 minutes with random maximum added time of 20 minutes

Thank you for the update. I have installed it and will test it.

What do you recommend for the "PTS/DTS Resync Type" setting? It defaults to 'ffmpeg' but can be set to 'internal' or 'default'....

So far, this version is working well above expectations. It does use ffmpeg; the log at info level will let you know when you start up a tuner that it is in use and using ffmpeg. The internal version (as stated above)

"a version of PTS Resync without ffmpeg will be created in the future."

If you set the PTS Resync to internal, the log will throw a warning that it is not implemented and it will do nothing.
So, to use PTS Resync currently requires ffmpeg. On unix, it assumes that $PATH is set to pickup the executable. On Windows, the path is set in the config file. A config entry can be used to defined the absolute path to the executable, if needed, but those are the defaults.
The "default" setting for PTS Resync will default back to ffmpeg, which is currently the only type that is implemented.

Some details. If you look at the process list (unix: ps -ef) you will normally see three python3 processes running (these are: the webadmin port 6077 server, the webtuner port 5004 server and the main process running the scheduler). If you start up a tuner, then an additional python3 process will start. If you have PTS Resync enabled, then you will also see an ffmpeg process running as well. Both of these processes will terminate when the tuner stops.

Just wanted to say thanks for this, I have enabled PTS resync and now it works much better. I can now actually record the entire show (including the beginning) and still skip around within the show and not have it go back to the beginning!

As for using ffmpeg, the interactions between ffmpeg and Tvheadend are always a wonder to behold. Sometimes ffmpeg fixes problems but occasionally it causes them.

For example there are a small group of streams from a particular source; I don't want to name that source specifically because I don't know anything about the source or the intended usage but I will just say that a few of the streams from that source (though not necessarily the most interesting ones) appear in the huge list at https://iptv-org.github.io/iptv/index.m3u , anyway the weird thing about those streams is this:

- If you play them in VLC, they play just fine, no problems at all.
- If you put the exact same URL into Tvheadend (in an IPTV network), it will play BUT what you see might be anywhere from a few minutes to several hours behind. Yes, it appears that Tvheadend has invented a form of time travel to the past! And that would be great if there were a fixed amount of time difference between when the show was aired and when you start watching, but there isn't, it seems pretty much totally random. And that ONLY happens in Tvheadend!
- But if you put that same URL into a ffmpeg pipe such as "pipe:///usr/local/bin/ffmpeg -loglevel fatal -i https://address_of_stream.m3u8 -c copy -metadata service_provider=Stream_Name -metadata service_name=Stream_Name -f mpegts pipe:1" and put THAT into Tvheadend, then it plays what is currently airing, same as in VLC, however in Tvheadend the audio and video can be significantly out of sync on some streams, and if you try to play those streams live in Kodi it sometimes thinks they are audio only and doesn't show the video. But if you record such a stream then Kodi will play it just fine, except that the audio and video might still be out of sync.

If there were a way to send PM's on this board I would offer to send you a link so you could see what I mean, if you are at all interested, but as far as I can tell there's no way to send PM's here. But I just thought I'd mention that because both Tvheadend and ffmpeg can be a little weird sometimes, as I'm sure you have discovered.

Finishing up XUMO. Its kinda like the movie... The Good, The Bad, and The Ugly.

1) Good part is it provides a 24 hour EPG. Ugly part is it is based on UTC within a single day. Not bad until you realize midnight UTC in the US is in the evening, so you do not get the epg update until then. As an example, if you are Central Time, then the EPG will end at 7:00pm (with no data after that) and you can update the EPG between 7p and 8p to get the full next day's programming (which for central would be from 7p to 7p the next day.

2) Channels are either live or VOD. The bad is that the live ones do not have programming data. The good is that the VOD channels will play whatever is currently playing from the start, even if you start it at the end! The ugly is if you start the recording 3 minutes prior, you will get the previous movie instead of the one you wanted. Not sure how this plays into TVH, but it will be interesting.

3) Some of the channels have a tiny resolution, (less than 480) while others have 1080. From what I have seen, the VOD channels will change resolutions between movies. Cabernet will probably have a fit when that happens. Have not tested what happens at the end of a movie on a VOD channel.

4) Have not seen any commercials, but I am sure they are there. The URL filter is easy to manage, so I may leave it blank and allow users to tune it.

I have it into TVH and playing fine. Working on cleaning up the code and then probably release something on Thursday.