Project

General

Profile

DaddyLive, PlutoTV, XUMO, M3U/XMLTV, SamsungTV, Plex, TVGuide interfaces (appliance)

Added by G Kazaroth almost 4 years ago

Cabernet for (Cable Network) version 0.9.12 (4/13/2023)
TVGuide, DaddyLive, PlutoTV, XUMO, M3U/XMLTV, SamsungTV, Plex

Provides a configurable interface from providers to TVHeadend.

  • Direct streaming plugins for DaddyLive, PlutoTV, XUMO
  • EPG supplemented using TVGuide.com data
  • M3U Plugin provides channels for: SamsungTVPlus, Plex, PBS, Stirr, and others
  • From: https://i.mjh.nz/
  • Import your own M3U file

https://github.com/cabernetwork/cabernet/releases
Purpose is to get a service that will take the DaddyLive, PlutoTV or XUMO server stream, clean it and feed it into TVHeadend and other DVRs (Also tested on Emby, JellyFin and Plex). Also it runs on Linux, Windows as a service and Docker. Windows has a installer. Once installed, setup is easy with TVHeadend. Also includes a TVGuide.com plugin to obtain TVGuide data.

1) Setup the automatic IPTV network (recommend new URL is http://[host]:6077/PlutoTV/channels.m3u)

Make sure to set the "Maximum # input streams". PlutoTV is set to a max of 4 and tvheadend uses 2 per tuner during initial screening. Doing a force scan will create the mux and service values. Also, turn View level to Advanced and set the Re-fetch period (mins) to a very large number. TVheadend has a tenancy to cause issues when channels change (Changed Services will not be mapped to channels). The Maximum Timeout is used to wait for a reply during a Forced scan. Recommend keep this low, like 15-20 seconds. Some of the channels may fail, but it is faster than having a high setting and waiting for all channels to scan. Just individually rescan those that failed by setting each mux back to PEND from IDLE.
2) Next you can setup the grabber. I use a URL grabber written in Unix bash and is an extremely small file. It can be found in the github repo at
https://github.com/cabernetwork/cabernet/tree/master/lib/tvheadend/service/Unix called tv_grab_url
Place the grabber file in the same location as the other TVHeadend tv_grab* files, change the permissions to executable and restart TVHeadend. This should allow TVHeadend to pickup the new grabber. While in the grabber list, make sure and disable any OTA grabbers. Stations no longer send this information and will only cause TVHeadend to use a tuner for scanning. Displaying the log window by clicking the three ^ in the bottom right is helpful at this time.

Have the grabber run and populate the EPG data into the EPG Grabber Channels tab. The log should show a quantity of channels were detected.
Pop over to the EPG Grabber tab and disable the OTA grabber cron. Also, update/replace the Internal grabber cron schedule using something like below. The example will pull the TV guide at 6:04am and 5:52pm. Add more if you need. It is recommended to use static cron times.

4 6 * * *
52 17 * * *

3) In the Channel view, select Map all channels.

This will tie the services, EPG and channels together, automatically. After this, re-grab the EPG data. This will populate the EPG tab with shows. For TVH version 4.3, the Number column will auto-populate. For TVH 4.2, you will need to manually add channel numbers.
4) Display the TVGUIDE. This appliance has special features which maps the tvheadend genre, giving colors on tvguides. It also has enhanced guide descriptions and optional additional channel notations if you use many streams. Below is the Kodi tvguide using the pvr.hts plugin.

For Kodi, go to the settings for PVR and turn on the General setting "Use channel numbers from backend".


Replies (960)

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by Sterling Rorden about 3 years ago

First, Thanks @C Island and @G Kazaroth for your help!

The revised autostart.sh above allows Cabernet to run after reboot! Someday I should probably clean up this self-induced mess.

Now on to getting Pluto TV to work.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by Sterling Rorden about 3 years ago

So I still get no results if I do a Force Scan of my Pluto TV network.
I don't know if it matters but my network discovery for the Pluto TV network is set to Disable. If I set it to something else it automatically reverts to Disable when I do a force scan.

My URL is set to: http://192.168.1.83:6077/PlutoTV/channels.m3u

If I put http://192.168.1.83:6077/PlutoTV/channels.m3u in my browser I get a chanels.m3u file that only has #EXTM3U in it.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by G Kazaroth about 3 years ago

Your last report on what is in your config.ini says you have plutotv tabs disabled, so it won't produce anything until you turn it on and request the channel data be updated.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by C Island about 3 years ago

Sterling Rorden wrote:

First, Thanks @C Island and @G Kazaroth for your help!

The revised autostart.sh above allows Cabernet to run after reboot! Someday I should probably clean up this self-induced mess.

Sterling, from what I can tell what you have is perfectly OK for a CoreELEC environment. CoreELEC makes heavy use of python and they lock it down so users can not modify or add packages like cryptography to it. The maintainers of CoreELEC recomend exactly what you have done when there is a need add modules to python (i.e. install entware and then use entware/opkg to install a 2nd python, pip, and python modules all in the /opt directory). Once you have a 2nd python installed you just need to fully qualify all uses of python3 so you are using the correct one (especially in cron or services like autostart.sh).

@G Kazaroth, I probably should have used '.' instead of 'source'. In many shells they are now equvalent but in some (especially non Posix shells) they operate slightly differently. I will update the recomended script in my previous message.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by Sterling Rorden about 3 years ago

@C Island
Good to know I actually did it right, thanks.

@G Kazaroth
I enabled my Pluto TV Instance and that allowed the Muxes to show in TVHeadend. I did a Force Scan but they all failed. I tried changing one from Idle to Pend but it failed again.

I left Maximum Timeout at 15 and Re-fetch period at 60. I also left the PlutoTV Network discovery set to Disable. I haven't setup the Grabber yet if that matters. The channels do show up in the Cabernet PlutoTV channels tab.

This would probably also be a good time for any recommended strategies for changing any settings from default. I mainly use TVHeadend for OTA viewing and recording.

Edit: I tried changing Maximum Timeout to 60 seconds but the Muxes still failed.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by C Island almost 3 years ago

G Kazaroth is the authority but you could try this:

In your cabernet "PlutoTV" provider tab
  1. set 'Enabled' to checked
  2. set tuner_count to 4
In your cabernet "PlutoTV Instance" provider tab
  1. set 'stream_type' to internalproxy
  2. set 'Non-VOD Segments to start' to 3
  3. set 'Enable URL Filtering' to checked
  4. your URL Filter should be ^.*(_ad/|google.com|plutotv_filler|Well_be_right|_AdPod|_Promo).*$
  5. set 'Enable PTS/DTS Resync' to checked
  6. set 'PTS/DTS Resync Type' to ffmpeg
  7. set 'XMLTV Enabled' to checked
In your tvheadend network
  1. set 'Maximum # input streams:' to 2
  2. set 'Maximum timeout (seconds):' to 15
  3. set 'Re-fetch period (mins):' to 60
  4. force a rescan of your PlutoTV network

tvheadend then scans all your muxes (it will take a while). If some fail you can try to change those back to 'PEND' again (I had to do this a few times on some muxes)

During the forced scan you should check your log (/storage/tvheadend-locast/out.log) for errors. If you still have cabernet logging enabled and Level set to DEBUG in 'System Log Handler', you should see cabernet successfully tuning to channels and you should see log messages showing segments being 'Added', 'Serving' and 'Removed'.

You might also need to check your autostart.sh service for errors with this command

systemctl status kodi-autostart.service -l --no-pager

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by Sterling Rorden almost 3 years ago

I made your suggested setting changes and restarted my TVHeadend box. The Muxes scan still fails.

/storage/tvheadend-locast/out.log doesn't exist. When I attempted to update Cabernet it said it couldn't update because it saw that file in the folder. So I deleted the file and removed the command to create the log from my autostart.sh file which allowed me to update Cabernet. I can temporarily put that command back into the autostart.sh file if that will help.

I entered the "systemctl status kodi-autostart.service -l --no-pager" command and got the following:

CoreELEC:~ # systemctl status kodi-autostart.service -l --no-pager
● kodi-autostart.service - Kodi user autostart script
Loaded: loaded (/usr/lib/systemd/system/kodi-autostart.service; disabled; vendor preset: disabled)
Active: active (exited) since Sun 2021-11-07 17:08:55 PST; 51min ago
Process: 5162 ExecStart=/bin/sh -c . /etc/profile; exec /bin/sh /storage/.config/autostart.sh (code=exited, status=0/SUCCESS)
Main PID: 5162 (code=exited, status=0/SUCCESS)
Tasks: 27 (limit: 1735)
Memory: 98.7M
CGroup: /system.slice/kodi-autostart.service
├─5230 /opt/bin/python3 /storage/tvheadend-locast/tvh_main.py
├─5545 /opt/bin/python3 /storage/tvheadend-locast/tvh_main.py
└─5554 /opt/bin/python3 /storage/tvheadend-locast/tvh_main.py

Nov 07 17:56:31 CoreELEC sh7832: self.m3u8_q = M3U8Queue(_config, _channel_dict)
Nov 07 17:56:31 CoreELEC sh7832: File "/storage/tvheadend-locast/lib/streams/m3u8_queue.py", line 77, in init
Nov 07 17:56:31 CoreELEC sh7832: self.pts_resync = PTSResync(_config, self.config_section, _channel_dict['uid'])
Nov 07 17:56:31 CoreELEC sh7832: File "/storage/tvheadend-locast/lib/streams/pts_resync.py", line 37, in init
Nov 07 17:56:31 CoreELEC sh7832: self.ffmpeg_proc = self.open_ffmpeg_proc()
Nov 07 17:56:31 CoreELEC sh7832: File "/storage/tvheadend-locast/lib/streams/pts_resync.py", line 112, in open_ffmpeg_proc
Nov 07 17:56:31 CoreELEC sh7832: ffmpeg_process = subprocess.Popen(ffmpeg_command,
Nov 07 17:56:31 CoreELEC sh7832: File "/opt/lib/python3.9/subprocess.py", line 951, in init
Nov 07 17:56:31 CoreELEC sh7832: File "/opt/lib/python3.9/subprocess.py", line 1821, in _execute_child
Nov 07 17:56:31 CoreELEC sh7832: FileNotFoundError: [Errno 2] No such file or directory: 'ffmpeg'

It looks like it has a problem finding ffmpeg.

Thanks for your help!

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by C Island almost 3 years ago

on CoreELEC you can install ffmpeg using entware

opkg install ffmpeg

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by C Island almost 3 years ago

Moving forward, it might be good to have a log file. Maybe just put it outside of the tvheadend-locast directory. This will put the log in the /storage directory and it will be named tvheadend-locast.log

#!/bin/sh
(
sleep 10
. /opt/etc/profile
cd /storage/tvheadend-locast
/opt/bin/python3 /storage/tvheadend-locast/tvh_main.py
) > /storage/tvheadend-locast.log 2>&1 &

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by G Kazaroth almost 3 years ago

Cabernet is also setup to create and manage a log file. By default it does a 10 file rotation with each file being 10MB. To enable, go to Settings > Logging > File Log Handler. Enable it and change the log level to what ever you need. The log file will appear in the cabernet/data/log folder.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by Sterling Rorden almost 3 years ago

OK, I've installed ffmpeg per the instructions above. I assumed (I guess incorrectly) since Kodi uses ffmpeg for playback I didn't need to install it.

I made some progress. 2 services showed up during the scan and I was able to map them. I tried to play them but they would barely play then I get a Tvheadend HTSP Client no signal available.

I've enabled the recommended logging in both the autostart.sh file and within Cabernet at the Debug level (should I be running both of these or are they redundant?).

I've attached the logs.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by G Kazaroth almost 3 years ago

Cabernet can stream to either logs. tvheadend-locast.log is controlled by the first tab in the settings > logging area while cabernet.log is controlled by the second tab. So, it is flexible. Normally, the first tab is output to stdout/stderr, which goes to syslog, while the second tab goes to an app specific log.

The issue you are having is the multiprocessing queues class does not contain the standard python method qsize(). I will see what is needed to get around this issue. According to internet, some python installs do not have that method and throws a NotImplementedError.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by G Kazaroth almost 3 years ago

Try 0.9.6.4. I have removed the use of qsize()

In regard to not finding ffmpeg and using Kodi's version, you had 3 choices.
  1. Install ffmpeg so that the PATH environment variable can find it
  2. Update the PATH environment variable to include the location of the ffmpeg executable in the Kodi install
  3. Update the Cabernet ffmpeg path from a relative path which uses $PATH to an absolute path to the Kodi executable

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by Sterling Rorden almost 3 years ago

I updated to 0.9.6.4, rebooted, did a scan, and mapped an additional 56 services. However, a majority of the muxes that failed the scan will fail again when I change the status from IDLE to PEND. The channels that do get scanned and mapped seem to play properly.

I've attached the latest cabernet.log and tvheadend-locast.log. They seem to be complaining about not finding _cffi_backend.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by G Kazaroth almost 3 years ago

From the cryptography site, here is the issue regarding your error
https://github.com/pyca/cryptography/issues/3703

also something similar...
https://foss.heptapod.net/pypy/cffi/-/issues/451

Does not look like they have a good resolution and seems to say the cffi module was not installed or installed correctly. I recommend doing some debugging.
Go into python3 by typing /opt/bin/python3
Then type "import _cffi_backend"
That worked on my computer
Also try
"from cryptography.hazmat.bindings._openssl import ffi, lib"
That also worked.

To quit, type "quit()"

You can try to install the dependency (cffi) (it should have been installed when you installed crypto) with whatever command you use to install python modules. then try the tests again.

Island may be able to shed some light on this.

Last piece of info. There is a pdf that claims having an import in cabernet could fix this. You can try to mod the file and see if that works.
Go to cabernet/lib/streams/m3u8_queue.py and edit the file
At the top near "import datetime" add the line
"import _cffi_backend"
Then restart the service and see if that works...

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by Sterling Rorden almost 3 years ago

both of those gave me:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named '_cffi_backend'

I tried to install cffi and got:

CoreELEC:~ # pip install cffi
Requirement already satisfied: cffi in ./.opt/lib/python3.9/site-packages (1.15.0)
Requirement already satisfied: pycparser in ./.opt/lib/python3.9/site-packages (from cffi) (2.20)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

I believe that is the same warning when I installed the cryptography package as I remember.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by G Kazaroth almost 3 years ago

They claim you need to include -U... pip install -U cffi

Also, I assume you are making sure it is installing into the correct python install...

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by Sterling Rorden almost 3 years ago

I tried with the -U and got the same message.

Putting "import _cffi_backend" in cabernet/lib/streams/m3u8_queue.py caused Force Scan not to work and made Cabernet unreachable so I took it back out.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by G Kazaroth almost 3 years ago

Recommend waiting for Island to determine why your cffi is corrupted.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by C Island almost 3 years ago

one sec. i have a solution. just need to test it a bit....

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by C Island almost 3 years ago

Yes, your installation is missing some key modules. The following command should fix things:

opkg install python3-cryptography

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by C Island almost 3 years ago

As far as I can tell when you used 'pip install cryptography' to install cryptography some key modules that CoreELEC needs were not installed and you were left in a bad state. The python3-cryptography package was created to properly install python3 with cryptography in an entware environment like CoreELEC. They advertize it as:

python3-cryptography - 3.4.7-1 - cryptography is a package which provides cryptographic recipes and primitives to Python developers.  Our goal is for it to be your "cryptographic standard library". It supports Python 2.6-2.7, Python 3.3+, and PyPy 2.6+.

If in the future you need to wipe out your current CoreELEC's entware environment including python3, pip, cryptography, ffmpeg, etc installs and start from scratch I think doing as much of the install as possible with the 'opkg' command is the way to go. The following should work:

# remove entware including all installed packages/modules/etc
rm -rf /opt/* 
#  
# install entware (reboot when prompted)
installentware
#
# install ffmpeg
opkg install ffmpeg
#
# install python3 with cryptography (including ALL needed other modules)
opkg install python3-cryptography
#
# at this point the "from cryptography.hazmat.bindings._openssl import ffi, lib" python command will work
# however, I did not install and test cabernet (I do not use my CoreELEC system for cabernet)
#
# install pip (I think installing pip is optional becuase opkg installed all that was needed)
# I suggest when you want to install anything you check if you can do it using the 'opkg' command before trying pip
# FYI - the command 'opkg list' will show you all packages you can install with opkg
opkg install python3-pip
#
# to upgrade all your pip modules to the most up to date versions use this command
# I am not sure if upgrading all packages to the most current version is a good idea or not. 
# I tried it and it worked for me but I did not install and test cabernet 
pip3 list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip3 install -U 

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by Sterling Rorden almost 3 years ago

Well, I did the "opkg install python3-cryptography" command and rebooted. I'm still having the problem.

Do you think it is time to nuke my current CoreELEC entware environment and start over per your commands above?

I've attached the cabernet.log file.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by C Island almost 3 years ago

Yes, I think it is best to follow my suggested commands to nuke entware and start again. I suggest you try to run cabernet before you install python3-pip or use any pip or pip3 commands.

Do me a favor and post the results of the following commands before you nuke your environment

opkg list-installed
pip list
echo "from cryptography.hazmat.bindings._openssl import ffi, lib"|python3

Then after you nuke entware and install entware, ffmpeg and python3-cryptography run these commands and post the results as well

opkg list-installed
echo "from cryptography.hazmat.bindings._openssl import ffi, lib"|python3

If the echo/python3 command does not produce an error give cabernet a go and see if it works.

RE: PlutoTV, USTVGO, XUMO, M3U/XMLTV, SamsungTV, Plex interfaces (appliance) - Added by Sterling Rorden almost 3 years ago

Sorry, I nuked it before I fully read your message. But here is the after information:

CoreELEC:~ # opkg list-installed
ca-certificates - 20210119-1
entware-opt - 227000-3
entware-release - 1.0-2
entware-upgrade - 1.0-1
findutils - 4.8.0-1
grep - 3.6-1a
libbz2 - 1.0.8-1a
libc - 2.27-11
libffi - 3.4.2-1
libgcc - 8.4.0-11
libopenssl - 1.1.1k-1
libpcre - 8.45-1
libpthread - 2.27-11
libpython3 - 3.9.6-2
librt - 2.27-11
libssp - 8.4.0-11
libstdcpp - 8.4.0-11
libtirpc - 1.3.2-1
libuuid - 2.37-1
locales - 2.27-9
opkg - 2021-06-13-1bf042dd-1
python3-base - 3.9.6-2
python3-cffi - 1.14.6-1
python3-cryptography - 3.4.7-1
python3-email - 3.9.6-2
python3-light - 3.9.6-2
python3-openssl - 3.9.6-2
python3-ply - 3.11-2
python3-pycparser - 2.20-5
python3-six - 1.16.0-1
python3-urllib - 3.9.6-2
terminfo - 6.2-2a
zlib - 1.2.11-3
zoneinfo-asia - 2021a-1
zoneinfo-europe - 2021a-1

Echo didn't return anything:

CoreELEC:~ # echo "from cryptography.hazmat.bindings._openssl import ffi, lib"|python3
CoreELEC:~ #

The Cabernet web page won't open. I didn't install pip.

Below is the output from the tvheadend-locast.log:

Traceback (most recent call last):
File "/storage/tvheadend-locast/tvh_main.py", line 10, in <module>
from lib import main
File "/storage/tvheadend-locast/lib/main.py", line 21, in <module>
import logging
ModuleNotFoundError: No module named 'logging'

(376-400/960)