Tvheadend

First, Thanks @C Island and @G Kazaroth for your help!

The revised autostart.sh above allows Cabernet to run after reboot! Someday I should probably clean up this self-induced mess.

Now on to getting Pluto TV to work.

So I still get no results if I do a Force Scan of my Pluto TV network.
I don't know if it matters but my network discovery for the Pluto TV network is set to Disable. If I set it to something else it automatically reverts to Disable when I do a force scan.

My URL is set to: http://192.168.1.83:6077/PlutoTV/channels.m3u

If I put http://192.168.1.83:6077/PlutoTV/channels.m3u in my browser I get a chanels.m3u file that only has #EXTM3U in it.

Your last report on what is in your config.ini says you have plutotv tabs disabled, so it won't produce anything until you turn it on and request the channel data be updated.

Sterling Rorden wrote:

First, Thanks @C Island and @G Kazaroth for your help!

The revised autostart.sh above allows Cabernet to run after reboot! Someday I should probably clean up this self-induced mess.

Sterling, from what I can tell what you have is perfectly OK for a CoreELEC environment. CoreELEC makes heavy use of python and they lock it down so users can not modify or add packages like cryptography to it. The maintainers of CoreELEC recomend exactly what you have done when there is a need add modules to python (i.e. install entware and then use entware/opkg to install a 2nd python, pip, and python modules all in the /opt directory). Once you have a 2nd python installed you just need to fully qualify all uses of python3 so you are using the correct one (especially in cron or services like autostart.sh).

@G Kazaroth, I probably should have used '.' instead of 'source'. In many shells they are now equvalent but in some (especially non Posix shells) they operate slightly differently. I will update the recomended script in my previous message.

@C Island
Good to know I actually did it right, thanks.

@G Kazaroth
I enabled my Pluto TV Instance and that allowed the Muxes to show in TVHeadend. I did a Force Scan but they all failed. I tried changing one from Idle to Pend but it failed again.

I left Maximum Timeout at 15 and Re-fetch period at 60. I also left the PlutoTV Network discovery set to Disable. I haven't setup the Grabber yet if that matters. The channels do show up in the Cabernet PlutoTV channels tab.

This would probably also be a good time for any recommended strategies for changing any settings from default. I mainly use TVHeadend for OTA viewing and recording.

Edit: I tried changing Maximum Timeout to 60 seconds but the Muxes still failed.

G Kazaroth is the authority but you could try this:

1. set 'Enabled' to checked
2. set tuner_count to 4

1. set 'stream_type' to internalproxy
2. set 'Non-VOD Segments to start' to 3
3. set 'Enable URL Filtering' to checked
4. your URL Filter should be ^.*(_ad/|google.com|plutotv_filler|Well_be_right|_AdPod|_Promo).*$
5. set 'Enable PTS/DTS Resync' to checked
6. set 'PTS/DTS Resync Type' to ffmpeg
7. set 'XMLTV Enabled' to checked

1. set 'Maximum # input streams:' to 2
2. set 'Maximum timeout (seconds):' to 15
3. set 'Re-fetch period (mins):' to 60
4. force a rescan of your PlutoTV network

tvheadend then scans all your muxes (it will take a while). If some fail you can try to change those back to 'PEND' again (I had to do this a few times on some muxes)

During the forced scan you should check your log (/storage/tvheadend-locast/out.log) for errors. If you still have cabernet logging enabled and Level set to DEBUG in 'System Log Handler', you should see cabernet successfully tuning to channels and you should see log messages showing segments being 'Added', 'Serving' and 'Removed'.

You might also need to check your autostart.sh service for errors with this command

systemctl status kodi-autostart.service -l --no-pager

I made your suggested setting changes and restarted my TVHeadend box. The Muxes scan still fails.

/storage/tvheadend-locast/out.log doesn't exist. When I attempted to update Cabernet it said it couldn't update because it saw that file in the folder. So I deleted the file and removed the command to create the log from my autostart.sh file which allowed me to update Cabernet. I can temporarily put that command back into the autostart.sh file if that will help.

I entered the "systemctl status kodi-autostart.service -l --no-pager" command and got the following:

CoreELEC:~ # systemctl status kodi-autostart.service -l --no-pager
● kodi-autostart.service - Kodi user autostart script
Loaded: loaded (/usr/lib/systemd/system/kodi-autostart.service; disabled; vendor preset: disabled)
Active: active (exited) since Sun 2021-11-07 17:08:55 PST; 51min ago
Process: 5162 ExecStart=/bin/sh -c . /etc/profile; exec /bin/sh /storage/.config/autostart.sh (code=exited, status=0/SUCCESS)
Main PID: 5162 (code=exited, status=0/SUCCESS)
Tasks: 27 (limit: 1735)
Memory: 98.7M
CGroup: /system.slice/kodi-autostart.service
├─5230 /opt/bin/python3 /storage/tvheadend-locast/tvh_main.py
├─5545 /opt/bin/python3 /storage/tvheadend-locast/tvh_main.py
└─5554 /opt/bin/python3 /storage/tvheadend-locast/tvh_main.py

Nov 07 17:56:31 CoreELEC sh⁷⁸³²: self.m3u8_q = M3U8Queue(_config, _channel_dict)
Nov 07 17:56:31 CoreELEC sh⁷⁸³²: File "/storage/tvheadend-locast/lib/streams/m3u8_queue.py", line 77, in init
Nov 07 17:56:31 CoreELEC sh⁷⁸³²: self.pts_resync = PTSResync(_config, self.config_section, _channel_dict['uid'])
Nov 07 17:56:31 CoreELEC sh⁷⁸³²: File "/storage/tvheadend-locast/lib/streams/pts_resync.py", line 37, in init
Nov 07 17:56:31 CoreELEC sh⁷⁸³²: self.ffmpeg_proc = self.open_ffmpeg_proc()
Nov 07 17:56:31 CoreELEC sh⁷⁸³²: File "/storage/tvheadend-locast/lib/streams/pts_resync.py", line 112, in open_ffmpeg_proc
Nov 07 17:56:31 CoreELEC sh⁷⁸³²: ffmpeg_process = subprocess.Popen(ffmpeg_command,
Nov 07 17:56:31 CoreELEC sh⁷⁸³²: File "/opt/lib/python3.9/subprocess.py", line 951, in init
Nov 07 17:56:31 CoreELEC sh⁷⁸³²: File "/opt/lib/python3.9/subprocess.py", line 1821, in _execute_child
Nov 07 17:56:31 CoreELEC sh⁷⁸³²: FileNotFoundError: [Errno 2] No such file or directory: 'ffmpeg'

It looks like it has a problem finding ffmpeg.

Thanks for your help!

on CoreELEC you can install ffmpeg using entware

opkg install ffmpeg

Moving forward, it might be good to have a log file. Maybe just put it outside of the tvheadend-locast directory. This will put the log in the /storage directory and it will be named tvheadend-locast.log

#!/bin/sh
(
sleep 10
. /opt/etc/profile
cd /storage/tvheadend-locast
/opt/bin/python3 /storage/tvheadend-locast/tvh_main.py
) > /storage/tvheadend-locast.log 2>&1 &

Cabernet is also setup to create and manage a log file. By default it does a 10 file rotation with each file being 10MB. To enable, go to Settings > Logging > File Log Handler. Enable it and change the log level to what ever you need. The log file will appear in the cabernet/data/log folder.

OK, I've installed ffmpeg per the instructions above. I assumed (I guess incorrectly) since Kodi uses ffmpeg for playback I didn't need to install it.

I made some progress. 2 services showed up during the scan and I was able to map them. I tried to play them but they would barely play then I get a Tvheadend HTSP Client no signal available.

I've enabled the recommended logging in both the autostart.sh file and within Cabernet at the Debug level (should I be running both of these or are they redundant?).

I've attached the logs.

Cabernet can stream to either logs. tvheadend-locast.log is controlled by the first tab in the settings > logging area while cabernet.log is controlled by the second tab. So, it is flexible. Normally, the first tab is output to stdout/stderr, which goes to syslog, while the second tab goes to an app specific log.

The issue you are having is the multiprocessing queues class does not contain the standard python method qsize(). I will see what is needed to get around this issue. According to internet, some python installs do not have that method and throws a NotImplementedError.

Try 0.9.6.4. I have removed the use of qsize()

1. Install ffmpeg so that the PATH environment variable can find it
2. Update the PATH environment variable to include the location of the ffmpeg executable in the Kodi install
3. Update the Cabernet ffmpeg path from a relative path which uses $PATH to an absolute path to the Kodi executable

I updated to 0.9.6.4, rebooted, did a scan, and mapped an additional 56 services. However, a majority of the muxes that failed the scan will fail again when I change the status from IDLE to PEND. The channels that do get scanned and mapped seem to play properly.

I've attached the latest cabernet.log and tvheadend-locast.log. They seem to be complaining about not finding _cffi_backend.

From the cryptography site, here is the issue regarding your error
https://github.com/pyca/cryptography/issues/3703

also something similar...
https://foss.heptapod.net/pypy/cffi/-/issues/451

Does not look like they have a good resolution and seems to say the cffi module was not installed or installed correctly. I recommend doing some debugging.
Go into python3 by typing /opt/bin/python3
Then type "import _cffi_backend"
That worked on my computer
Also try
"from cryptography.hazmat.bindings._openssl import ffi, lib"
That also worked.

To quit, type "quit()"

You can try to install the dependency (cffi) (it should have been installed when you installed crypto) with whatever command you use to install python modules. then try the tests again.

Island may be able to shed some light on this.

Last piece of info. There is a pdf that claims having an import in cabernet could fix this. You can try to mod the file and see if that works.
Go to cabernet/lib/streams/m3u8_queue.py and edit the file
At the top near "import datetime" add the line
"import _cffi_backend"
Then restart the service and see if that works...

both of those gave me:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named '_cffi_backend'

I tried to install cffi and got:

CoreELEC:~ # pip install cffi
Requirement already satisfied: cffi in ./.opt/lib/python3.9/site-packages (1.15.0)
Requirement already satisfied: pycparser in ./.opt/lib/python3.9/site-packages (from cffi) (2.20)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

I believe that is the same warning when I installed the cryptography package as I remember.

They claim you need to include -U... pip install -U cffi

Also, I assume you are making sure it is installing into the correct python install...

I tried with the -U and got the same message.

Putting "import _cffi_backend" in cabernet/lib/streams/m3u8_queue.py caused Force Scan not to work and made Cabernet unreachable so I took it back out.

Recommend waiting for Island to determine why your cffi is corrupted.

one sec. i have a solution. just need to test it a bit....

Yes, your installation is missing some key modules. The following command should fix things:

opkg install python3-cryptography

As far as I can tell when you used 'pip install cryptography' to install cryptography some key modules that CoreELEC needs were not installed and you were left in a bad state. The python3-cryptography package was created to properly install python3 with cryptography in an entware environment like CoreELEC. They advertize it as:

python3-cryptography - 3.4.7-1 - cryptography is a package which provides cryptographic recipes and primitives to Python developers.  Our goal is for it to be your "cryptographic standard library". It supports Python 2.6-2.7, Python 3.3+, and PyPy 2.6+.

If in the future you need to wipe out your current CoreELEC's entware environment including python3, pip, cryptography, ffmpeg, etc installs and start from scratch I think doing as much of the install as possible with the 'opkg' command is the way to go. The following should work:

# remove entware including all installed packages/modules/etc
rm -rf /opt/* 
#  
# install entware (reboot when prompted)
installentware
#
# install ffmpeg
opkg install ffmpeg
#
# install python3 with cryptography (including ALL needed other modules)
opkg install python3-cryptography
#
# at this point the "from cryptography.hazmat.bindings._openssl import ffi, lib" python command will work
# however, I did not install and test cabernet (I do not use my CoreELEC system for cabernet)
#
# install pip (I think installing pip is optional becuase opkg installed all that was needed)
# I suggest when you want to install anything you check if you can do it using the 'opkg' command before trying pip
# FYI - the command 'opkg list' will show you all packages you can install with opkg
opkg install python3-pip
#
# to upgrade all your pip modules to the most up to date versions use this command
# I am not sure if upgrading all packages to the most current version is a good idea or not. 
# I tried it and it worked for me but I did not install and test cabernet 
pip3 list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip3 install -U 

Well, I did the "opkg install python3-cryptography" command and rebooted. I'm still having the problem.

Do you think it is time to nuke my current CoreELEC entware environment and start over per your commands above?

I've attached the cabernet.log file.

Yes, I think it is best to follow my suggested commands to nuke entware and start again. I suggest you try to run cabernet before you install python3-pip or use any pip or pip3 commands.

Do me a favor and post the results of the following commands before you nuke your environment

opkg list-installed
pip list
echo "from cryptography.hazmat.bindings._openssl import ffi, lib"|python3

Then after you nuke entware and install entware, ffmpeg and python3-cryptography run these commands and post the results as well

opkg list-installed
echo "from cryptography.hazmat.bindings._openssl import ffi, lib"|python3

If the echo/python3 command does not produce an error give cabernet a go and see if it works.

Sorry, I nuked it before I fully read your message. But here is the after information:

CoreELEC:~ # opkg list-installed
ca-certificates - 20210119-1
entware-opt - 227000-3
entware-release - 1.0-2
entware-upgrade - 1.0-1
findutils - 4.8.0-1
grep - 3.6-1a
libbz2 - 1.0.8-1a
libc - 2.27-11
libffi - 3.4.2-1
libgcc - 8.4.0-11
libopenssl - 1.1.1k-1
libpcre - 8.45-1
libpthread - 2.27-11
libpython3 - 3.9.6-2
librt - 2.27-11
libssp - 8.4.0-11
libstdcpp - 8.4.0-11
libtirpc - 1.3.2-1
libuuid - 2.37-1
locales - 2.27-9
opkg - 2021-06-13-1bf042dd-1
python3-base - 3.9.6-2
python3-cffi - 1.14.6-1
python3-cryptography - 3.4.7-1
python3-email - 3.9.6-2
python3-light - 3.9.6-2
python3-openssl - 3.9.6-2
python3-ply - 3.11-2
python3-pycparser - 2.20-5
python3-six - 1.16.0-1
python3-urllib - 3.9.6-2
terminfo - 6.2-2a
zlib - 1.2.11-3
zoneinfo-asia - 2021a-1
zoneinfo-europe - 2021a-1

Echo didn't return anything:

CoreELEC:~ # echo "from cryptography.hazmat.bindings._openssl import ffi, lib"|python3
CoreELEC:~ #

The Cabernet web page won't open. I didn't install pip.

Below is the output from the tvheadend-locast.log:

Traceback (most recent call last):
File "/storage/tvheadend-locast/tvh_main.py", line 10, in <module>
from lib import main
File "/storage/tvheadend-locast/lib/main.py", line 21, in <module>
import logging
ModuleNotFoundError: No module named 'logging'