Tvheadend

Here are the relevant log entries:

020-12-03 11:13:18.671 subscription: 0005: "::ffff:10.1.13.3 [ muzicman0 | Kodi Media Center ]" subscribing on channel "KABC", weight: 100, adapter: "IPTV #1", network: "Prime2", mux: "channels2.m3u - 7", service: "7", profile="SCW", hostname="::ffff:10.1.13.3", username="muzicman0", client="Kodi Media Center" 
2020-12-03 11:13:19.140 transcode: 0005: 01:MPEG2VIDEO: ==> Using profile NVENC
2020-12-03 11:13:19.140 transcode: 0005: 02:AC3: ==> Copy
2020-12-03 11:13:19.140 transcode: 0005: 03:AC3: ==> Copy
2020-12-03 11:13:20.103 libav: AVCodecContext: OpenEncodeSessionEx failed: out of memory (10)
2020-12-03 11:13:20.127 libav: AVCodecContext: No NVENC capable devices found
2020-12-03 11:13:29.247 subscription: 0005: "::ffff:10.1.13.3 [ muzicman0 | Kodi Media Center ]" unsubscribing from "KABC", hostname="::ffff:10.1.13.3", username="muzicman0", client="Kodi Media Center" 

The system is not out of memory. Also, some more info, it seems to happen after a few sessions, not so much time. So if I watch a few seconds of a channel, stop, repeat, after a few times I get this error.

Here is a log from when it workid:

020-12-03 11:09:00.889 mpegts: channels.m3u - 2 in Prime1 - tuning on IPTV #1
2020-12-03 11:09:00.890 subscription: 0001: "::ffff:10.1.13.3 [ muzicman0 | Kodi Media Center ]" subscribing on channel "KCBS", weight: 100, adapter: "IPTV #1", network: "Prime1", mux: "channels.m3u - 2", service: "2", profile="SCW", hostname="::ffff:10.1.13.3", username="muzicman0", client="Kodi Media Center" 
2020-12-03 11:09:01.343 transcode: 0001: 01:MPEG2VIDEO: ==> Using profile NVENC
2020-12-03 11:09:01.343 transcode: 0001: 02:AC3: ==> Copy
2020-12-03 11:09:01.343 transcode: 0001: 03:AC3: ==> Copy
2020-12-03 11:09:26.707 subscription: 0001: "::ffff:10.1.13.3 [ muzicman0 | Kodi Media Center ]" unsubscribing from "KCBS", hostname="::ffff:10.1.13.3", username="muzicman0", client="Kodi Media Center" 
Loglevel debug: enabled

Do you have more than 2 Sessions open at that time? How much GPU Memory is actually left?

I assumed that was system memory. I am not sure how much GPU memory is left, but it is a GTX1050 with 2 GB of memory.

I was not running multiple occurrences. Also, once it happens, the only way to recover is to reboot the PC.

It seems that it is on the 3rd attempt.

I wonder if the encodes are not stopping when I stop playback in Kodi?

Use

watch -d nvidia-smi
nvtop

to see GPU load

Newest driver have 3 sessions limit ;)
else patch it

1080p channel take near 280Mb GPU memory.

Thanks for the info. I am aware of the 3 concurrent transcoding limit, but I am only doing 1. Although, it does appear that after 3 attempts is when it stops, but I am not doing concurrent transcodes...just 1 at a time.

as far as memory goes, depending on how quickly after a reboot I do it, it is between 500 and 600 MB when it fails (out of 1997 MB available).

I did some more testing on this, and it seems to only happen when using the Kodi plugin. I downloaded m3u files and played them in VLC on a remote PC, and did not see the same problem. Also, the overall GPU memory use is lower (98 MB at idle, 249 Mb when playing). So it does appear to be something within the Kodi plugin. Also if I use x264 instead of NVENC, I don't see this issue, but using CPU encoding is not an option for long term use.

Kodi uses HTSP which doesn't support transcoding at all. So you most likely misconfigured something.

interesting. I put the profile that I created in, and it is taking the transcoded stream (source is mpeg2, but Kodi shows it as h.264), so it is working in a limited sort of way, but obviously, isn't completely working.

If I can't get transcoded streams remotely on Kodi, then this just won't work for me, and I will have to keep my existing solution (Channels), it would just be way more convenient to have everything in one app on my Shields.

ADDITIONAL INFO EDIT: It only seems to transcode live TV, not recorded files. This would be fine, if it worked reliably.

If this limitation doesn't work for you there are 2 options:
1. Submit a PR and fix the problem
2. Move on to a different Software

I get it. Wasn't trying to be snippy. Not being a developer, fixing it myself is beyond what I can do. I can script in VB, but that is about it.

My office is upgrading our internet service to a much faster upload speed in the near future. I will re-address then and decide if I need transcoding or not.

Flole Systems wrote:

Kodi uses HTSP which doesn't support transcoding at all. So you most likely misconfigured something.

Not true, can make demonstration.
Set transcoding in user profile and it will transcode.

Please show me how to create a HTSP profile that offers transcoding. When you specify any non-HTSP profile in Kodi you'll get an error (unless they changed something in the addon and are no longer using HTSP).

yeah, I created a new streaming profile that isn't HTSP based, and it works fine...well, other than failing after 3 attempts. Also, x264 works fine (it doesn't fail after 3 attempts), so it does appear to be somewhat supported.