Tvheadend

I am using Build: 4.2.8-36~g5bdcfd8ac (2019-11-15T18:09:47+0000) on Debian 9 running 4.9.0-11-amd64 kernel.
TVH has IPv6 enabled as well for local access and IPv4 range is 192.168.1.0/24

On the same machine I have set up WireGuard with 192.168.99.0/24 range.

When connected to WirGuard VPN (only set up for IPv4) and trying to access TVH I would get this error message (that is with user wireguard):
Loglevel debug: enabled
htsp: Got connection from ::ffff:192.168.99.16
htsp: ::ffff:192.168.99.16: Identified as user 'wireguard' (unverified)
htsp: ::ffff:192.168.99.16 [ wireguard ]: Welcomed client software: TVHClient (HTSPv32)
htsp: ::ffff:192.168.99.16 [ wireguard | TVHClient ]: Unauthorized access

When I allow any user () to connect from any ip range (0.0.0.0/0,::/0), I would still get this error:
htsp: ::ffff:192.168.99.16: Identified as user '' (unverified)
htsp: ::ffff:192.168.99.16 [ * ]: Welcomed client software: TVHClient (HTSPv32)
htsp: ::ffff:192.168.99.16 [ * | TVHClient ]: Unauthorized access
htsp: ::ffff:192.168.99.16 [ * | TVHClient ]: Disconnected

How can I get access from the WireGuard connection that only uses IPv4 to TVH?