Tvheadend

After days (weeks) of trying to figure out why I get temporary stream freezes every few seconds/minutes, I finally realized that I only get them when the ecm reply arrives faster than (around) 220ms after the request.
I suppose this has something to do with locking, but I cannot figure out where the problem lies (maybe the reply comes before all requests are sent and they both try to access the same object at the same time).
As soon as the replies return to the normal speeds of around 300ms, the stream resumes normally.
Please note that this happens using an irdeto card (multiple ecm channel ids), don't know if it happens with different card types.
Any help would be appreciated, it is killing me!