Project

General

Profile

Bug #5668

tvheadend dvr adds invalid utf-8 character in filename for recording where title contains/ends with: 'в)' symbol

Added by Nikolay Kichukov over 5 years ago. Updated over 5 years ago.

Status:
Fixed
Priority:
Normal
Assignee:
Adam Sutton
Category:
PVR / DVR
Target version:
-
Start date:
2019-06-25
Due date:
% Done:

0%

Estimated time:
Found in version:
tvheadend-4.2.8
Affected Versions:

Description

Hello,

What was initially believed to be a bug with the iOS client for tvheadend, see https://github.com/zipleen/tvheadend-iphone-client/issues/246, turns out to be a tvheadend bug.

The issue:
When you create a recording from a title that contains/ends with 'в)' symbol, the filename attribute inside the dvr file contains an invalid utf-8 byte which prevents tvheadend from serving the recording (500 error) and causes the side defect of the iOS app as per the bug referenced earlier.

Reproducible: always

Steps to reproduce:
1.) Craft a specially designed unicode title into your EPG, import into EPG the below example file epg.xml(adjust the dates to match current or future time and your channel id)
2.) Configure your DVR profile like the attached dvr profile: dvr-profile.json

Files

epg.xml (398 Bytes) epg.xml epg in xml for a channel that contains the title string in unicode Nikolay Kichukov, 2019-06-25 21:09
dvr-profile.json (1.13 KB) dvr-profile.json dvr-profile settings Nikolay Kichukov, 2019-06-25 21:14

History

#1

Updated by Nikolay Kichukov over 5 years ago

The report submitted too early, continuing with steps to reproduce:

3.) Schedule a recording for the EPG entry from epg.xml
4.) Observe how TVHeadend creates a filename with invalid utf-8 character:

Jun 25 22:54:40 localhost tvheadend[25053]: subscription: 0001: "DVR: Осъдени души (110 години от рождението на Димитър Дим" subscribing on channel "BG Top Music", weight: 300, adapter: "IPTV", ... profile="matroska" 
Jun 25 22:54:40 localhost tvheadend[25053]: spawn: Executing "/usr/bin/ffmpeg" 
Jun 25 22:54:41 localhost tvheadend[25053]: dvr: /home/kodi/videos/recordings/BG-Top-Music-Осъдени-души-(110-години-от-рождението-на-Димитър-Димо�-2019-06-25_21-00.mkv from adapter: ...

5.) Look at the filename on the filesystem:
rw-rw-r- 1 tvheadend video 798M Jun 25 23:18 'BG-Top-Music-Осъдени-души-(110-години-от-рождението-на-Димитър-Димо'$'\320''-2019-06-25_21-00.mkv'
6.) Look at the dvr log file for it:
        "files": [
                {
                        "filename": "/somewhere/BG-Top-Music-Осъдени-души-(110-години-от-рождението-на-Димитър-Димо�-2019-06-25_21-00.mkv",
                        "info": [
                                {

7.) Try to play the recording over http and see the 500 error from tvheadend: 'error: HTTP/1.0 500 Internal Server Error'
HTTP/1.0 GET /dvrfile/380554f189c1a1a0b1923583a69e73cd -- 500

What has been attempted:
1.) Initially I believed the invalid byte/character came through the EPG, so I sanitized it to strip any invalid utf-8 sequences before uploading to tvheadend and yet the issue reappeared.

#2

Updated by Nikolay Kichukov over 5 years ago

The recording can be played over the HTSP protocol fine.

Expected behaviour:
Do not replace 'в)' with invalid utf-8 in the filename attribute in the dvr recording.

System information:
GNU/Gentoo Linux, arm architecture with 5.1.9 kernel.

Any additional questions, let me know.
Thank you,
-Nik

#3

Updated by Jaroslav Kysela over 5 years ago

From the config the title is limited to 99 bytes. The utf-8 validation check was added in 4.3 (utf8_validate_inplace() function).

#4

Updated by Nikolay Kichukov over 5 years ago

Can you refer me to the commit that introduced this validation so I can try to backport to 4.2.8(source based distro here)?
Cheers,
-N

#5

Updated by Jaroslav Kysela over 5 years ago

  • Status changed from New to Fixed
#6

Updated by Nikolay Kichukov over 5 years ago

Thanks. This indeed solves it.

Unrelated to this, I see a difference in one character in the log(this was like that before patching too):

tvheadend[19308]: subscription: 0001: "DVR: Осъдени души (110 години от рождението на Димитър Дим" subscribing on channel "BG Top Music", weight: 300,...profile="matroska" 
tvheadend[19308]: spawn: Executing "/usr/bin/ffmpeg" 
tvheadend[19308]: dvr: /somewhere/BG-Top-Music-Осъдени-души-(110-години-от-рождението-на-Димитър-Димо-2019-06-26_00-10.mkv from adapter: "IPTV", ...

On line 1, it ends with 'Дим' and in the filename on line 3 it truncates to 'Димо'.

This can be closed. I appreciate the quick response.
Cheers,
-N

Also available in: Atom PDF