Project

General

Profile

Bug #2665

DTV_CLEAR failed error on 64-bit systems

Added by virtual dj almost 10 years ago. Updated over 9 years ago.

Status:
Fixed
Priority:
Normal
Assignee:
-
Category:
DVB
Target version:
-
Start date:
2015-02-07
Due date:
% Done:

0%

Estimated time:
Found in version:
3.9.2293~g74c563f
Affected Versions:

Description

Hi, first sorry for the "double" post but I would like to add more visibility to the thread added in the forum HERE by submitting it as a ticket.

There are a lot of users on the QNAP forum owning an x86_64 NAS that asked for my help about the TVHeadend version that I compile as QPKG. But I cannot help them because I don't know how to do that.

I've compiled 2 versions of TVHeadend, in this case the build 3.9.2293~g74c563f, one for ARM and another for x86 (32 bit); I use two Debian Squeeze VMs to build them with these parameters:

./configure --disable-avahi --enable-bundle --disable-libav --disable-dbus_1 --release

People running ARM do not have any issue; the same is for people running the Intel version on x86 NASes (32 bit, like me). The QNAP kernel version is v3.4.6.
Unfortunately, people running the same binary on x86_64 are not able to tune any frequency and instead get this error continuosly:

2015-01-08 19:31:31.117 mpegts: 578MHz in Durris - tuning on DiBcom 7000PC : DVB-T #0
2015-01-08 19:31:31.117 linuxdvb: DiBcom 7000PC : DVB-T #0 - DTV_CLEAR failed [e=Inappropriate ioctl for device]

I said "the same binary" because actually the QNAPs with 64-bit do not have the whole libraries and cannot run the application if I compile TVHeadend in 64-bit mode (they get permission denied errors even though the permissions are correct).
So I actually load the 64-bit kernel modules and then run TVHeadend 32-bit.

Another thing to say is that this behaviour happens when using any tuner that loads the dvb-core.ko module I think, because it happens even with the HDHomerun which uses only dvb-core.ko, dvb_hdhomerun.ko, dvb_hdhomerun_core.ko and dvb_hdhomerun_fe.ko.
Those on 64-bit that have the Sundtek tuner (which doesn't use any kernel module) do not have this error and TVHeadend works perfectly.

This, at first, let me think it was a kernel module issue, but it shouldn't, because we've tried running some older TVHeadend versions (always 32-bit) that I compiled in the past with the same kernel modules loaded (same files).

For example, the version 3.9.422~gc84bc2b still fails, but with a different error (a generic failed to start):

2015-01-12 19:27:41.000 [   INFO] mpegts: 538000 - starting for 'initscan' (weight 2)
2015-01-12 19:27:41.000 [   INFO] mpegts: 538000 - tuning on DiBcom 3000MC/P : DVB-T #0
2015-01-12 19:27:41.000 [  DEBUG] linuxdvb: DiBcom 3000MC/P : DVB-T #0 - starting 538000
2015-01-12 19:27:41.000 [WARNING] mpegts: 538000 - failed to start, try another

When trying the older version v3.5.126~g8c5f9af, again with the same modules, it works correctly:
Jan 12 18:51:16.688 [  DEBUG] dvb: /dev/dvb/adapter0 opened frontend /dev/dvb/adapter0/frontend0
Jan 12 18:51:16.688 [  DEBUG] dvb: "/dev/dvb/adapter0" tuning to "CESKE RADIOKOMUNIKACE: 626,000 kHz" (Initial autoscan)
Jan 12 18:51:17.051 [  DEBUG] dvb: /dev/dvb/adapter0 started dvr thread
Jan 12 18:51:17.053 [  DEBUG] viasat_baltic: install table handlers
Jan 12 18:51:17.053 [  DEBUG] uk_freesat: install table handlers
Jan 12 18:51:17.053 [  DEBUG] eit: install table handlers
Jan 12 18:51:17.070 [  DEBUG] eit: begin processing
Jan 12 18:51:17.568 [  DEBUG] PSI: Service "DiBcom 3000MC/P/CESKE RADIOKOMUNIKACE: 626,000 kHz/NOVA" PMT (version 12) updated
Jan 12 18:51:17.720 [  DEBUG] PSI: Service "DiBcom 3000MC/P/CESKE RADIOKOMUNIKACE: 626,000 kHz/Prima" PMT (version 19) updated
Jan 12 18:51:17.942 [  DEBUG] PSI: Service "DiBcom 3000MC/P/CESKE RADIOKOMUNIKACE: 626,000 kHz/BARRANDOV TV" PMT (version 5) updated
Jan 12 18:51:18.002 [  DEBUG] dvb: "CESKE RADIOKOMUNIKACE: 626,000 kHz" on adapter "DiBcom 3000MC/P", status changed to Bursty FEC
Jan 12 18:51:19.001 [  DEBUG] dvb: DiBcom 3000MC/P: FE_READ_UNCORRECTED_BLOCKS returns delta updates (delta=-82)
Jan 12 18:51:27.047 [  DEBUG] eit: processing complete
Jan 12 18:51:28.002 [  DEBUG] dvb: "CESKE RADIOKOMUNIKACE: 626,000 kHz" on adapter "DiBcom 3000MC/P", status changed to OK

My gcc version on the VM is 4.4.5 and, to make the binary run on the QNAP which has 4.2.1, I've copied the libc libraries and run TVHeadend with the ld-linux trick (for the v3.9.2293 version, while v3.9.422 and v3.5.126 run directly because the installed gcc version was enough to compile them).

I've also tried to compare the two versions (non working v3.9.422 and working v3.5.125) using GitHub but there are a lot of differences and I'm not able to isolate the problem, probably becase I'm not very comfortable with C programming.

Finally, /usr/include/linux/dvb/version.h on the VM that I use for compiling is:

#ifndef _DVBVERSION_H_
#define _DVBVERSION_H_

#define DVB_API_VERSION 5
#define DVB_API_VERSION_MINOR 1

#endif /*_DVBVERSION_H_*/

Then by adding some log instructions into the TVHeadend sources I think the problem may be located in this section.

However, for both the working and non-working systems the DVB_API_VERSION >= 5 condition is true.
The offending line that fails is:

((ioctl(lfe->lfe_fe_fd, FE_SET_PROPERTY, &clear_cmdseq)) != 0)

I've tried also to add debugging info on dvb-core.ko and this is what dmesg says on a WORKING system:

dvb_frontend_open
dvb_frontend_start
dvb_frontend_thread
dvb_frontend_ioctl
DVB: initialising adapter 0 frontend 0 (DiBcom 7000PC)...
dvb_frontend_ioctl_properties
dvb_frontend_ioctl_properties() properties.num = 1
dvb_frontend_ioctl_properties() properties.props = bfd82033
dtv_property_dump: tvp.cmd = 0x0000002c undefined
dvb_frontend_ioctl
dvb_frontend_release
dvb_frontend_open
dvb_frontend_start

On the contrary, here's a NON WORKING one:
dvb_frontend_open
dvb_frontend_start
dvb_frontend_thread
DVB: initialising adapter 1 frontend 0 (DiBcom 7000PC)...
dvb_frontend_ioctl (61)
---
dvb_frontend_release
dvb_frontend_open
dvb_frontend_start
dvb_frontend_stop
---
dvb_frontend_thread
DVB: initialising adapter 1 frontend 0 (DiBcom 7000PC)...
dvb_frontend_open
dvb_frontend_start
dvb_frontend_thread
dvb_frontend_ioctl (61)
DVB: initialising adapter 0 frontend 0 (DiBcom 7000PC)...
---
dvb_frontend_release
dvb_frontend_open
dvb_frontend_start
dvb_frontend_stop

Is the dvb_frontend_ioctl (61) the symptom of the error?
I've checked again the sources and I do not see differences between the 32 bit and 64 bit kernel modules. But something has to be there if the latter fail.

Please, anyone, if you have some suggestions on what to try please help.
By the way, on my 32-bit NAS all is working correctly, so I'll need to submit the test to the QNAP forum community.


Files

Logs.zip (3.69 MB) Logs.zip Trace log files virtual dj, 2015-02-21 12:44
E.txt (6.94 KB) E.txt virtual dj, 2015-02-23 18:04
frontend.h (11.1 KB) frontend.h virtual dj, 2015-03-21 11:17
compat_ioctl.c (45.2 KB) compat_ioctl.c virtual dj, 2015-03-21 11:17
strace.txt (15.5 KB) strace.txt strace verbose output virtual dj, 2015-03-27 19:12

History

#1

Updated by Jaroslav Kysela almost 10 years ago

The file linux/dvb/version.h must match the file in kernel which is run on the device. I would probably use some older <5.0 header file to compile tvh which you may find in vendor/dvb-api directory (3.2 for example), because it seems that the compability layer for 32-bit applications does not work correctly in your case.

#2

Updated by virtual dj almost 10 years ago

Hi Jaroslav and thank you VERY MUCH for answering.

The file linux/dvb/version.h must match the file in kernel which is run on the device.

I thought the same intially and it's the reason why I actually post the version.h information.

However on the picture below you can see the original kernel sources of the QNAP and the version is 5.5, i.e. newer than the version that I used on the build virtual machine (5.1).

When you say the version must match, you mean exactly or an older version.h version is OK too?
And as the 32-bit is working, is the 32-bit compatibility layer conflicting with the version.h file?

I would probably use some older <5.0 header file to compile tvh which you may find in vendor/dvb-api directory (3.2 for example), because it seems that the compability layer for 32-bit applications does not work correctly in your case.

I've tried. On the build VM I deleted the /usr/include/linux/dvb folder and replaced it with ./vendor/dvb-api/3.2/linux/dvb so it should be correct.
But after a make clean and a new configure I'm not able to compile the 74c563f version anymore:

CC              src/input/mpegts/linuxdvb/linuxdvb.o
CC              src/input/mpegts/linuxdvb/linuxdvb_adapter.o
CC              src/input/mpegts/linuxdvb/linuxdvb_frontend.o
cc1: warnings being treated as errors
src/input/mpegts/linuxdvb/linuxdvb_frontend.c:921: error: ‘linuxdvb2tvh’ defined but not used
make: *** [/root/tvheadend/tvheadend/build.linux/src/input/mpegts/linuxdvb/linuxdvb_frontend.o] Error 1

Finally - if I'm not misusing your patience - I moved to the DVB api v5 because of supporting DVB-S2 with 8PSK, which I think couldn't be done with v3.2. What would be the correct way to use DVB-S2 with 8PSK then?

#3

Updated by Jaroslav Kysela almost 10 years ago

The compilation issue should be fixed in v3.9-2486-g4a9dfec for ver 3.2.

In this case 5.5 dvb headers should be used (include/linux/dvb/*.h), but it won't solve the DVB_CLEAR issue, if the 32-bit support in the kernel is broken - the kernel must be fixed. Basically, it seems that the ioctl is not converted properly between the 32-bit request and 64-bit kernel driver (it's called as the compat layer in the kernel). I'm sure that there is a patch which fixes this issue for linuxdvb. Just try to search in the kernel sources.

#4

Updated by virtual dj almost 10 years ago

In this case 5.5 dvb headers should be used (include/linux/dvb/*.h), but it won't solve the DVB_CLEAR issue, if the 32-bit support in the kernel is broken - the kernel must be fixed.

OK, we tried the same version (3.9.2293~g74c563f) compiled with v5.5 DVB headers from TVHeadend vendor folder and the error is still there:

2015-02-11 11:46:41.777 [ NOTICE] START: HTS Tvheadend version 3.9.2293~g74c563f started, running as PID:31803 UID:0 GID:0, CWD:/root CNF:/share/CACHEDEV1_DATA/.qpkg/TVHeadend/config
...
2015-02-11 11:48:51.000 [   INFO] mpegts: 698MHz in West Midlands - tuning on DiBcom 7000PC : DVB-T #0
2015-02-11 11:48:51.000 [  DEBUG] linuxdvb: DiBcom 7000PC : DVB-T #0 - starting 698MHz in West Midlands
2015-02-11 11:48:51.000 [  ERROR] linuxdvb: DiBcom 7000PC : DVB-T #0 - DTV_CLEAR failed [e=Inappropriate ioctl for device]
2015-02-11 11:48:51.000 [WARNING] mpegts: 698MHz in West Midlands - failed to start on DiBcom 7000PC : DVB-T #0, try another

How can I know which version of the DVB headers TVHeadend uses from a binary? I mean, how can I differentiate the two binaries that I've compiled as the current one should have 5.5 dvb headers while the older one should have 5.1?

Basically, it seems that the ioctl is not converted properly between the 32-bit request and 64-bit kernel driver (it's called as the compat layer in the kernel).

OK, that sound right, but why older TVHeadend binaries such as v3.5.126~g8c5f9af are working then? The kernel modules are the same and the running kernel driver and its compat layer should be bugged too, but TVHeadend is able to workaround in some ways.
I know that the DVB "core" of TVHeadend has been rewritten recently, I'm just asking if this is something we can investigate...

I'm sure that there is a patch which fixes this issue for linuxdvb. Just try to search in the kernel sources.

Sorry, I'm not very skilled on this subject (but opened to suggestions of course!).
Where am I supposed to look for kernel patches? I searched for ".patch" files on the QNAP GPLs (those on the image posted on the previous message) but there is nothing related to DVB.
I tried also HERE but cannot find anything related to dvb-core.

The compilation issue should be fixed in v3.9-2486-g4a9dfec for ver 3.2.

Unfortunately, it is not... Here it is what I get using that version:

root@debian-x86:~/tvheadend/tvheadend# git reset --hard 4a9dfec 
HEAD is now at 4a9dfec linuxdvb: fix compilation for dvb < 5.0
root@debian-x86:~/tvheadend/tvheadend# make clean
root@debian-x86:~/tvheadend/tvheadend# ./configure --disable-avahi --enable-bundle --disable-libav --disable-dbus_1 --release
Checking support/features
  checking for cc execinfo.h ...                    ok
  checking for cc -mmmx ...                         ok
  checking for cc -msse2 ...                        ok
  checking for cc getloadavg ...                    ok
  checking for cc atomic64 ...                      ok
  checking for cc lockowner ...                     ok
  checking for cc qsort_r ...                       ok
  checking for cc recvmmsg ...                      fail
  checking for cc libiconv ...                      fail
  checking for py module gzip ...                   ok
  checking for bzip2 ...                            ok
  checking for pkg openssl  ...                     ok (detected 0.9.8o)
  checking for cc linux/dvb/version.h ...           ok
  checking for cc libhdhomerun/hdhomerun.h ...      fail
  checking for pkg zlib  ...                        ok (detected 1.2.3.4)
  checking for pkg liburiparser  ...                fail (detected <none>)
  checking for cc sys/inotify.h ...                 ok
  fetching dvb-scan files ...                       ok

Compiler:
  Using C compiler:                        cc
  Build for arch:                          i686

Binaries:
  Using PYTHON:                            python

Options:
  cwc                                      yes
  capmt                                    yes
  constcw                                  yes
  v4l                                      no
  linuxdvb                                 yes
  satip_client                             yes
  hdhomerun_client                         no
  hdhomerun_static                         no
  iptv                                     yes
  tsfile                                   yes
  dvbscan                                  yes
  timeshift                                yes
  trace                                    yes
  imagecache                               yes
  avahi                                    no
  zlib                                     yes
  libav                                    no
  libffmpeg_static                         no
  inotify                                  yes
  epoll                                    yes
  uriparser                                no
  ccache                                   no
  tvhcsa                                   yes
  bundle                                   yes
  dvbcsa                                   no
  kqueue                                   no
  dbus_1                                   no
  android                                  no
  tsdebug                                  no
  execinfo                                 yes
  mmx                                      yes
  sse2                                     yes
  getloadavg                               yes
  atomic64                                 yes
  lockowner                                yes
  qsort_r                                  yes
  py_gzip                                  yes
  bin_bzip2                                yes
  ssl                                      yes
  linuxdvbapi                              yes
  upnp                                     yes
  inotify_h                                yes
  mpegts                                   yes
  mpegps                                   no
  mpegts_dvb                               yes

Packages:
  openssl                                  0.9.8o
  zlib                                     1.2.3.4

Installation paths:
  Prefix:                                  /usr/local
  Binaries:                                ${prefix}/bin
  Libraries:                               ${prefix}/lib
  Data files:                              ${prefix}/share
  Man pages:                               ${datadir}/man

Final Binary:
  /root/tvheadend/tvheadend/build.linux/tvheadend

Tvheadend Data Directory:
  /usr/local/share/tvheadend

root@debian-x86:~/tvheadend/tvheadend# make
CC              src/version.o
CC              src/uuid.o
CC              src/main.o
CC              src/tvhlog.o
CC              src/idnode.o
CC              src/prop.o
CC              src/utils.o
CC              src/wrappers.o
CC              src/access.o
CC              src/tcp.o
CC              src/udp.o
cc1: warnings being treated as errors
src/udp.c:529: error: no previous prototype for ‘recvmmsg’
make: *** [/root/tvheadend/tvheadend/build.linux/src/udp.o] Error 1

#5

Updated by Jaroslav Kysela almost 10 years ago

virtual dj wrote:

The compilation issue should be fixed in v3.9-2486-g4a9dfec for ver 3.2.

Unfortunately, it is not... Here it is what I get using that version:
[...]

Fixed in v3.9-2496-gd386ed0 .

#6

Updated by virtual dj almost 10 years ago

Jaroslav Kysela wrote:

Fixed in v3.9-2496-gd386ed0 .

Thanks, the v3.9-2496 now compiles fine with the DVB 3.2 headers.
Unfortunately for all the guys with the 64-bit NAS, it doesn't solve the issue.

The error message on the TVHeadend log has changed, though.
This is what people get with a Dibcom tuner and v3.9-2496:

2015-02-13 11:57:03.000 [   INFO] mpegts: 826MHz in West Midlands - tuning on DiBcom 7000PC : DVB-T #0
2015-02-13 11:57:03.000 [  DEBUG] linuxdvb: DiBcom 7000PC : DVB-T #0 - starting 826MHz in West Midlands
2015-02-13 11:57:03.000 [  ERROR] linuxdvb: DiBcom 7000PC : DVB-T #0 - failed to tune [e=Bad address]
2015-02-13 11:57:03.000 [WARNING] mpegts: 826MHz in West Midlands - failed to start on DiBcom 7000PC : DVB-T #0, try another

Those with an HDHomerun insted get:
2015-02-13 17:44:56.005 linuxdvb: HDHomeRun ATSC 105134FA-0 : ATSC #0 - failed to tune [e=Bad address]
2015-02-13 17:44:56.005 mpegts: 63.28MHz in USTV - failed to start on HDHomeRun ATSC 105134FA-0 : ATSC #0, try another
2015-02-13 17:44:56.005 mpegts: 57.28MHz in USTV - tuning on HDHomeRun ATSC 105134FA-0 : ATSC #0
2015-02-13 17:44:56.005 linuxdvb: HDHomeRun ATSC 105134FA-0 : ATSC #0 - failed to tune [e=Bad address]
2015-02-13 17:44:56.005 mpegts: 57.28MHz in USTV - failed to start on HDHomeRun ATSC 105134FA-0 : ATSC #0, try another

So basically DTV_CLEAR failed [e=Inappropriate ioctl for device] has becomed failed to tune [e=Bad address].

All the 64-bit QNAP users and I are available for any test you would like us to perform, but we really like to have TVHeadend working, as it was working in the past with the same OS and kernel.
Please help! :)

#7

Updated by Jaroslav Kysela almost 10 years ago

Compile tvheadend with --enable-trace and provide log "--trace linuxdvb,diseqc -l <logfile>" (command line options) for working and non-working cases.

#8

Updated by virtual dj almost 10 years ago

Jaroslav Kysela wrote:

Compile tvheadend with --enable-trace and provide log "--trace linuxdvb,diseqc -l <logfile>" (command line options) for working and non-working cases.

Sorry for the long delay, but I took the time to build a QPKG for the QNAP to do the tests on the users machines.

I used v3.4.6 kernel modules and a menu to run the two different versions of TVHeadend with the trace logs; on the attached files you'll see v3.9-2496-gd386ed0 (marked as New_DVB_5_5) and v3.5.126~g8c5f9af (marked as Old_DVB). I recompiled the latter with the trace option so you can compare them.

As there were no advantages on using DVB Headers v3.2, I used v5.5 for both the two versions (which is the version that QNAP GPL ships).
Again, v3.9 runs with the ld-linux trick (due to its newer glibc) while v3.5 runs directly, but I don't think this should be a problem.
So the kernel modules are the same (and the script to load them); the user choose what binary to run and then it grabs the trace file (with "--trace linuxdvb,diseqc -l" for v3.9 and "--trace --logfile" for v3.5).

On the attached zipped file you'll find:
  • Log A.txt = User running v3.9 (32-bit) with a dual DiBcom 7000PC tuner on QNAP kernel 3.4.6 (64-bit) [NON-WORKING]
  • Log B.txt = User running v3.5 (32-bit) with a dual DiBcom 7000PC tuner on QNAP kernel 3.4.6 (64-bit) [WORKING]
  • Log C.txt = User running v3.9 (32-bit) with a HDHomerun tuner on QNAP kernel 3.4.6 (64-bit) [NON-WORKING]
  • Log D.txt = My NAS running v3.9 (32-bit) with a DiBcom 7000PC tuner on QNAP kernel 2.6.33.2 (32-bit) [WORKING]

At the moment we're still missing a log of a working HDHomerun with v3.5, because the v3.5 version that I compiled with the trace option fails with this message to the user:

Feb 18 19:59:40.723 [ NOTICE]:START: HTS Tvheadend version 3.5.126~g8c5f9af-unknown started, running as PID:19521 UID:0 GID:0, settings located in '/share/CACHEDEV1_DATA/.qpkg/TVHeadend/config'
Feb 18 20:03:08.000 [   INFO]:epgdb: saved
Feb 18 20:03:08.000 [   INFO]:epgdb:   brands     0
Feb 18 20:03:08.000 [   INFO]:epgdb:   seasons    0
Feb 18 20:03:08.000 [   INFO]:epgdb:   episodes   0
Feb 18 20:03:08.000 [   INFO]:epgdb:   broadcasts 0
Feb 18 20:03:08.001 [  TRACE]:timeshift: reaper thread exit
Feb 18 20:03:08.001 [ NOTICE]:STOP: Exiting HTS Tvheadend
It might be a configuration error though, because the old v3.5 without trace was working correctly for the same user on the same system (and I don't know what timeshift: reaper thread exit means).
Unfortunately I don't have a HDHomerun and a 64-bit NAS (with 3.4.6) to try myself.

I gave a look at the logs and cannot spot any interesting data, at least for the failing ones; if you need me to build a test version to submit to the users, just let me know! We're in your hands...
Thanks, as always!

#9

Updated by virtual dj almost 10 years ago

Here's the log E.txt of the same user of log C.txt with v3.5 (32-bit) and an HDHomerun tuner on QNAP kernel 3.4.6 (64-bit) [WORKING].

#10

Updated by virtual dj almost 10 years ago

Hi Jaroslav, are the logs OK? Did you have time to identify a possible cause?
We're here if you need further data, just ask! :)

Thank you

#11

Updated by Jaroslav Kysela almost 10 years ago

Unfortunately, the "DTV_CLEAR failed [e=Inappropriate ioctl for device]" error is present in A.txt and C.txt. The linuxdvb subsystem v5.0+ should be able to handle this ioctl.

In E.txt tvheadend v3.5-126 is used. This version uses always old ioctl for DVB-T (FE_SET_FRONTEND), but recent tvh uses 5.0+ API - FE_SET_PROPERTY ioctl - also for DVB-T.

Do you have kernel sources?

#12

Updated by virtual dj almost 10 years ago

In E.txt tvheadend v3.5-126 is used. This version uses always old ioctl for DVB-T (FE_SET_FRONTEND), but recent tvh uses 5.0+ API - FE_SET_PROPERTY ioctl - also for DVB-T.

Yes, definitely this difference should be the culprit.

Do you have kernel sources?

You can download the sources from the QNAP GPL 3.8.0 here.
Inside the file GPL_TS-20130601-3.8.x.tar.gz you can find the kernel sources in GPL_TS/src/linux_3.4.6.

#13

Updated by virtual dj over 9 years ago

Hi Jaroslav, have you got any news on this subject?
Thanks.

#14

Updated by virtual dj over 9 years ago

Even though I'm not good at all at Linux programming, I've tried some googling about your FE_SET_PROPERTY suggestion which may not work on the QNAP kernel.

I found these interesting links:
S2API: Problem with 64/32bit compatibility
Cannot tune DVB-T with a recent kaffeine and 2.6.28

Both are from the same author which I think has exactly the same problem described on this ticket.
As I'm not able to dig and interpret the sources, I'm attaching the two files (hoping you'll be able to):
  • include/linux/dvb/frontend.h
  • fs/compat_ioctl.c

to this post; they're extracted from the whole QNAP kernel sources.

I also found this patch:
[PATCH] [media] compat_ioctl: add compat handler for FE_SET_PROPERTY
but I don't think it may be useful, right? As I cannot replace the QNAP kernel which is actually "embedded" into the firmware.

So the question... Is there something we can do to circumvent this issue or is it just a kernel incompatibility/bug which cannot be resolved?
Is a configuration switch on the TVHeadend sources possible, that makes it use FE_SET_FRONTEND instead of FE_SET_PROPERTY? What will we lose if we don't use FE_SET_PROPERTY, maybe DVB-S2? Or is it just a different API?

#15

Updated by Jaroslav Kysela over 9 years ago

Yes, it looks like that FE_SET_PROPERTY has no compatibility converter, so it's impossible to use 32-bit ioctl on 64-bit kernel.

FE_SET_FRONTEND has limitations.

I would try to resolve the 64-bit compilation issue. Have you tried the static linking to resolve the problems with libraries ?

#16

Updated by virtual dj over 9 years ago

Yes, it looks like that FE_SET_PROPERTY has no compatibility converter, so it's impossible to use 32-bit ioctl on 64-bit kernel.

FE_SET_FRONTEND has limitations.

That's bad news. :(

I would try to resolve the 64-bit compilation issue.

I had compiled the pure 64-bit version of TVHeadend on a 64-bit Debian VM with the following switches:

./configure --disable-avahi --enable-bundle --disable-libav --disable-dbus_1 --release

On the VM it compiled and worked correctly of course, but when moving to the QNAP (together with the complete libc, due to the old version available on the NAS) there were errors.
For example, even when providing this command line:

tvheadend -c ${QPKG_DIR}/config -d -u admin -g administrators -C
they got:
2014-12-27 17:54:55.040 [   INFO] main: Log started
2014-12-27 17:54:55.040 [  ERROR] tcp: bind: *: Address already in use
2014-12-27 17:54:55.040 [  ERROR] tcp: bind: *: Address already in use
2014-12-27 17:54:55.041 [WARNING] START: configuration path /share/CACHEDEV1_DATA/.qpkg/TVHeadend/config is not r/w for UID:1 GID:1 [e=Permission denied], settings will not be saved
and the web interface was not available.
Be aware that on the QNAP the administrator user is 'admin' and not 'root'.

Then we abandoned the 64-bit compilation because we though it was better to focus on the 32-bit, as it always worked in the past.

Have you tried the static linking to resolve the problems with libraries ?

Can you suggest how should I do it?
At the moment using a Debian x64 squeeze VM I get:

root@debian-x64:~/tvheadend/tvheadend# git reset --hard d386ed0
root@debian-x64:~/tvheadend/tvheadend# make clean
root@debian-x64:~/tvheadend/tvheadend# ./configure --disable-avahi --enable-bundle --disable-libav --disable-dbus_1 --release
Checking support/features
  checking for cc execinfo.h ...                    ok
  checking for cc -mmmx ...                         ok
  checking for cc -msse2 ...                        ok
  checking for cc getloadavg ...                    ok
  checking for cc atomic64 ...                      ok
  checking for cc lockowner ...                     ok
  checking for cc qsort_r ...                       ok
  checking for cc recvmmsg ...                      fail
  checking for cc libiconv ...                      fail
  checking for py module gzip ...                   ok
  checking for bzip2 ...                            ok
  checking for pkg openssl  ...                     ok (detected 0.9.8o)
  checking for cc linux/dvb/version.h ...           ok
  checking for cc libhdhomerun/hdhomerun.h ...      fail
  checking for pkg zlib  ...                        ok (detected 1.2.3.4)
  checking for pkg liburiparser  ...                fail (detected <none>)
  checking for cc sys/inotify.h ...                 ok
  fetching dvb-scan files ...                       ok

Compiler:
  Using C compiler:                        cc
  Build for arch:                          x86_64

Binaries:
  Using PYTHON:                            python

Options:
  cwc                                      yes
  capmt                                    yes
  constcw                                  yes
  v4l                                      no
  linuxdvb                                 yes
  satip_client                             yes
  hdhomerun_client                         no
  hdhomerun_static                         no
  iptv                                     yes
  tsfile                                   yes
  dvbscan                                  yes
  timeshift                                yes
  trace                                    yes
  imagecache                               yes
  avahi                                    no
  zlib                                     yes
  libav                                    no
  libffmpeg_static                         no
  inotify                                  yes
  epoll                                    yes
  uriparser                                no
  ccache                                   no
  tvhcsa                                   yes
  bundle                                   yes
  dvbcsa                                   no
  kqueue                                   no
  dbus_1                                   no
  android                                  no
  tsdebug                                  no
  execinfo                                 yes
  mmx                                      yes
  sse2                                     yes
  getloadavg                               yes
  atomic64                                 yes
  lockowner                                yes
  qsort_r                                  yes
  py_gzip                                  yes
  bin_bzip2                                yes
  ssl                                      yes
  linuxdvbapi                              yes
  upnp                                     yes
  inotify_h                                yes
  mpegts                                   yes
  mpegps                                   no
  mpegts_dvb                               yes

Packages:
  openssl                                  0.9.8o
  zlib                                     1.2.3.4

Installation paths:
  Prefix:                                  /usr/local
  Binaries:                                ${prefix}/bin
  Libraries:                               ${prefix}/lib
  Data files:                              ${prefix}/share
  Man pages:                               ${datadir}/man

Final Binary:
  /root/tvheadend/tvheadend/build.linux/tvheadend

Tvheadend Data Directory:
  /usr/local/share/tvheadend

root@debian-x64:~/tvheadend/tvheadend# make
... omitted ...
CC              src/descrambler/libaesdec/libaesdec.o
MKBUNDLE        bundle.c
CC              bundle.o
CC              tvheadend

root@debian-x64:~/tvheadend/tvheadend# ldd build.linux/tvheadend
        linux-vdso.so.1 =>  (0x00007fffc3375000)
        libssl.so.0.9.8 => /usr/lib/libssl.so.0.9.8 (0x00007fe519aae000)
        libcrypto.so.0.9.8 => /usr/lib/libcrypto.so.0.9.8 (0x00007fe51970d000)
        libz.so.1 => /usr/lib/libz.so.1 (0x00007fe5194f5000)
        libdl.so.2 => /lib/libdl.so.2 (0x00007fe5192f1000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00007fe5190d5000)
        libm.so.6 => /lib/libm.so.6 (0x00007fe518e52000)
        librt.so.1 => /lib/librt.so.1 (0x00007fe518c4a000)
        libc.so.6 => /lib/libc.so.6 (0x00007fe5188e8000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fe519d0b000)

root@debian-x64:~/tvheadend/tvheadend# gcc -v
Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.4.5-8' --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.4 --enable-shared --enable-multiarch --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4 --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --with-arch-32=i586 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.4.5 (Debian 4.4.5-8)

And those libraries are the files that I copy to the QNAP to allow TVHeadend to run using the newer glibc by Debian, instead of the QNAP ones.
But the webserver socket errors make me thing something is going wrong there.

#17

Updated by Jaroslav Kysela over 9 years ago

What does say 'netstat -4an | grep 9981' before start? Also, does have the user with uid 1 and gid 1 access to all directories from the path /share/CACHEDEV1_DATA/.qpkg/TVHeadend/config ?

#18

Updated by virtual dj over 9 years ago

What does say 'netstat -4an | grep 9981' before start?

It said that there was another TVHeadend "clone" running on the NAS, hence the error.
So ignore that, sorry.

Also, does have the user with uid 1 and gid 1 access to all directories from the path /share/CACHEDEV1_DATA/.qpkg/TVHeadend/config ?

Thank you VERY much. You asked the right thing: the problem that prevented the 64-bit version to run was there!
Even though TVHeadend was launched with:

-u admin -g administrators -C
apparently it picked up a wrong UID. On the QNAP:
[~] # id -u root
id: root: No such user
[~] # id -u admin
0
[~] # id -g admin
0
[~] # id -G admin
0 100
[~] # id
uid=0(admin) gid=0(administrators) groups=0(administrators),100(everyone),101(kvm),102(libvirtd)
but TVHeadend interpreted "admin" with UID 1 (and GID 1). I don't know why.
As you can recall from the logs, this behaviour happens only on the NAS with 64-bit kernel.

So running TVHeadend without the "-u" and "-g" switches picked up UID 0 and now the user reports that it's working correctly (and tuning). Is this a bug?
I'm available if you have additional tests to do.

By the way, I was trying to compile the latest version (as I was still running 3.9-2496-gd386ed0) and it fails with:

root@debian-x64:~/tvheadend/tvheadend# make
CC              src/version.o
CC              src/uuid.o
CC              src/main.o
In file included from /root/tvheadend/tvheadend/src/input.h:128,
                 from src/satip/server.h:27,
                 from src/main.c:53:
/root/tvheadend/tvheadend/src/input/mpegts.h:43: error: redefinition of typedef ‘mpegts_psi_section_t’
/root/tvheadend/tvheadend/src/input/mpegts/dvb.h:216: note: previous declaration of ‘mpegts_psi_section_t’ was here
make: *** [/root/tvheadend/tvheadend/build.linux/src/main.o] Error 1
Can you fix it too?

#19

Updated by Jaroslav Kysela over 9 years ago

virtual dj wrote:

Also, does have the user with uid 1 and gid 1 access to all directories from the path /share/CACHEDEV1_DATA/.qpkg/TVHeadend/config ?

Thank you VERY much. You asked the right thing: the problem that prevented the 64-bit version to run was there!
Even though TVHeadend was launched with:
[...]apparently it picked up a wrong UID. On the QNAP:
[...]but TVHeadend interpreted "admin" with UID 1 (and GID 1). I don't know why.
As you can recall from the logs, this behaviour happens only on the NAS with 64-bit kernel.

So running TVHeadend without the "-u" and "-g" switches picked up UID 0 and now the user reports that it's working correctly (and tuning). Is this a bug?
I'm available if you have additional tests to do.

Ok, so we're close. TVH uses standard getgrnam() and getpwnam() functions: https://github.com/tvheadend/tvheadend/blob/master/src/main.c#L712-L745

As you can see, the 1 values are when these functions return NULL. It looks like a glibc issue... Perhaps, the /etc path is prefixed for compilation ? Maybe strace might show which files are read at runtime.

By the way, I was trying to compile the latest version (as I was still running 3.9-2496-gd386ed0) and it fails with:
[...]Can you fix it too?

v3.9-2659-g1f79d9d fixes it..

#20

Updated by virtual dj over 9 years ago

v3.9-2659-g1f79d9d fixes it..

Thanks, it compiles perfectly now.

As you can see, the 1 values are when these functions return NULL.

Yes, I've tried adding a tvhdebug statement on the "else" part of grp and pw: both of them are NULL.

It looks like a glibc issue... Perhaps, the /etc path is prefixed for compilation ?

Sorry, I miss the "/etc" prefix thing. What does it mean? I do not use prefixes (you can see how I compile on the first post of this ticket) nor /etc is in my PATH.

Maybe strace might show which files are read at runtime.

The strace result is attached. Let me know if I have to do it in a different way.
I wasn't able to mix the strace output with the TVHeadend log, anyway I hit CTRL+C when I saw the line:

START: HTS Tvheadend version 3.9.2660~g169b71b started, running as PID:4859 UID:1 GID:1, CWD:/share/CACHEDEV1_DATA/Public/mytest CNF:/share/CACHEDEV1_DATA/Public/mytest/config

#21

Updated by Jaroslav Kysela over 9 years ago

virtual dj wrote:

v3.9-2659-g1f79d9d fixes it..

Thanks, it compiles perfectly now.

As you can see, the 1 values are when these functions return NULL.

Yes, I've tried adding a tvhdebug statement on the "else" part of grp and pw: both of them are NULL.

It looks like a glibc issue... Perhaps, the /etc path is prefixed for compilation ?

Sorry, I miss the "/etc" prefix thing. What does it mean? I do not use prefixes (you can see how I compile on the first post of this ticket) nor /etc is in my PATH.

I meant the /etc location - something like "--sysconfdir=" when you configure glibc. Maybe, you picked binaries from other location - maybe you missed some .so binaries (/lib/libnss*?).

Maybe strace might show which files are read at runtime.

The strace result is attached. Let me know if I have to do it in a different way.
I wasn't able to mix the strace output with the TVHeadend log, anyway I hit CTRL+C when I saw the line:
[...]

Unfortunately, I don't see any open() calls for the /etc tree there. It seems a bit malformed. There should be line like:

open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
#22

Updated by virtual dj over 9 years ago

I meant the /etc location - something like "--sysconfdir=" when you configure glibc. Maybe, you picked binaries from other location - maybe you missed some .so binaries (/lib/libnss*?).

Actually I didn't configure glibc at all. I just copied the Debian squeeze binaries from the VM (only those reported by ldd tvheadend) on the QNAP and run the application from there using ld-linux-....

I've tried copying libnss too as you said, but it doesn't change anything (I don't think it picks them up at all, even if it's in the LD_LIBRARY_PATH).

Also "libc/ld-linux-x86-64.so.2 --list" reports that all the libraries are correctly loaded from the path where I've copied the files.

Unfortunately, I don't see any open() calls for the /etc tree there. It seems a bit malformed.

I do not see them too.

strace -e open,access libc/ld-linux-x86-64.so.2 --library-path /share/Public/mytest/libc ./tvheadend -d -u admin -g administrators -C

gives:
open(0xc0384b63, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016027320) = 0
open("libssl.so.0.9.8", O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016026060) = 0
open("libcrypto.so.0.9.8", O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016026000) = 0
open("libz.so.1", O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016025720) = 0
open("libdl.so.2", O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016025640) = 0
open("libpthread.so.0", O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016025560) = 0
open("libm.so.6", O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016025500) = 0
open("librt.so.1", O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016025420) = 0
open("libc.so.6", O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016025340) = 0
open(0xc0382630, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 02300320100) = 0
open(0x142fd020, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016023260) = 0
open(0x142fced0, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016023260) = 0
open(0x3, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016022600) = 0
open(0x142fd600, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016024140) = 0
open(0x142fd600, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016023420) = 0
open(0x142fe3e0, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016023400) = 0
open(0xc0382840, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016017440) = 0
open(0x3, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016016060) = 0
open(0x3, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016017320) = 0
open("/root/.hts/tvheadend/.lock", O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 030016021640) = 0
2015-03-28 11:03:47.840 [   INFO] main: Log started
2015-03-28 11:03:48.159 [WARNING] START: configuration path /root/.hts/tvheadend is not r/w for UID:1 GID:1 [e=Permission denied], settings will not be saved
2015-03-28 11:03:48.160 [  ALERT] settings: Unable to create dir "/root/.hts/tvheadend": Permission denied
2015-03-28 11:03:48.160 [  ALERT] settings: Unable to create dir "/root/.hts/tvheadend": Permission denied
2015-03-28 11:03:48.160 [  ALERT] settings: Unable to create dir "/root/.hts/tvheadend": Permission denied
2015-03-28 11:03:48.161 [   INFO] bouquet: new bouquet 'Canal Digitaal SD'
2015-03-28 11:03:48.161 [  ALERT] settings: Unable to create dir "/root/.hts/tvheadend": Permission denied
2015-03-28 11:03:48.161 [   INFO] bouquet: new bouquet 'Canal Digitaal HD'
2015-03-28 11:03:48.161 [  ALERT] settings: Unable to create dir "/root/.hts/tvheadend": Permission denied
2015-03-28 11:03:48.161 [   INFO] bouquet: new bouquet 'TV Vlaanderen SD'
2015-03-28 11:03:48.161 [  ALERT] settings: Unable to create dir "/root/.hts/tvheadend": Permission denied
... cut ...
2015-03-28 11:03:48.372 [   INFO] scanfile: DVB-T - loaded 43 regions with 729 networks
2015-03-28 11:03:48.373 [   INFO] scanfile: DVB-S - loaded 1 regions with 93 networks
2015-03-28 11:03:48.373 [   INFO] scanfile: DVB-C - loaded 16 regions with 51 networks
2015-03-28 11:03:48.373 [   INFO] scanfile: ATSC - loaded 2 regions with 14 networks
2015-03-28 11:03:48.373 [  DEBUG] fsmonitor: watch /dev
2015-03-28 11:03:48.373 [   INFO] dvr: Creating new configuration ''
2015-03-28 11:03:48.373 [  ALERT] settings: Unable to create dir "/root/.hts/tvheadend": Permission denied
2015-03-28 11:03:48.373 [WARNING] dvr: Output directory for video recording is not yet configured for DVR configuration "". Defaulting to to "/root". This can be changed from the web user interface.
2015-03-28 11:03:48.373 [  ALERT] settings: Unable to create dir "/root/.hts/tvheadend": Permission denied
2015-03-28 11:03:48.373 [  ALERT] settings: Unable to create dir "/root/.hts/tvheadend": Permission denied
2015-03-28 11:03:48.373 [WARNING] access: Created default wide open access controle entry
2015-03-28 11:03:48.374 [   INFO] CSA: Using SSE2 128bit parallel descrambling
... cut ...
2015-03-28 11:03:48.382 [  DEBUG] epgdb: database does not exist
2015-03-28 11:03:48.382 [ NOTICE] START: HTS Tvheadend version 3.9.2660~g169b71b-dirty started, running as PID:1783 UID:1 GID:1, CWD:/share/CACHEDEV1_DATA/Public/mytest CNF:/root/.hts/tvheadend
--- SIGABRT (Aborted) @ 0 (0) ---
--- SIGABRT (Aborted) @ 0 (0) ---
+++ killed by SIGABRT +++
Aborted

What are those HEX strings passed as the first parameter of the open call?

#23

Updated by virtual dj over 9 years ago

I made another test; please pardon me if it's totally wrong I just want to help you (I'm not a programmer and I only used Google).
I compiled the following program in the Debian VM (the same used to compile TVHeadend):

root@debian-x64:~/tvheadend/tvheadend# cat utest.c 

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>

int main(int argc, char *argv[])
{
        uid_t id = getuid();
        printf("Current UID = %d\n", id); 
        return 0;
}
Then I moved the a.out binary to the QNAP and I got:
[/share/Public/mytest] # ./a.out 
Current UID = 0
[/share/Public/mytest] # libc/ld-linux-x86-64.so.2 --library-path /share/Public/mytest/libc ./a.out 
Current UID = 0
[/share/Public/mytest] # su guest
[/share/Public/mytest] # ./a.out 
Current UID = 65534
[/share/Public/mytest] # exit
exit
It seems that it's working correctly, isn't it?
And this is the strace output.
[/share/Public/mytest] # strace -e open,access ./a.out 
open(0x3, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 07071523140) = 0
open("libc.so.6", O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 07071523260) = 0
open(0x26066780, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 07071523260) = 0
Current UID = 0

[/share/Public/mytest] # strace -e open,access libc/ld-linux-x86-64.so.2 --library-path /share/Public/mytest/libc ./a.out 
open(0x5b0ebb88, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 013303512460) = 0
open("libc.so.6", O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 013303511220) = 0
open(0x99062780, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 013303511260) = 0
Current UID = 0
Again, no "/etc/passwd" at all. :(

#24

Updated by Jaroslav Kysela over 9 years ago

You should test getgrnam() and getpwnam() functions, not getuid() - for getuid() the /etc files are not accessed, becauase this information is obtained from kernel using syscalls.

#25

Updated by virtual dj over 9 years ago

You should test getgrnam() and getpwnam() functions, not getuid() - for getuid() the /etc files are not accessed, becauase this information is obtained from kernel using syscalls.

Thanks for the explanation, I've updated the a.out binary with the following source code:

root@debian-x64:~/tvheadend/tvheadend# cat utest.c

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <pwd.h>

int main(int argc, char *argv[])
{
        struct passwd *user;

        if ((user= getpwnam(argv[1])) == NULL)
                printf("User not found.\n");
        else
                printf("User %s ID = %d.\n", user->pw_name, user->pw_uid);
        return 0;
}
On the Debian VM:
root@debian-x64:~/tvheadend/tvheadend# strace -e open ./a.out root
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/libc.so.6", O_RDONLY)        = 3
open("/etc/nsswitch.conf", O_RDONLY)    = 3
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/libnss_compat.so.2", O_RDONLY) = 3
open("/lib/libnsl.so.1", O_RDONLY)      = 3
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/libnss_nis.so.2", O_RDONLY)  = 3
open("/lib/libnss_files.so.2", O_RDONLY) = 3
open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
User root ID = 0.
root@debian-x64:~/tvheadend/tvheadend# strace -e open ./a.out web
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/libc.so.6", O_RDONLY)        = 3
open("/etc/nsswitch.conf", O_RDONLY)    = 3
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/libnss_compat.so.2", O_RDONLY) = 3
open("/lib/libnsl.so.1", O_RDONLY)      = 3
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/libnss_nis.so.2", O_RDONLY)  = 3
open("/lib/libnss_files.so.2", O_RDONLY) = 3
open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
User web ID = 1000.
While on the QNAP:
[/share/Public/mytest] # strace -e open libc/ld-linux-x86-64.so.2 --library-path /share/Public/mytest/libc ./a.out admin
open(0xeaa24b96, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 035250434320) = 0
open("libc.so.6", O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 035250433060) = 0
open(0xfa36b420, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 035250436140) = 0
open(0xeaa23da0, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 035250432200) = 0
open(0x3, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 035250430620) = 0
open(0x3, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 035250432060) = 0
open(0xfa143780, O_ACCMODE|O_CREAT|O_EXCL|O_NOCTTY|O_TRUNC|O_APPEND|O_NONBLOCK|O_SYNC|O_ASYNC|O_DIRECT|O_LARGEFILE|O_DIRECTORY|O_NOFOLLOW|O_NOATIME|0xfff8003c, 035250436700) = 0
User not found.
So it doesn't seem to load the files, even though libnss is there:
[/share/Public/mytest] # ls libc/
ld-linux-x86-64.so.2*     libnss_dns-2.11.3.so      libnss_nis.so.2
libc.so.6*                libnss_dns.so.2           libnss_nisplus-2.11.3.so
libcrypto.so.0.9.8        libnss_files-2.11.3.so    libnss_nisplus.so.2
libdl.so.2                libnss_files.so.2         libpthread.so.0*
libm.so.6                 libnss_hesiod-2.11.3.so   librt.so.1
libnss_compat-2.11.3.so   libnss_hesiod.so.2        libssl.so.0.9.8
libnss_compat.so.2        libnss_nis-2.11.3.so      libz.so.1
How can I force it to load libnss_compat.so.2?
On the QNAP the file /etc/nsswitch.conf is there and reports compat for passwd. But I must use the libs that comes from the Debian VM or TVHeadend won't run due to the old glibc that's on the QNAP firmware.

#26

Updated by virtual dj over 9 years ago

Aha, I think I've found the problem myself... It was libnsl.so.1 that was missing.

[/share/Public/mytest] # ls libc/
ld-linux-x86-64.so.2* libm.so.6             librt.so.1
libc.so.6*            libnsl.so.1           libssl.so.0.9.8
libcrypto.so.0.9.8    libnss_compat.so.2    libz.so.1
libdl.so.2            libpthread.so.0*
Now:
[/share/Public/mytest] # libc/ld-linux-x86-64.so.2 --library-path /share/Public/mytest/libc ./a.out admin
User admin ID = 0.
We now have the correct output, I think:
[/share/Public/mytest] # libc/ld-linux-x86-64.so.2 --library-path /share/Public/mytest/libc ./tvheadend -d -u admin -g administrators -C
2015-03-28 21:24:04.538 [   INFO] main: Log started
2015-03-28 21:24:04.556 [  DEBUG] config: no configuration, loading defaults
2015-03-28 21:24:04.596 [  DEBUG] settings: saving to /share/CACHEDEV1_DATA/homes/admin/.hts/tvheadend/config
2015-03-28 21:24:04.596 [   INFO] config: loaded
2015-03-28 21:24:04.597 [  DEBUG] settings: saving to /share/CACHEDEV1_DATA/homes/admin/.hts/tvheadend/profile/6eefe2c679eabb03252b98094b224e38
2015-03-28 21:24:04.597 [  DEBUG] settings: saving to /share/CACHEDEV1_DATA/homes/admin/.hts/tvheadend/profile/3195b8bde7e6394c9542a6eb42e4db7c
2015-03-28 21:24:04.597 [  DEBUG] settings: saving to /share/CACHEDEV1_DATA/homes/admin/.hts/tvheadend/profile/1caaf0095f58eded731f62076ec1dcce
2015-03-28 21:24:04.598 [   INFO] bouquet: new bouquet 'Canal Digitaal SD'
2015-03-28 21:24:04.598 [  DEBUG] settings: saving to /share/CACHEDEV1_DATA/homes/admin/.hts/tvheadend/bouquet/dd04a1b29a3e247f73597897393a0289
... cut ...
2015-03-28 21:24:04.601 [  DEBUG] settings: saving to /share/CACHEDEV1_DATA/homes/admin/.hts/tvheadend/bouquet/b860a0c6c5e0d7edcbbaf05fa00c2120
2015-03-28 21:24:04.806 [   INFO] scanfile: DVB-T - loaded 43 regions with 729 networks
2015-03-28 21:24:04.806 [   INFO] scanfile: DVB-S - loaded 1 regions with 93 networks
2015-03-28 21:24:04.806 [   INFO] scanfile: DVB-C - loaded 16 regions with 51 networks
2015-03-28 21:24:04.806 [   INFO] scanfile: ATSC - loaded 2 regions with 14 networks
2015-03-28 21:24:04.807 [  DEBUG] fsmonitor: watch /dev
2015-03-28 21:24:04.807 [   INFO] dvr: Creating new configuration ''
2015-03-28 21:24:04.807 [  DEBUG] settings: saving to /share/CACHEDEV1_DATA/homes/admin/.hts/tvheadend/dvr/config/b678ae36e35780443c22a3fc9658f179
2015-03-28 21:24:04.807 [WARNING] dvr: Output directory for video recording is not yet configured for DVR configuration "". Defaulting to to "/share/homes/admin". This can be changed from the web user interface.
2015-03-28 21:24:04.807 [  DEBUG] settings: saving to /share/CACHEDEV1_DATA/homes/admin/.hts/tvheadend/accesscontrol/4d445e48c5e13117f07bd3ea6fcc934b
2015-03-28 21:24:04.808 [  DEBUG] settings: saving to /share/CACHEDEV1_DATA/homes/admin/.hts/tvheadend/accesscontrol/4d445e48c5e13117f07bd3ea6fcc934b
2015-03-28 21:24:04.863 [WARNING] access: Created default wide open access controle entry
2015-03-28 21:24:04.863 [   INFO] CSA: Using SSE2 128bit parallel descrambling
2015-03-28 21:24:04.863 [   INFO] epggrab: module eit created
... cut ...
2015-03-28 21:24:04.871 [  DEBUG] epggrab: using internal grab search
2015-03-28 21:24:04.871 [  DEBUG] epgdb: database does not exist
2015-03-28 21:24:04.872 [  DEBUG] settings: saving to /share/CACHEDEV1_DATA/homes/admin/.hts/tvheadend/config
2015-03-28 21:24:04.872 [ NOTICE] START: HTS Tvheadend version 3.9.2660~g169b71b-dirty started, running as PID:10179 UID:0 GID:0, CWD:/share/CACHEDEV1_DATA/Public/mytest CNF:/share/CACHEDEV1_DATA/homes/admin/.hts/tvheadend
Aborted

#27

Updated by Jaroslav Kysela over 9 years ago

  • Status changed from New to Fixed

Yes, time to close this bug as fixed ? I think so...

Also available in: Atom PDF