Tvheadend

h1. Home Tv Streaming Protocol (HTSP) - Version 8

h2. General

HTSP is a TCP based protocol primarily intended for streaming of live TV and related meta data such as channels, group of channels (called tags in HTSP) and electronic program guide (EPG) information.

The transmission and reception of a channel over HTSP is referred to as a subscription. A single HTSP session can handle as many concurrent subscriptions as the bandwidth and CPU permits.

The HTSP server in tvheadend has a payload-aware scheduler for prioritizing more important packets (such as I-frames) before less important ones (such as B-frames). This makes HTSP suitable for long-distance transmissions and/or paths with non-perfect delivery.

(It has been tested with a server in Stockholm and the client in Berlin).

For information about the HTSP wire format please read [[htsmsgbinary|HTSMSG binary format]]

If you're looking to develop a new client, there are several existing client implementations from which you might be able to gain knowledge:

* "XBMC":https://github.com/opdenkamp/xbmc/tree/master/xbmc/pvrclients/tvheadend

* "Showtime":https://github.com/andoma/showtime/tree/master/src/backend/htsp

* "TVHGuide":https://github.com/john-tornblom/TVHGuide

* "PyHTSP":https://github.com/adamsutton/tvheadend/tree/master/lib/py/tvh/htsp.py (This is a demo client and is WIP, it has limited functionality).

----

h2. Communication

This communication is currently implemented by using htsmsg's. All strings are encoded as UTF-8.

There are two distinct ways for communication within HTSP.

Apart from this there is a number of messages that needs to be exchanged during login, see the login section below.

h3. RPC communication

There is a normal RPC way of doing things. I.e. the client sends a request and the server responds with a reply. All the RPC methods are listed below as the 'Client to Server' methods. Apart from all message fields listed within each message type the client can add an additional field:

RPC request extra fields:

<pre>

seq              int  optional   Sequence number. This field will be echoed back by the server in the reply.

username         str  optional   Username, in combination with 'digest' this can be used to raise the privileges

                                 for the session in combination with invocation of a method. 

digest           bin  optional   Used to raise privileges.

</pre>

The followings field should be used by the client to match the reply with the request.

All replies are guaranteed to arrive in the same order as the requests.

Even so, probably the best way to implement the request-reply client is by taking advantage of the 'seq' field.

RPC reply extra fields:

<pre>

seq              int  optional   Sequence number. Same as in the request.

error            str  optional   If present an error has occurred and the text describes the error.

noaccess         int  optional   If present and set to '1' the user is prohibited from invoking the method due to 

                                 access restrictions. 

</pre>

h3. Streaming communication

For streaming of live TV and various related messages the server will continuously push data to the client.

These messages are referred to as asynchronous messages and always have the 'method' field set and never have the 'seq' field set.

Also, the client can enable an additional asyncMetadata mode and by doing so it will be notified by the server when meta data changes. (EPG updates, creation of channels and tags, etc). 

h3. Authentication

In Tvheadend, each method has an associated access restriction. Currently there is only one restriction (Streaming). However, this may change in the future.

Privileges for these restrictions may be granted in two ways: Username + Password and/or Source IP address.

Therefore it is possible to gain permissions to the system without entering a username and password.

While this is really useful it also complicates the authentication schema a bit.

Upon connect the initial privileges will be raised based on the source address.

Before any username / password based authentication has taken place the client must have

obtained a challenge (which stays fixed for the session). This is done via the 'hello' method.

In principle it's possible to use two different authentication idioms with HTSP.

Depending on how your application works one or another may be more suitable.

While they do not really differ from a protocol point of view it's worth mentioning a bit about them here:

h3. Initial login authentication

The client performs all of its authentication using the 'login' method.

It may choose to send:

* Username and password: Privileges will be raised based on these credentials.

* Username only: Privileges will be based on just the source address. The username will be used for various logging purposes.

* Nothing: Privileges will be based on just the source address.

If no privileges are granted after the login message has been received by the server (i.e. both network and username + password based)

the server will reply with 'noaccess' set to 1. A client that only employs initial login should honor this flag and ask the

user for a username + password and retry by using the 'authenticate' method. I.e. it should not send the 'login' method again.

h3. On-demand authentication

The client performs all of its authentication when it needs to.

When using this method, the client will check every RPC reply for the 'noaccess' field.

If it set to 1 it whould ask the user for username + password and retry the request but also

add 'username' and 'digest' to the original message. (See _RPC request extra fields_ above)

Typically it would not send a username or digest during login.

----

h2. Client to Server (RPC) methods

h3. hello

Used to identify the client toward the server and to get the session challenge used to

hash passwords into digests. The client can request a different version of the HTSP

protocol with this method. If no 'hello' message is sent the server assumes latest version

is to be used.

Client/Server should select lowest common version, if this is not possible connection should be terminated.

Request message fields:

<pre>

htspversion        u32   required   Client preferred HTSP version.

clientname         str   required   Client software name.

clientversion      str   required   Client software version.

</pre>

Reply message fields:

<pre>

htspversion        u32   required   The server supports all versions of the protocol up to and including this number.

servername         str   required   Server software name.

serverversion      str   required   Server software version.

servercapability   str[] required   Server capabilities (Added in version 6)

challenge          bin   required   32 bytes randomized data used to generate authentication digests

</pre>

Note: possible values for servercapability[]:

<pre>

cwc                Descrambling available

v4l                Analogue TV available

linuxdvb           Linux DVB API available

imagecache         Image caching available

</pre>

h3. authenticate

This can be used to issue authentication without doing anything else.

If no privileges are gained it will return with 'noaccess' set to 1.

Request message fields:

<pre>

None   

</pre>

Reply message fields:

<pre>

noaccess           u32   optional   If set to 1, no privileges were granted.

</pre>

----

h3. getDiskSpace (Added in version 3)

Return diskspace status from Tvheadend's PVR storage

Request message fields:

<pre>

None

</pre>

Reply message fields:

<pre>

freediskspace      s64   required   Bytes available.

totaldiskspace     s64   required   Total capacity.

</pre>

h3. getSysTime (Added in version 3)

Return system time on the server.

Request message fields:

<pre>

None

</pre>

Reply message fields:

<pre>

time               s64  required   UNIX time.

timezone           s32  required   Minutes west of GMT.

</pre>

----

h3. enableAsyncMetadata

When this is enabled the client will get continuous updates from the server about added, update or deleted channels, tags, dvr and epg entries.

An interactive application that presents the user with information about these things should probably enable this and the implement the various server to client methods.

Request message fields:

<pre>

epg                u32   optional   Set to 1, to include EPG data in async, implied by epgMaxTime (Added in version 6).

lastUpdate         s64   optional   Only provide metadata that has changed since this time (Added in version 6).

epgMaxTime         s64   optional   Maximum time to return EPG data up to (Added in version 6)

language           str   optional   RFC 2616 compatible language list (Added in version 6)

</pre>

Reply message fields:

<pre>

None

</pre>

Once the reply as been sent the initial data set will be provided, and then updates will arrive asynchronously after that. The initial data dump is sent using the following messages:

<pre>

tagAdd                   optional

channelAdd               optional

tagUpdate                optional

dvrEntryAdd              optional

eventAdd                 optional   (Added in version 6)

initialSyncComplete      required

</pre>

----

h3. getEvent

Request information about the given event. An event typically corresponds to a program on a channel.

The language field in the request allows preference for languages to be requested for the various string fields.

Request message fields:

<pre>

eventId            u32   required   Event ID.

language           str   optional   RFC 2616 compatible language list (Added in version 6)

</pre>

Reply message fields:

<pre>

see eventAdd

</pre>

h3. getEvents (Added in version 4)

Request information about a set of events. If no options are specified the entire EPG database will be returned.

Request message fields:

<pre>

eventId            u32   optional   Event ID to begin from (Optional since version 6)

channelId          u32   optional   Channel ID to get data for (Added in version 6)

numFollowing       u32   optional   Number of events to add (Optional since version 6)

maxTime            u64   optional   Maximum time to return data up to (Added in version 6)

language           str   optional   RFC 2616 compatible language list (Added in version 6)

</pre>

Reply message fields:

<pre>

events             msg[] required   List of events, using response message fields from eventAdd

</pre>

h3. epgQuery (Added in version 4)

Query the EPG (event titles) and optionally restrict to channel/tag/content type.

Request message fields:

<pre>

query              str   required  Title regular expression to search for

channelId          u32   optional  [[ChannelId]] to restrict search to

tagId              u32   optional  [[TagId]] to restrict search to

contentType        u32   optional  DVB content type to restrict search to

language           str   optional  RFC 2616 compatible language list for title (Added in version 6)

full               u32   optional  Output full event list rather than just IDs

</pre>

Reply message fields:

if full == 1

<pre>

events             msg[] optional   List of events, using response message fields from eventAdd

</pre>

else

<pre>

eventIds           u32[] optional  List of eventIds that match the query

</pre>

h3. getEpgObject

Get detailed EPG Object info.

Request message fields:

<pre>

id                 u32   required  Object ID

type               u32   optional  Object type

</pre>

Reply message fields:

<pre>

TODO

</pre>

----

h3. addDvrEntry (Added in version 4)

Create a new DVR entry. Either eventId or channelId, start and stop must be specified.

Request message fields:

<pre>

eventId            u32   optional   Event ID (Optional since version 5).

channelId          u32   optional   Channel ID (Added in version 5)

start              s64   optional   Time to start recording (Added in version 5)

stop               s64   optional   Time to stop recording (Added in version 5)

creator            str   optional   Name of the event creator (Added in version 5)

priority           u32   optional   Recording priority (Added in version 5)

startExtra         s64   optional   Pre-recording buffer in minutes (Added in version 5)

stopExtra          s64   optional   Post-recording buffer in minutes (Added in version 5)

title              str   optional   Recording title, if no eventId (Added in version 6)

description        str   optional   Recording description, if no eventId (Added in version 5)

configName         str   optional   DVR configuration name

</pre>

Reply message fields:

<pre>

success            u32   required   1 if entry was added, 0 otherwise

id                 u32   optional   ID of created DVR entry

error              str   optional   English clear text of error message

</pre>

h3. updateDvrEntry (Added in version 5)

Update an existing DVR entry.

Request message fields:

<pre>

id                 u32   required   DVR Entry ID

start              s64   optional   New start time

stop               s64   optional   New stop time

title              str   optional   New entry title

description        str   optional   New entry description (Added in version 6)

startExtra         s64   optional   New pre-record buffer (Added in version 6)

stopExtra          s64   optional   New post-record buffer (Added in version 6)

configName         str   optional   New DVR configuration name

</pre>

Reply message fields:

<pre>

success            u32   required   1 if update as successful, otherwise 0

error              str   optional   Error message if update failed

</pre>

h3. cancelDvrEntry (Added in version 5)

Cancel an existing recording, but don't remove the entry from the database.

Request message fields:

<pre>

id                 u32   required   dvrEnryId to delete

</pre>

Reply message fields:

<pre>

success            int   required   1 if entry was cancelled

error              str   optional   Error message if cancellation failed

</pre>

h3. deleteDvrEntry (Added in version 4)

Delete an existing DVR entry from the database.

Request message fields:

<pre>

id                 u32   required   DVR Entry ID

</pre>

Reply message fields:

<pre>

success            u32   required   1 if entry was removed

error              str   optional   Error message if the delete fails

</pre>

----

h3. getTicket (Added in version 5)

Get a ticket to allow access to a channel or recording via HTTP

Request message fields:

<pre>

channelId          u32  optional   Channel to gain access for

dvrId              u32  optional   DVR file to gain access for

</pre>

Reply message fields:

<pre>

path               str  required   The full path for access URL (no scheme, host or port)

ticket             str  required   The ticket to pass in the URL query string

</pre>

h3. subscribe

Request subscription to the given channel. 

Request message fields:

<pre>

channelId          u32   required   ID for channel. 

subscriptionId     u32   required   Subscription ID. Selected by client. This value is not interpreted by the server in any form. 

                                    The value is used from now on in all messages related to the subscription.

weight             u32   optional   Weighting of this subscription (same as recording priority).

</pre>

Reply message fields:

<pre>

None

</pre>

h3. unsubscribe

Stop a subscription.

Request message fields:

<pre>

subscriptionId     u32   required   Subscription ID.

</pre>

Reply message fields:

<pre>

None

</pre>

h3. subscriptionChangeWeight (Added in version 5)

Change the weight of an existing subscription

Request message fields:

<pre>

subscriptionId     u32   required   Subscription ID.

weight             u32   optional   The new subscription weight.

</pre>

Reply message fields:

<pre>

None

</pre>

h3. fileOpen (Added in version 8)

Open a file within the Tvheadend file system. This is now the preferred method (in place of HTTP) for accessing recordings.

Accessing recordings via HTSP will overcome the limitations of standard HTTP streaming which cannot handle both skipping and growing files (i.e. in-progress recordings) at the same time.

Request message fields:

<pre>

file               str   required   File path to open

</pre>

Valid file paths:

<pre>

/dvrfile/ID        will open the file associated with DVR entry ID

/imagecache/ID     will open the file associated with imagecache entry ID

</pre>

Reply message fields:

<pre>

id                 u32   required   The file handle used for further file operations

size               u64   optional   The size of the file in bytes

mtime              u64   optional   The last time the file was modified

</pre>

h3. fileRead (Added in version 8)

Read data from a file.

Request message fields:

<pre>

id                 u32   required   The file handle used for further file operations

size               u64   required   The amount of data to read

offset             u64   optional   Offset into the file to read (default is current position)

</pre>

Reply message fields:

<pre>

data               bin   required   The data read from the file (size may be less than requested)

</pre>

h3. fileClose (Added in version 8)

Close an opened file.

Request message fields:

<pre>

id                 u32   required   The file handle to be closed

</pre>

Reply message fields:

<pre>

None

</pre>

h3. fileStat (Added in version 8)

Get status of a file.

Request message fields:

<pre>

id                 u32   required   The file handle to be stat'd

</pre>

Reply message fields:

<pre>

size               u64   optional   The size of the file in bytes

mtime              u64   optional   The last time the file was modified

</pre>

h3. fileSeek (Added in version 8)

Seek to position in a file.

Request message fields:

<pre>

id                 u32   required   The file handle to be stat'd

offset             u64   required   The position to seek in the file

whence             str   required   Where to seek relative to (default=SEEK_SET)

</pre>

Note: valid values for whence

<pre>

SEEK_SET           Seek relative to start of file

SEEK_CUR           Seek relative to current position in file

SEEK_END           Seek relative (backwards) to end of file

</pre>

Reply message fields:

<pre>

offset             u64   optional   The new absolute position within the file

</pre>

h1. Server to Client methods

----

h3. channelAdd

A new channel has been created on the server.

Message fields:

<pre>

channelId          u32   required   ID of channel.

channelNumber      u32   required   Channel number, 0 means unconfigured.

channelName        str   required   Name of channel.

channelIcon        str   optional   URL to an icon representative for the channel

                                    (For v8+ clients this could be a relative /imagecache/ID URL

                                     intended to be fed to fileOpen().)

eventId            u32   optional   ID of the current event on this channel.

nextEventId        u32   optional   ID of the next event on the channel.

tags               u32[] optional   Tags this channel is mapped to.

services           msg[] optional   List of available services (Added in version 5)

</pre>

Service fields:

<pre>

name               str   required   Service name

type               str   required   Service type

caid               u32   optional   Encryption CA ID

caname             str   optional   Encryption CA name

</pre>

h3. channelUpdate

Same as channelAdd, but all fields (except channelId) are optional.

h3. channelDelete

A channel has been deleted on the server.

Message fields:

<pre>

channelId          u32   required   ID of channel.

</pre>

----

h3. tagAdd

A new tag has been created on the server.

Message fields:

<pre>

tagId              u32   required   ID of tag.

tagName            str   required   Name of tag.

tagIcon            str   optional   URL to an icon representative for the channel.

tagTitledIcon      u32   optional   Icon includes a title

members            u32[] optional   Channel IDs of those that belong to the tag

</pre>

h3. tagUpdate

Same as tagAdd, but all fields (except tagId) are optional.

h3. tagDelete

A tag has been deleted from the server.

Message fields:

<pre>

tagId              u32   required   ID of tag.

</pre>

----

h3. dvrEntryAdd (Added in version 4)

A new recording has been created on the server.

Message fields:

<pre>

id                 u32   required   ID of dvrEntry.

channel            u32   required   Channel of dvrEntry.

start              s64   required   Time of when this entry was scheduled to start recording.

stop               s64   required   Time of when this entry was scheduled to stop recording.

eventId            u32   optional   Associated EPG Event ID.

title              str   optional   Title of recording

summary            str   optional   Short description of the recording (Added in version 6).

description        str   optional   Long description of the recording.

state              str   required   Recording state

error              str   optional   Plain english error description (e.g. "Aborted by user").

</pre>

TODO: recording states

h3. dvrEntryUpdate

Message fields:

<pre>

Same as dvrEntryAdd, but all fields (except id) are optional.

</pre>

h3. dvrEntryDelete (Added in version 4)

A recording has been deleted from the server.

Message fields:

<pre>

id                 u32   required   ID of recording to delete.

</pre>

----

h3. eventAdd (Added in version 6)

Message fields:

<pre>

eventId            u32   required   Event ID

channelId          u32   required   The channel this event is related to.

start              u64   required   Start time of event, UNIX time.

stop               u64   required   Ending time of event, UNIX time.

title              str   optional   Title of event.

summary            str   optional   Short description of the event (Added in version 6).

description        str   optional   Long description of the event.

serieslinkId       u32   optional   Series Link ID (Added in version 6).

episodeId          u32   optional   Episode ID (Added in version 6).

seasonId           u32   optional   Season ID (Added in version 6).

brandId            u32   optional   Brand ID (Added in version 6).

contentType        u32   optional   DVB content code (Added in version 4, Modified in version 6*).

ageRating          u32   optional   Minimum age rating (Added in version 6).

starRating         u32   optional   Star rating (1-5) (Added in version 6).

firstAired         s64   optional   Original broadcast time, UNIX time (Added in version 6).

seasonNumber       u32   optional   Season number (Added in version 6).

seasonCount        u32   optional   Show season count (Added in version 6).

episodeNumber      u32   optional   Episode number (Added in version 6).

episodeCount       u32   optional   Season episode count (Added in version 6).

partNumber         u32   optional   Multi-part episode part number (Added in version 6).

partCount          u32   optional   Multi-part episode part count (Added in version 6).

episodeOnscreen    str   optional   Textual representation of episode number (Added in version 6).

image              str   optional   URL to a still capture from the episode (Added in version 6).

dvrId              u32   optional   ID of a recording (Added in version 5).

nextEventId        u32   optional   ID of next event on the same channel.

</pre>

* *contentType previously had the major DVB category in the bottom 4 bits, 

  however in v6 this has been changed to properly match DVB, so top 4 bits

  as major category and bottom 4 bits has sub-category. Clients requesting

  v5 or lower will get the old output.

h3. eventUpdate (Added in version 6)

Message fields:

<pre>

Same as eventAdd but all fields (except eventId) are optional.

</pre>

h3. eventDeleted (Added in version 6)

Message fields:

<pre>

eventId            u32   required   Event ID

</pre>

---

h3. initialSyncCompleted (Added in version 2)

Sent after all the initial metadata has been sent when session has been set to async mode.

Message fields:

<pre>

</pre>

----

h3. subscriptionStart

Asynchronous message output when a new subscription is successfully started.

Message fields:

<pre>

subscriptionId     u32   required   Subscription ID.

streams            msg[] required   Array of messages with stream information

sourceinfo         msg   optional   Source information.

</pre>

Stream message fields:

<pre>

index              u32   required   Index for this stream

type               str   required   Type of stream

language           str   optional   Language for stream

Video components only:

width              u32   optional   Width of video in pixels

height             u32   optional   Height of video in pixels

aspect_num         u32   optional   Aspect ratio numerator (Added in version 5) *Can be incorrect and should not be relied upon*

aspect_den         u32   optional   Aspect ratio denonminator (Added in version 5) *Can be incorrect and should not be relied upon*

Audio components only:

channels           u32   optional   Number of audio channels (Added in version 5)

rate               u32   optional   Audio bitrate (Added in version 5)

Subtitle components only:

composition_id     u32   optional   ??? (Added in version 5)

ancillary_id       u32   optional   ??? (Added in version 5)

</pre>

Sourceinfo message fields:

<pre>

adapter            str   optional   Adapter name

mux                str   optional   Transponder name

network            str   optional   Network name

provider           str   optional   ???

service            str   optional   Service name

</pre>

Video Stream types:

<pre>

MPEG2VIDEO                    MPEG2 video

H264                          H264 video

</pre>

Audio Stream types:

<pre>

MPEG2AUDIO                    MPEG2 audio (MP2)

AC3                           AC3 audio

AAC                           ADTS framed AAC (one AAC packet per ADTS frame)

EAC3                          Enhanced AC3 audio

</pre>

Subtitle Stream types:

<pre>

TELETEXT                      ???

DVBSUB                        ???

</pre>

h3. subscriptionStop

A subscription has been stopped.

Message fields:

<pre>

subscriptionId     u32   required   Subscription ID.

status             str   optional   Error message if subscription stopped unexpectedly.

</pre>

h3. subscriptionStatus

Subscription status update.

Message fields:

<pre>

subscriptionId     u32   required   Subscription ID.

status             str   optional   English clear text of error status. Absence of this field means that the status is OK. 

</pre>

h3. queueStatus

The queueStatus message is sent every second during normal data delivery.

The transmit scheduler have different drop thresholds for different frame types.

If congestion occurs it will favor dropping B-frames before P-frames before I-frames.

All audio is recognized as I-frames. 

Message fields:

<pre>

subscriptionId     u32   required   Subscription ID.

packets            u32   required   Number of data packets in queue.

bytes              u32   required   Number of bytes in queue.

delay              u32   optional   Estimated delay of queue (in µs).

Bdrops             u32   required   Number of B-frames dropped

Pdrops             u32   required   Number of P-frames dropped

Idrops             u32   required   Number of I-frames dropped

</pre>

h3. signalStatus

The signalStatus message is sent every second during normal data delivery.

The optional fields may not have been implemented or may not be supported by the active adapter.

Message fields:

<pre>

subscriptionId     u32   required   Subscription ID.

feStatus           str   required   Frontend status.

feSNR              u32   optional   Signal to noise ratio.

feSignal           u32   optional   Signal status percentage.

feBER              u32   optional   Bit error rate.

feUNC              u32   optional   Uncorrected blocks.

</pre>

h3. muxpkt

Streaming data.

Message fields:

<pre>

subscriptionId     u32   required   Subscription ID.

frametype          u32   required   Type of frame as ASCII value: 'I', 'P', 'B'

stream             u32   required   Stream index. Corresponds to the streams reported in the subscriptionStart message.

dts                s64   optional   Decode Time Stamp in µs.

pts                s64   optional   Presentation Time Stamp in µs.

duration           u32   required   Duration of frame in µs.

payload            bin   required   Actual frame data.

</pre>