Home Tv Streaming Protocol (HTSP) - Version 9¶

General¶

HTSP is a TCP based protocol primarily intended for streaming of live TV and related meta data such as channels, group of channels (called tags in HTSP) and electronic program guide (EPG) information.

The transmission and reception of a channel over HTSP is referred to as a subscription. A single HTSP session can handle as many concurrent subscriptions as the bandwidth and CPU permits.

The HTSP server in tvheadend has a payload-aware scheduler for prioritizing more important packets (such as I-frames) before less important ones (such as B-frames). This makes HTSP suitable for long-distance transmissions and/or paths with non-perfect delivery.
(It has been tested with a server in Stockholm and the client in Berlin).

For information about the HTSP wire format please read HTSMSG binary format

If you're looking to develop a new client, there are several existing client implementations from which you might be able to gain knowledge:

• XBMC
• Showtime
• TVHGuide
• PyHTSP (This is a demo client and is WIP, it has limited functionality).

---

Communication¶

This communication is currently implemented by using htsmsg's. All strings are encoded as UTF-8.

There are two distinct ways for communication within HTSP.

Apart from this there is a number of messages that needs to be exchanged during login, see the login section below.

RPC communication¶

There is a normal RPC way of doing things. I.e. the client sends a request and the server responds with a reply. All the RPC methods are listed below as the 'Client to Server' methods. Apart from all message fields listed within each message type the client can add an additional field:

RPC request extra fields:

seq              int  optional   Sequence number. This field will be echoed back by the server in the reply.
username         str  optional   Username, in combination with 'digest' this can be used to raise the privileges
                                 for the session in combination with invocation of a method. 
digest           bin  optional   Used to raise privileges.

The followings field should be used by the client to match the reply with the request.
All replies are guaranteed to arrive in the same order as the requests.
Even so, probably the best way to implement the request-reply client is by taking advantage of the 'seq' field.

RPC reply extra fields:

seq              int  optional   Sequence number. Same as in the request.
error            str  optional   If present an error has occurred and the text describes the error.
noaccess         int  optional   If present and set to '1' the user is prohibited from invoking the method due to 
                                 access restrictions. 

Streaming communication¶

For streaming of live TV and various related messages the server will continuously push data to the client.
These messages are referred to as asynchronous messages and always have the 'method' field set and never have the 'seq' field set.
Also, the client can enable an additional asyncMetadata mode and by doing so it will be notified by the server when meta data changes. (EPG updates, creation of channels and tags, etc).

Authentication¶

In Tvheadend, each method has an associated access restriction. Currently there is only one restriction (Streaming). However, this may change in the future.

Privileges for these restrictions may be granted in two ways: Username + Password and/or Source IP address.
Therefore it is possible to gain permissions to the system without entering a username and password.
While this is really useful it also complicates the authentication schema a bit.
Upon connect the initial privileges will be raised based on the source address.

Before any username / password based authentication has taken place the client must have
obtained a challenge (which stays fixed for the session). This is done via the 'hello' method.

In principle it's possible to use two different authentication idioms with HTSP.
Depending on how your application works one or another may be more suitable.
While they do not really differ from a protocol point of view it's worth mentioning a bit about them here:

Initial login authentication¶

The client performs all of its authentication using the 'login' method.

It may choose to send:

• Username and password: Privileges will be raised based on these credentials.
• Username only: Privileges will be based on just the source address. The username will be used for various logging purposes.
• Nothing: Privileges will be based on just the source address.

If no privileges are granted after the login message has been received by the server (i.e. both network and username + password based)
the server will reply with 'noaccess' set to 1. A client that only employs initial login should honor this flag and ask the
user for a username + password and retry by using the 'authenticate' method. I.e. it should not send the 'login' method again.

On-demand authentication¶

The client performs all of its authentication when it needs to.

When using this method, the client will check every RPC reply for the 'noaccess' field.
If it set to 1 it whould ask the user for username + password and retry the request but also
add 'username' and 'digest' to the original message. (See RPC request extra fields above)

Typically it would not send a username or digest during login.

---

Client to Server (RPC) methods¶

hello¶

Used to identify the client toward the server and to get the session challenge used to
hash passwords into digests. The client can request a different version of the HTSP
protocol with this method. If no 'hello' message is sent the server assumes latest version
is to be used.

Client/Server should select lowest common version, if this is not possible connection should be terminated.

Request message fields:

htspversion        u32   required   Client preferred HTSP version.
clientname         str   required   Client software name.
clientversion      str   required   Client software version.

Reply message fields:

htspversion        u32   required   The server supports all versions of the protocol up to and including this number.
servername         str   required   Server software name.
serverversion      str   required   Server software version.
servercapability   str[] required   Server capabilities (Added in version 6)
challenge          bin   required   32 bytes randomized data used to generate authentication digests
webroot            str   optional   Server HTTP webroot (Added in version 8)
                                    Note: any access to TVH webserver should include this at start of URL path

Note: possible values for servercapability[]:

cwc                Descrambling available
v4l                Analogue TV available
linuxdvb           Linux DVB API available
imagecache         Image caching available
timeshift          Timeshifting available (Added in version 9).

authenticate¶

This can be used to issue authentication without doing anything else.
If no privileges are gained it will return with 'noaccess' set to 1.

Request message fields:

None   

Reply message fields:

noaccess           u32   optional   If set to 1, no privileges were granted.

---

getDiskSpace (Added in version 3)¶

Return diskspace status from Tvheadend's PVR storage

Request message fields:

None

Reply message fields:

freediskspace      s64   required   Bytes available.
totaldiskspace     s64   required   Total capacity.

getSysTime (Added in version 3)¶

Return system time on the server.

Request message fields:

None

Reply message fields:

time               s64  required   UNIX time.
timezone           s32  required   Minutes west of GMT.

---

enableAsyncMetadata¶

When this is enabled the client will get continuous updates from the server about added, update or deleted channels, tags, dvr and epg entries.

An interactive application that presents the user with information about these things should probably enable this and the implement the various server to client methods.

Request message fields:

epg                u32   optional   Set to 1, to include EPG data in async, implied by epgMaxTime (Added in version 6).
lastUpdate         s64   optional   Only provide metadata that has changed since this time (Added in version 6).
epgMaxTime         s64   optional   Maximum time to return EPG data up to (Added in version 6)
language           str   optional   RFC 2616 compatible language list (Added in version 6)

Reply message fields:

None

Once the reply as been sent the initial data set will be provided, and then updates will arrive asynchronously after that. The initial data dump is sent using the following messages:

tagAdd                   optional
channelAdd               optional
tagUpdate                optional
dvrEntryAdd              optional
eventAdd                 optional   (Added in version 6)
initialSyncComplete      required

---

getEvent¶

Request information about the given event. An event typically corresponds to a program on a channel.

The language field in the request allows preference for languages to be requested for the various string fields.

Request message fields:

eventId            u32   required   Event ID.
language           str   optional   RFC 2616 compatible language list (Added in version 6)

Reply message fields:

see eventAdd

getEvents (Added in version 4)¶

Request information about a set of events. If no options are specified the entire EPG database will be returned.

Request message fields:

eventId            u32   optional   Event ID to begin from (Optional since version 6)
channelId          u32   optional   Channel ID to get data for (Added in version 6)
numFollowing       u32   optional   Number of events to add (Optional since version 6)
maxTime            u64   optional   Maximum time to return data up to (Added in version 6)
language           str   optional   RFC 2616 compatible language list (Added in version 6)

Reply message fields:

events             msg[] required   List of events, using response message fields from eventAdd

epgQuery (Added in version 4)¶

Query the EPG (event titles) and optionally restrict to channel/tag/content type.

Request message fields:

query              str   required  Title regular expression to search for
channelId          u32   optional  [[ChannelId]] to restrict search to
tagId              u32   optional  [[TagId]] to restrict search to
contentType        u32   optional  DVB content type to restrict search to
language           str   optional  RFC 2616 compatible language list for title (Added in version 6)
full               u32   optional  Output full event list rather than just IDs

Reply message fields:

if full == 1

events             msg[] optional   List of events, using response message fields from eventAdd

else

eventIds           u32[] optional  List of eventIds that match the query

getEpgObject¶

Get detailed EPG Object info.

Request message fields:

id                 u32   required  Object ID
type               u32   optional  Object type

Reply message fields:

TODO

---

addDvrEntry (Added in version 4)¶

Create a new DVR entry. Either eventId or channelId, start and stop must be specified.

Request message fields:

eventId            u32   optional   Event ID (Optional since version 5).
channelId          u32   optional   Channel ID (Added in version 5)
start              s64   optional   Time to start recording (Added in version 5)
stop               s64   optional   Time to stop recording (Added in version 5)
creator            str   optional   Name of the event creator (Added in version 5)
priority           u32   optional   Recording priority (Added in version 5)
startExtra         s64   optional   Pre-recording buffer in minutes (Added in version 5)
stopExtra          s64   optional   Post-recording buffer in minutes (Added in version 5)
title              str   optional   Recording title, if no eventId (Added in version 6)
description        str   optional   Recording description, if no eventId (Added in version 5)
configName         str   optional   DVR configuration name

Reply message fields:

success            u32   required   1 if entry was added, 0 otherwise
id                 u32   optional   ID of created DVR entry
error              str   optional   English clear text of error message

updateDvrEntry (Added in version 5)¶

Update an existing DVR entry.

Request message fields:

id                 u32   required   DVR Entry ID
start              s64   optional   New start time
stop               s64   optional   New stop time
title              str   optional   New entry title
description        str   optional   New entry description (Added in version 6)
startExtra         s64   optional   New pre-record buffer (Added in version 6)
stopExtra          s64   optional   New post-record buffer (Added in version 6)
configName         str   optional   New DVR configuration name

Reply message fields:

success            u32   required   1 if update as successful, otherwise 0
error              str   optional   Error message if update failed

cancelDvrEntry (Added in version 5)¶

Cancel an existing recording, but don't remove the entry from the database.

Request message fields:

id                 u32   required   dvrEnryId to delete

Reply message fields:

success            int   required   1 if entry was cancelled
error              str   optional   Error message if cancellation failed

deleteDvrEntry (Added in version 4)¶

Delete an existing DVR entry from the database.

Request message fields:

id                 u32   required   DVR Entry ID

Reply message fields:

success            u32   required   1 if entry was removed
error              str   optional   Error message if the delete fails

---

getTicket (Added in version 5)¶

Get a ticket to allow access to a channel or recording via HTTP

Request message fields:

channelId          u32  optional   Channel to gain access for
dvrId              u32  optional   DVR file to gain access for

Reply message fields:

path               str  required   The full path for access URL (no scheme, host or port)
ticket             str  required   The ticket to pass in the URL query string

subscribe¶

Request subscription to the given channel.

Request message fields:

channelId          u32   required   ID for channel. 
subscriptionId     u32   required   Subscription ID. Selected by client. This value is not interpreted by the server in any form. 
                                    The value is used from now on in all messages related to the subscription.
weight             u32   optional   Weighting of this subscription (same as recording priority).
90khz              u32   optional   Request PTS/DTS in default 90kHz timebase, default is 1MHz (Added in version 7)
normts             u32   optional   Request PTS/DTS normalisation (Added in version 7)
                                    Note: this will mean missing timestamps are added, first packet should be ~0 and will always be an i-frame.
                                    Note: this is implied if timeshiftPeriod is enabled
queueDepth         u32   optional   Change the default packet queue lengths, default 500000 bytes (Added in version 7)
                                    Note: I-frame depth is 3*queueDepth, P-frame is 2*queueDepth and B-frame is queueDepth
timeshiftPeriod    u32   optional   The number of seconds to keep in the timeshift buffer (Added in version 9)
                                    Note: this may be bounded by server configuration settings

Reply message fields:

90khz              u32   optional   Indicates 90khz timestamps will be used
normts             u32   optional   Indicates timestamps will be normalised (and fixed)
timeshiftPeriod    u32   optional   The actual timeshiftPeriod to be used

unsubscribe¶

Stop a subscription.

Request message fields:

subscriptionId     u32   required   Subscription ID.

Reply message fields:

None

subscriptionChangeWeight (Added in version 5)¶

Change the weight of an existing subscription

Request message fields:

subscriptionId     u32   required   Subscription ID.
weight             u32   optional   The new subscription weight.

Reply message fields:

None

subscriptionSkip (Added in version 9)¶

Skip a timeshift enabled subscription. The response will be asynchronous subscriptionSkip().

Request message fields:

subscriptionId     u32   required   Subscription ID.
absolute           u32   optional   The skip request is absolute
time               u64   optional   Specify skip using time (units are as for PTS)
size               u64   optional   Specify skip using size (Not currently used)

Reply message fields:

None

subscriptionSeek (Added in version 9)¶

Synonym for subscriptionSkip

subscriptionSpeed (Added in version 9)¶

Set the playback speed for the subscription. The response will be asynchronous subscriptionSpeed().

Request message fields:

subscriptionId     u32   required   Subscription ID.
speed              u32   required   Specify speed (0=pause, 100=1x fwd, -100=1x backward)

Reply message fields:

None

subscriptionLive (Added in version 9)¶

Return a timeshifted session to live. Reply will be asynchronous subscriptionSkip().

Request message fields:

subscriptionId     u32   required   Subscription ID.

Reply message fields:

None

fileOpen (Added in version 8)¶

Open a file within the Tvheadend file system. This is now the preferred method (in place of HTTP) for accessing recordings.

Accessing recordings via HTSP will overcome the limitations of standard HTTP streaming which cannot handle both skipping and growing files (i.e. in-progress recordings) at the same time.

Request message fields:

file               str   required   File path to open

Valid file paths:

/dvrfile/ID        will open the file associated with DVR entry ID
/imagecache/ID     will open the file associated with imagecache entry ID

Reply message fields:

id                 u32   required   The file handle used for further file operations
size               u64   optional   The size of the file in bytes
mtime              u64   optional   The last time the file was modified

fileRead (Added in version 8)¶

Read data from a file.

Request message fields:

id                 u32   required   The file handle used for further file operations
size               u64   required   The amount of data to read
offset             u64   optional   Offset into the file to read (default is current position)

Reply message fields:

data               bin   required   The data read from the file (size may be less than requested)

fileClose (Added in version 8)¶

Close an opened file.

Request message fields:

id                 u32   required   The file handle to be closed

Reply message fields:

None

fileStat (Added in version 8)¶

Get status of a file.

Request message fields:

id                 u32   required   The file handle to be stat'd

Reply message fields:

size               u64   optional   The size of the file in bytes
mtime              u64   optional   The last time the file was modified

fileSeek (Added in version 8)¶

Seek to position in a file.

Request message fields:

id                 u32   required   The file handle to be stat'd
offset             u64   required   The position to seek in the file
whence             str   required   Where to seek relative to (default=SEEK_SET)

Note: valid values for whence

SEEK_SET           Seek relative to start of file
SEEK_CUR           Seek relative to current position in file
SEEK_END           Seek relative (backwards) to end of file

Reply message fields:

offset             u64   optional   The new absolute position within the file

Server to Client methods¶

---

channelAdd¶

A new channel has been created on the server.

Message fields:

channelId          u32   required   ID of channel.
channelNumber      u32   required   Channel number, 0 means unconfigured.
channelName        str   required   Name of channel.
channelIcon        str   optional   URL to an icon representative for the channel
                                    (For v8+ clients this could be a relative /imagecache/ID URL
                                     intended to be fed to fileOpen() or HTTP server)
eventId            u32   optional   ID of the current event on this channel.
nextEventId        u32   optional   ID of the next event on the channel.
tags               u32[] optional   Tags this channel is mapped to.
services           msg[] optional   List of available services (Added in version 5)

Service fields:

name               str   required   Service name
type               str   required   Service type
caid               u32   optional   Encryption CA ID
caname             str   optional   Encryption CA name

channelUpdate¶

Same as channelAdd, but all fields (except channelId) are optional.

channelDelete¶

A channel has been deleted on the server.

Message fields:

channelId          u32   required   ID of channel.

---

tagAdd¶

A new tag has been created on the server.

Message fields:

tagId              u32   required   ID of tag.
tagName            str   required   Name of tag.
tagIcon            str   optional   URL to an icon representative for the channel.
tagTitledIcon      u32   optional   Icon includes a title
members            u32[] optional   Channel IDs of those that belong to the tag

tagUpdate¶

Same as tagAdd, but all fields (except tagId) are optional.

tagDelete¶

A tag has been deleted from the server.

Message fields:

tagId              u32   required   ID of tag.

---

dvrEntryAdd (Added in version 4)¶

A new recording has been created on the server.

Message fields:

id                 u32   required   ID of dvrEntry.
channel            u32   required   Channel of dvrEntry.
start              s64   required   Time of when this entry was scheduled to start recording.
stop               s64   required   Time of when this entry was scheduled to stop recording.
eventId            u32   optional   Associated EPG Event ID.
title              str   optional   Title of recording
summary            str   optional   Short description of the recording (Added in version 6).
description        str   optional   Long description of the recording.
state              str   required   Recording state
error              str   optional   Plain english error description (e.g. "Aborted by user").

TODO: recording states

dvrEntryUpdate¶

Message fields:

Same as dvrEntryAdd, but all fields (except id) are optional.

dvrEntryDelete (Added in version 4)¶

A recording has been deleted from the server.

Message fields:

id                 u32   required   ID of recording to delete.

---

eventAdd (Added in version 6)¶

Message fields:

eventId            u32   required   Event ID
channelId          u32   required   The channel this event is related to.
start              u64   required   Start time of event, UNIX time.
stop               u64   required   Ending time of event, UNIX time.
title              str   optional   Title of event.
summary            str   optional   Short description of the event (Added in version 6).
description        str   optional   Long description of the event.
serieslinkId       u32   optional   Series Link ID (Added in version 6).
episodeId          u32   optional   Episode ID (Added in version 6).
seasonId           u32   optional   Season ID (Added in version 6).
brandId            u32   optional   Brand ID (Added in version 6).
contentType        u32   optional   DVB content code (Added in version 4, Modified in version 6*).
ageRating          u32   optional   Minimum age rating (Added in version 6).
starRating         u32   optional   Star rating (1-5) (Added in version 6).
firstAired         s64   optional   Original broadcast time, UNIX time (Added in version 6).
seasonNumber       u32   optional   Season number (Added in version 6).
seasonCount        u32   optional   Show season count (Added in version 6).
episodeNumber      u32   optional   Episode number (Added in version 6).
episodeCount       u32   optional   Season episode count (Added in version 6).
partNumber         u32   optional   Multi-part episode part number (Added in version 6).
partCount          u32   optional   Multi-part episode part count (Added in version 6).
episodeOnscreen    str   optional   Textual representation of episode number (Added in version 6).
image              str   optional   URL to a still capture from the episode (Added in version 6).
dvrId              u32   optional   ID of a recording (Added in version 5).
nextEventId        u32   optional   ID of next event on the same channel.

• *contentType previously had the major DVB category in the bottom 4 bits,
  however in v6 this has been changed to properly match DVB, so top 4 bits
  as major category and bottom 4 bits has sub-category. Clients requesting
  v5 or lower will get the old output.

eventUpdate (Added in version 6)¶

Message fields:

Same as eventAdd but all fields (except eventId) are optional.

eventDeleted (Added in version 6)¶

Message fields:

eventId            u32   required   Event ID

---

initialSyncCompleted (Added in version 2)¶

Sent after all the initial metadata has been sent when session has been set to async mode.

Message fields:

---

subscriptionStart¶

Asynchronous message output when a new subscription is successfully started.

Message fields:

subscriptionId     u32   required   Subscription ID.
streams            msg[] required   Array of messages with stream information
sourceinfo         msg   optional   Source information.

Stream message fields:

index              u32   required   Index for this stream
type               str   required   Type of stream
language           str   optional   Language for stream
Video components only:
width              u32   optional   Width of video in pixels
height             u32   optional   Height of video in pixels
aspect_num         u32   optional   Aspect ratio numerator (Added in version 5) *Can be incorrect and should not be relied upon*
aspect_den         u32   optional   Aspect ratio denonminator (Added in version 5) *Can be incorrect and should not be relied upon*
Audio components only:
channels           u32   optional   Number of audio channels (Added in version 5)
rate               u32   optional   Audio bitrate (Added in version 5)
Subtitle components only:
composition_id     u32   optional   ??? (Added in version 5)
ancillary_id       u32   optional   ??? (Added in version 5)

Sourceinfo message fields:

adapter            str   optional   Adapter name
mux                str   optional   Transponder name
network            str   optional   Network name
provider           str   optional   ???
service            str   optional   Service name

Video Stream types:

MPEG2VIDEO                    MPEG2 video
H264                          H264 video

Audio Stream types:

MPEG2AUDIO                    MPEG2 audio (MP2)
AC3                           AC3 audio
AAC                           ADTS framed AAC (one AAC packet per ADTS frame)
EAC3                          Enhanced AC3 audio

Subtitle Stream types:

TELETEXT                      ???
DVBSUB                        ???

subscriptionStop¶

A subscription has been stopped.

Message fields:

subscriptionId     u32   required   Subscription ID.
status             str   optional   Error message if subscription stopped unexpectedly.

subscriptionSkip (Added version 9)¶

A subscription has been skipped.

Message fields:

subscriptionId     u32   required   Subscription ID.
error              u32   optional   The last skip command caused an error.
absolute           u32   optional   Indicates the output is absolute (Note: should always be 1 at the moment)
time               u64   optional   The time the subscription has skipped to.
size               u64   optional   The position in the file we've skipped to (Note: not currently used).

subscriptionSpeed (Added version 9)¶

A subscription's playback speed has changed.

This can happen even without a speed request, should playback reach either end of the buffer.

Message fields:

subscriptionId     u32   required   Subscription ID.
speed              u32   required   The new speed of the subscription

subscriptionStatus¶

Subscription status update.

Message fields:

subscriptionId     u32   required   Subscription ID.
status             str   optional   English clear text of error status. Absence of this field means that the status is OK. 

queueStatus¶

The queueStatus message is sent every second during normal data delivery.

The transmit scheduler have different drop thresholds for different frame types.

If congestion occurs it will favor dropping B-frames before P-frames before I-frames.
All audio is recognized as I-frames.

Message fields:

subscriptionId     u32   required   Subscription ID.
packets            u32   required   Number of data packets in queue.
bytes              u32   required   Number of bytes in queue.
delay              u32   optional   Estimated delay of queue (in µs).
Bdrops             u32   required   Number of B-frames dropped
Pdrops             u32   required   Number of P-frames dropped
Idrops             u32   required   Number of I-frames dropped
delay              s64   required   Delta between first and last DTS (Added in version 9)

signalStatus¶

The signalStatus message is sent every second during normal data delivery.

The optional fields may not have been implemented or may not be supported by the active adapter.

Message fields:

subscriptionId     u32   required   Subscription ID.
feStatus           str   required   Frontend status.
feSNR              u32   optional   Signal to noise ratio.
feSignal           u32   optional   Signal status percentage.
feBER              u32   optional   Bit error rate.
feUNC              u32   optional   Uncorrected blocks.

timeshiftStatus¶

Provide status every second about the timeshift buffer.

Message fields:

subscriptionId     u32   required   Subscription ID.
full               u32   required   Indicates whether the buffer is full
shift              s64   required   Current position relative to live
start              s64   optional   PTS of the first frame in the buffer
end                s64   optional   PTS of the last frame in the buffer

muxpkt¶

Streaming data.

Message fields:

subscriptionId     u32   required   Subscription ID.
frametype          u32   required   Type of frame as ASCII value: 'I', 'P', 'B'
stream             u32   required   Stream index. Corresponds to the streams reported in the subscriptionStart message.
dts                s64   optional   Decode Time Stamp in µs.
pts                s64   optional   Presentation Time Stamp in µs.
duration           u32   required   Duration of frame in µs.
payload            bin   required   Actual frame data.