Htsp » History » Revision 28
Revision 27 (Andreas Smas, 2009-02-28 23:33) → Revision 28/163 (Andreas Smas, 2009-03-03 22:31)
= Home Tv Streaming Protocol (HTSP) =
== General ==
HTSP is a TCP based protocol primarily intended for streaming of live TV and related meta data such as channels, group of channels (called tags in HTSP) and electronic program guide (EPG) information.
The transmission and reception of a channel over HTSP is referred to a subscription. A single HTSP session can handle as many concurrent subscriptions as the bandwidth and CPU permits.
The HTSP server in tvheadend has a payload-aware scheduler for prioritizing more important packets (such as I-frames) before less important ones (such as B-frames). This makes HTSP is suitable for long-distance transmissions and/or paths with non-perfect delivery.
(It has been tested with a server in Stockholm and the client in Berlin).
The HTS Showtime client can be found [http://trac.lonelycoder.com/hts/browser/trunk/showtime/tv/htsp.c here].
----
== Communication ==
This communication is currently implemented by using htsmsg:s. All strings are encoded as UTF-8.
There are two distinct ways for communication within HTSP.
Apart from this there is a number of messages that needs to be exchanged during login, see the login section below.
=== RPC communication ===
There is a normal RPC way of doing things. I.e. the client sends a request and the server responds with a reply. All the RPC methods are listed below as the 'Client to Server' methods. Apart from all message fields listed within each message type the client can add an additional field:
RPC request extra fields:
{{{
seq int optional Sequence number. This field will be echoed back by the server in the reply.
username str optional Username, in combination with 'digest' this can be used to raise the privileges
for the session in combination with invocation of a method.
digest bin optional Used to raise privileges. }}}
}}}
The followings This field should be used by the client to match the reply with the request.
All replies are guaranteed to arrive in the same order as the requests.
Even so, probably the best way to implement the request-reply client is by taking advantage of the 'seq' field.
RPC reply extra fields:
{{{
seq int optional Sequence number. Same as in the request.
error str optional If present an error has occurred and the text describes the error.
noaccess int optional If present and set to '1' the user is prohibited from invoking the method due to
access restrictions.
The client should first try to authenticate and then try to re-invoke the method.
challenge bin optional If 'noaccess' is set the server also provides a challenge to be used in the, if
the user decides so, upcoming 'authentication' request.
}}}
=== Streaming communication ===
For streaming of live TV and various related messages the server will continuously push data to the client.
These messages are referred to as asynchronous messages and always have the 'method' field set and never have the 'seq' field set.
Also, the client can enable an additional asyncMetadata mode and by doing so it will be notified by the server when meta data changes. (EPG updates, creation of channels and tags, etc).
=== Login Authentication ===
Upon connect If the server sends client receives a welcome message looking like this:
{{{
method str required Contains 'welcome'
htspversion int required The server supports all versions of reply with the protocol up 'noaccess' field set it means that it was not allowed to and including this number. invoke that method.
servername str required Server software name.
serverversion str required Server software version.
challenge bin required 32 bit randomized data used to generate authentication digests
}}}
The client is can try to raise it's access by invoking the 'authenticate' method and then expected try to reply with: re-invoke the original method.
{{{
method str required Contains 'login'
htspversion int required HTSP protocol version the A typical client wish to use. would do something like this:
username str optional Login username. If omitted the login is considered anonymous.
digest bin required SHA-1 hash {{{
sendRequest() {
while(1) {
reply = send(request)
if(reply.noaccess == 0) {
return reply}
else {
do {
ask-user-for-credentials-and-or-get-them-from-keychain
authrequest.username = name of [password (not including terminating NUL)] user
authrequest.digest = sha1(password + [challenge from welcome message] reply.challenge)
reply = send(authrequest);
} while(reply.noaccess == 1)
}
}}}
The server challenge will in turn reply with: always be present when 'noaccess' is set.
{{{
It is also possible to have the client authenticate itself without it having to be a direct result of a denied method str required Contains 'loginAck' invocation.
error str optional If set an error occurred during login. The connection will This can be terminated done by just asking for a challenge (using the server.
}}}
After this procedure normal communication may take place. 'getChallenge' request) and then perform an 'authenticate' request.
----
= Client to Server (RPC) methods =
----
=== getChallenge ===
Request a 32 byte challenge used to generate a authentication digest.
Request message fields:
{{{
None
}}}
Reply message fields:
{{{
challenge bin required 32 bytes of random challenge to be used in next authentication request from client.
}}}
----
=== authenticate ===
Request message fields:
{{{
username str required Username.
digest bin required SHA-1 hash of [password (not including terminating NUL)] + [challenge]
}}}
Reply message fields:
{{{
noaccess int optional Set to '1' if access was denied.
challenge bin optional 32 bytes of random challenge to be used in next authentication request from client.
Always present if 'noaccess' is set. This is supplied so the client does not have
to issue an extra 'getChallenge' request.
}}}
----
=== getInfo ===
Request information about the server.
Request message fields:
{{{
}}}
Reply message fields:
{{{
appname str required Server software name.
appver str required Server software version.
protover int required HTSP version.
}}}
----
=== enableAsyncMetadata ===
When this is enabled the client will get continuous updates from the server about channels and tags.
This also includes creation and deletion of channels and tags.
An interactive application that presents the user with information about channels and tags would probably want to switch to this mode.
Request message fields:
{{{
None
}}}
Reply message fields:
{{{
None
}}}
----
=== getEvent ===
Request information about the given event. An event typically corresponds to a program on a channel.
Request message fields:
{{{
eventId int required Event ID.
}}}
Reply message fields:
{{{
start int required Start time of event (Seconds since the epoch, in UTC)
stop int required Ending time of event (Seconds since the epoch, in UTC)
title str required Title of event.
description str required Description of event. This can be quite huge and may also contain newlines.
nextEventId int optional ID of next event on the same channel.
}}}
----
=== subscribe ===
Request subscription to the given channel.
Request message fields:
{{{
channelId int required ID for channel.
subscriptionId int required Subscription ID. Selected by client. This value is not interpreted by the server in any form.
The value is used from now on in all messages related to the subscription.
}}}
Reply message fields:
{{{
None.
}}}
----
=== unsubscribe ===
Stop a subscription.
Attributes
{{{
subscriptionId int required Subscription ID.
}}}
Reply message fields:
{{{
None.
}}}
----
= Server to Client methods =
----
=== channelAdd ===
A new channel has been created on the server.
Message fields:
{{{
channelId int required ID of channel.
channelName str required Name of channel.
channelIcon str required URL to an icon representative for the channel.
eventId int optional ID of the current (or next to be) event on this channel.
tags int[] optional Tags this channel is mapped to.
}}}
----
=== channelUpdate ===
A channel has been updated on the server. All fields will be sent even if they are not changed. Most clients can process this and the 'channelAdd' message
with the very same code.
Message fields:
{{{
channelId int required ID of channel.
channelName str required Name of channel.
channelIcon str optioanl URL to an icon representative for the channel.
eventId int optional ID of the current (or next to be) event on this channel.
tags int[] required Tags this channel is mapped to.
}}}
----
=== channelDelete ===
A channel has been deleted on the server.
This message is only sent if session is in asynchronous mode.
Message fields:
{{{
channelId int required ID of channel.
}}}
----
=== tagAdd ===
A new tag has been created on the server.
Message fields:
{{{
tagId int required ID of tag.
tagName str required Name of tag.
tagIcon str optional URL to an icon representative for the channel.
channels int[] required Channels this tag is mapped to.
}}}
----
=== tagUpdate ===
A tag has been updated on the server.
Message fields:
{{{
tagId int required ID of tag.
tagName str required Name of tag.
tagIcon str optional URL to an icon representative for the channel.
channels int[] required Channels this tag is mapped to.
}}}
----
=== tagDelete ===
A tag has been deleted from the server.
Message fields:
{{{
tagId str required ID of tag.
}}}
----
=== subscriptionStart ===
Message fields:
{{{
subscriptionId int required Subscription ID.
streams msg[] required Array of messages with stream information
'streams' message:
index int required Index for this stream
type str required Type of stream
language str optional Language for stream
Stream types:
AC3 AC3 audio
MPEG2AUDIO MPEG2 audio (MP2)
MPEG2VIDEO MPEG2 video
H264 H264 video
}}}
----
=== subscriptionStop ===
Message fields:
{{{
subscriptionId int required Subscription ID.
reason str optional Reason for subscription stop.
}}}
----
=== subscriptionStatus ===
Message fields:
{{{
subscriptionId int required Subscription ID.
status str optional English clear text of status. Absence of this field means that the status is OK.
}}}
----
=== queueStatus ===
The queueStatus message is sent every second during normal data delivery.
The transmit scheduler have different drop thresholds for different frame types.
If congestion occurs it will favor dropping B-frames before P-frames before I-frames.
All audio is recognized as I-frames.
Message fields:
{{{
subscriptionId int required Subscription ID.
packets int required Number of data packets in queue.
bytes int required Number of bytes in queue.
delay int required Estimated delay of queue (in µs)
Bdrops int required Number of B-frames dropped
Pdrops int required Number of P-frames dropped
Idrops int required Number of I-frames dropped
}}}
----
=== muxpkt ===
Streaming data.
Message fields:
{{{
subscriptionId int required Subscription ID.
frametype int required Type of frame as ASCII value: 'I', 'P', 'B'
stream int required Stream index. Corresponds to the streams reported in the subscriptionStart message.
dts int required Decode Time Stamp in µs.
pts int required Presentation Time Stamp in µs.
duration int required Duration of frame in µs.
payload bin required Actual frame data.
}}}
== General ==
HTSP is a TCP based protocol primarily intended for streaming of live TV and related meta data such as channels, group of channels (called tags in HTSP) and electronic program guide (EPG) information.
The transmission and reception of a channel over HTSP is referred to a subscription. A single HTSP session can handle as many concurrent subscriptions as the bandwidth and CPU permits.
The HTSP server in tvheadend has a payload-aware scheduler for prioritizing more important packets (such as I-frames) before less important ones (such as B-frames). This makes HTSP is suitable for long-distance transmissions and/or paths with non-perfect delivery.
(It has been tested with a server in Stockholm and the client in Berlin).
The HTS Showtime client can be found [http://trac.lonelycoder.com/hts/browser/trunk/showtime/tv/htsp.c here].
----
== Communication ==
This communication is currently implemented by using htsmsg:s. All strings are encoded as UTF-8.
There are two distinct ways for communication within HTSP.
Apart from this there is a number of messages that needs to be exchanged during login, see the login section below.
=== RPC communication ===
There is a normal RPC way of doing things. I.e. the client sends a request and the server responds with a reply. All the RPC methods are listed below as the 'Client to Server' methods. Apart from all message fields listed within each message type the client can add an additional field:
RPC request extra fields:
{{{
seq int optional Sequence number. This field will be echoed back by the server in the reply.
username str optional Username, in combination with 'digest' this can be used to raise the privileges
for the session in combination with invocation of a method.
digest bin optional Used to raise privileges. }}}
}}}
The followings This field should be used by the client to match the reply with the request.
All replies are guaranteed to arrive in the same order as the requests.
Even so, probably the best way to implement the request-reply client is by taking advantage of the 'seq' field.
RPC reply extra fields:
{{{
seq int optional Sequence number. Same as in the request.
error str optional If present an error has occurred and the text describes the error.
noaccess int optional If present and set to '1' the user is prohibited from invoking the method due to
access restrictions.
The client should first try to authenticate and then try to re-invoke the method.
challenge bin optional If 'noaccess' is set the server also provides a challenge to be used in the, if
the user decides so, upcoming 'authentication' request.
}}}
=== Streaming communication ===
For streaming of live TV and various related messages the server will continuously push data to the client.
These messages are referred to as asynchronous messages and always have the 'method' field set and never have the 'seq' field set.
Also, the client can enable an additional asyncMetadata mode and by doing so it will be notified by the server when meta data changes. (EPG updates, creation of channels and tags, etc).
=== Login Authentication ===
Upon connect If the server sends client receives a welcome message looking like this:
{{{
method str required Contains 'welcome'
htspversion int required The server supports all versions of reply with the protocol up 'noaccess' field set it means that it was not allowed to and including this number. invoke that method.
servername str required Server software name.
serverversion str required Server software version.
challenge bin required 32 bit randomized data used to generate authentication digests
}}}
The client is can try to raise it's access by invoking the 'authenticate' method and then expected try to reply with: re-invoke the original method.
{{{
method str required Contains 'login'
htspversion int required HTSP protocol version the A typical client wish to use. would do something like this:
username str optional Login username. If omitted the login is considered anonymous.
digest bin required SHA-1 hash {{{
sendRequest() {
while(1) {
reply = send(request)
if(reply.noaccess == 0) {
return reply}
else {
do {
ask-user-for-credentials-and-or-get-them-from-keychain
authrequest.username = name of [password (not including terminating NUL)] user
authrequest.digest = sha1(password + [challenge from welcome message] reply.challenge)
reply = send(authrequest);
} while(reply.noaccess == 1)
}
}}}
The server challenge will in turn reply with: always be present when 'noaccess' is set.
{{{
It is also possible to have the client authenticate itself without it having to be a direct result of a denied method str required Contains 'loginAck' invocation.
error str optional If set an error occurred during login. The connection will This can be terminated done by just asking for a challenge (using the server.
}}}
After this procedure normal communication may take place. 'getChallenge' request) and then perform an 'authenticate' request.
----
= Client to Server (RPC) methods =
----
=== getChallenge ===
Request a 32 byte challenge used to generate a authentication digest.
Request message fields:
{{{
None
}}}
Reply message fields:
{{{
challenge bin required 32 bytes of random challenge to be used in next authentication request from client.
}}}
----
=== authenticate ===
Request message fields:
{{{
username str required Username.
digest bin required SHA-1 hash of [password (not including terminating NUL)] + [challenge]
}}}
Reply message fields:
{{{
noaccess int optional Set to '1' if access was denied.
challenge bin optional 32 bytes of random challenge to be used in next authentication request from client.
Always present if 'noaccess' is set. This is supplied so the client does not have
to issue an extra 'getChallenge' request.
}}}
----
=== getInfo ===
Request information about the server.
Request message fields:
{{{
}}}
Reply message fields:
{{{
appname str required Server software name.
appver str required Server software version.
protover int required HTSP version.
}}}
----
=== enableAsyncMetadata ===
When this is enabled the client will get continuous updates from the server about channels and tags.
This also includes creation and deletion of channels and tags.
An interactive application that presents the user with information about channels and tags would probably want to switch to this mode.
Request message fields:
{{{
None
}}}
Reply message fields:
{{{
None
}}}
----
=== getEvent ===
Request information about the given event. An event typically corresponds to a program on a channel.
Request message fields:
{{{
eventId int required Event ID.
}}}
Reply message fields:
{{{
start int required Start time of event (Seconds since the epoch, in UTC)
stop int required Ending time of event (Seconds since the epoch, in UTC)
title str required Title of event.
description str required Description of event. This can be quite huge and may also contain newlines.
nextEventId int optional ID of next event on the same channel.
}}}
----
=== subscribe ===
Request subscription to the given channel.
Request message fields:
{{{
channelId int required ID for channel.
subscriptionId int required Subscription ID. Selected by client. This value is not interpreted by the server in any form.
The value is used from now on in all messages related to the subscription.
}}}
Reply message fields:
{{{
None.
}}}
----
=== unsubscribe ===
Stop a subscription.
Attributes
{{{
subscriptionId int required Subscription ID.
}}}
Reply message fields:
{{{
None.
}}}
----
= Server to Client methods =
----
=== channelAdd ===
A new channel has been created on the server.
Message fields:
{{{
channelId int required ID of channel.
channelName str required Name of channel.
channelIcon str required URL to an icon representative for the channel.
eventId int optional ID of the current (or next to be) event on this channel.
tags int[] optional Tags this channel is mapped to.
}}}
----
=== channelUpdate ===
A channel has been updated on the server. All fields will be sent even if they are not changed. Most clients can process this and the 'channelAdd' message
with the very same code.
Message fields:
{{{
channelId int required ID of channel.
channelName str required Name of channel.
channelIcon str optioanl URL to an icon representative for the channel.
eventId int optional ID of the current (or next to be) event on this channel.
tags int[] required Tags this channel is mapped to.
}}}
----
=== channelDelete ===
A channel has been deleted on the server.
This message is only sent if session is in asynchronous mode.
Message fields:
{{{
channelId int required ID of channel.
}}}
----
=== tagAdd ===
A new tag has been created on the server.
Message fields:
{{{
tagId int required ID of tag.
tagName str required Name of tag.
tagIcon str optional URL to an icon representative for the channel.
channels int[] required Channels this tag is mapped to.
}}}
----
=== tagUpdate ===
A tag has been updated on the server.
Message fields:
{{{
tagId int required ID of tag.
tagName str required Name of tag.
tagIcon str optional URL to an icon representative for the channel.
channels int[] required Channels this tag is mapped to.
}}}
----
=== tagDelete ===
A tag has been deleted from the server.
Message fields:
{{{
tagId str required ID of tag.
}}}
----
=== subscriptionStart ===
Message fields:
{{{
subscriptionId int required Subscription ID.
streams msg[] required Array of messages with stream information
'streams' message:
index int required Index for this stream
type str required Type of stream
language str optional Language for stream
Stream types:
AC3 AC3 audio
MPEG2AUDIO MPEG2 audio (MP2)
MPEG2VIDEO MPEG2 video
H264 H264 video
}}}
----
=== subscriptionStop ===
Message fields:
{{{
subscriptionId int required Subscription ID.
reason str optional Reason for subscription stop.
}}}
----
=== subscriptionStatus ===
Message fields:
{{{
subscriptionId int required Subscription ID.
status str optional English clear text of status. Absence of this field means that the status is OK.
}}}
----
=== queueStatus ===
The queueStatus message is sent every second during normal data delivery.
The transmit scheduler have different drop thresholds for different frame types.
If congestion occurs it will favor dropping B-frames before P-frames before I-frames.
All audio is recognized as I-frames.
Message fields:
{{{
subscriptionId int required Subscription ID.
packets int required Number of data packets in queue.
bytes int required Number of bytes in queue.
delay int required Estimated delay of queue (in µs)
Bdrops int required Number of B-frames dropped
Pdrops int required Number of P-frames dropped
Idrops int required Number of I-frames dropped
}}}
----
=== muxpkt ===
Streaming data.
Message fields:
{{{
subscriptionId int required Subscription ID.
frametype int required Type of frame as ASCII value: 'I', 'P', 'B'
stream int required Stream index. Corresponds to the streams reported in the subscriptionStart message.
dts int required Decode Time Stamp in µs.
pts int required Presentation Time Stamp in µs.
duration int required Duration of frame in µs.
payload bin required Actual frame data.
}}}