Network Working Group Stephan Wenger INTERNET-DRAFT Umesh Chandra Expires: May 2007 Nokia Magnus Westerlund Bo Burman Ericsson March 5, 2007 Codec Control Messages in the RTP Audio-Visual Profile with Feedback (AVPF) draft-ietf-avt-avpf-ccm-04.txt> Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The IETF Trust (2007). Abstract This document specifies a few extensions to the messages defined in the Audio-Visual Profile with Feedback (AVPF). They are helpful primarily in conversational multimedia scenarios where centralized multipoint functionalities are in use. However some are also usable in smaller multicast environments and point-to-point calls. The Wenger, et al. [Page 1] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 extensions discussed are messages related to the ITU-T H.271 Video Back Channel, Full Intra Request, Temporary Maximum Media Stream Bit- rate and Temporal Spatial Trade-off. Wenger, et al. Standards Track [Page 2] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 TABLE OF CONTENTS 1. Introduction....................................................5 2. Definitions.....................................................7 2.1. Glossary...................................................7 2.2. Terminology................................................8 2.3. Topologies.................................................9 3. Motivation (Informative).......................................10 3.1. Use Cases.................................................10 3.2. Using the Media Path......................................12 3.3. Using AVPF................................................13 3.3.1. Reliability..........................................13 3.4. Multicast.................................................13 3.5. Feedback Messages.........................................13 3.5.1. Full Intra Request Command...........................13 3.5.1.1. Reliability.....................................14 3.5.2. Temporal Spatial Trade-off Request and Announcement..15 3.5.2.1. Point-to-point..................................16 3.5.2.2. Point-to-Multipoint using Multicast or Translators16 3.5.2.3. Point-to-Multipoint using RTP Mixer.............17 3.5.2.4. Reliability.....................................17 3.5.3. H.271 Video Back Channel Message conforming to ITU-T Rec. H.271.......................................................17 3.5.3.1. Reliability.....................................20 3.5.4. Temporary Maximum Media Bit-rate Request.............20 3.5.4.1. MCU based Multi-point operation.................25 3.5.4.2. Point-to-Multipoint using Multicast or Translators27 3.5.4.3. Point-to-point operation........................27 3.5.4.4. Reliability.....................................28 4. RTCP Receiver Report Extensions................................29 4.1. Design Principles of the Extension Mechanism..............29 4.2. Transport Layer Feedback Messages.........................30 4.2.1. Temporary Maximum Media Bit-rate Request (TMMBR).....30 4.2.1.1. Semantics.......................................31 4.2.1.2. Message Format..................................33 4.2.1.3. Timing Rules....................................34 4.2.2. Temporary Maximum Media Bit-rate Notification (TMMBN) 35 4.2.2.1. Semantics.......................................35 4.2.2.2. Message Format..................................36 4.2.2.3. Timing Rules....................................36 4.3. Payload Specific Feedback Messages........................37 4.3.1. Full Intra Request (FIR) command.....................37 4.3.1.1. Semantics.......................................37 4.3.1.2. Message Format..................................39 4.3.1.3. Timing Rules....................................40 4.3.1.4. Remarks.........................................40 4.3.2. Temporal-Spatial Trade-off Request (TSTR)............41 Wenger, et al. Standards Track [Page 3] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 4.3.2.1. Semantics.......................................41 4.3.2.2. Message Format..................................41 4.3.2.3. Timing Rules....................................42 4.3.2.4. Remarks.........................................42 4.3.3. Temporal-Spatial Trade-off Announcement (TSTA).......43 4.3.3.1. Semantics.......................................43 4.3.3.2. Message Format..................................44 4.3.3.3. Timing Rules....................................44 4.3.3.4. Remarks.........................................45 4.3.4. H.271 VideoBackChannelMessage (VBCM).................45 5. Congestion Control.............................................48 6. Security Considerations........................................48 7. SDP Definitions................................................49 7.1. Extension of rtcp-fb attribute............................49 7.2. Offer-Answer..............................................51 7.3. Examples..................................................51 8. IANA Considerations............................................54 9. Acknowledgements...............................................54 10. References....................................................56 10.1. Normative references.....................................56 10.2. Informative references...................................56 11. Authors' Addresses............................................57 12. List of Changes relative to previous draftsError! Bookmark not defined. Wenger, et al. Standards Track [Page 4] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 1. Introduction When the Audio-Visual Profile with Feedback (AVPF) [RFC4585] was developed, the main emphasis lay in the efficient support of point- to-point and small multipoint scenarios without centralized multipoint control. However, in practice, many small multipoint conferences operate utilizing devices known as Multipoint Control Units (MCUs). Long standing experience of the conversational video conferencing industry suggests that there is a need for a few additional feedback messages, to efficiently support centralized multipoint conferencing. Some of the messages have applications beyond centralized multipoint, and this is indicated in the description of the message. This is especially true for the message intended to carry ITU-T Rec. H.271 [H.271] bitstrings for Video Back Channel messages. In RTP [RFC3550] terminology, MCUs comprise mixers and translators. Most MCUs also include signaling support. During the development of this memo, it was noticed that there is considerable confusion in the community related to the use of terms such as mixer, translator, and MCU. In response to these concerns, a number of topologies have been identified that are of practical relevance to the industry, but not documented in sufficient detail in RTP. These topologies are documented in [Topologies], and understanding this memo requires previous or parallel study of [Topologies]. Some of the messages defined here are forward only, in that they do not require an explicit notification to the message emitter indicating their reception and/or the message receiver's actions. Other messages require notification, leading to a two way communication model that could suggest to some to be useful for control purposes. It is not the intention of this memo to open up RTCP to a generalized control protocol. All mentioned messages have relatively strict real-time constraints -- in the sense that their value diminishes with increased delay. This makes the use of more traditional control protocol means, such as SIP re-invites [RFC3261], undesirable. Furthermore, all messages are of a very simple format that can be easily processed by an RTP/RTCP sender/receiver. Finally, all messages infer only to the RTP stream they are related to, and not to any other property of a communication system. The Full Intra Request (FIR) requires the receiver of the message (and sender of the stream) to immediately insert a decoder refresh point. In video coding, one commonly used form of a decoder refresh point is an IDR or Intra picture, depending on the video compression Wenger, et al. Standards Track [Page 5] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 technology in use. Other codecs may have other forms of decoder refresh points. In order to fulfill congestion control constraints, sending a decoder refresh point may imply a significant drop in frame rate, as they are commonly much larger than regular predicted content. The use of this message is restricted to cases where no other means of decoder refresh can be employed, e.g. during the join- phase of a new participant in a multipoint conference. It is explicitly disallowed to use the FIR command for error resilience purposes, and instead it is referred to AVPF's [RFC4585] PLI message, which reports lost pictures and has been included in AVPF for precisely that purpose. The message does not require a reception notification, as the presence of a decoder refresh point can be easily derived from the media bit stream. Today, the FIR message appears to be useful primarily with video streams, but in the future it may also prove helpful in conjunction with other media codecs that support prediction across RTP packets. The Temporary Maximum Media Stream Bitrate Request (TMMBR) allows to signal, from media receiver to media sender, the current maximum media stream bit-rate for a given media stream. The maximum media stream bit-rate is defined as a tuple. The first value is the bit- rate available for the packet stream at the layer reported on. The second value is the measured header sizes between the start of the header for the layer reported on and the beginning of the RTP payload. Once, the media sender has received the TMMBR request on the bitrate limitation, it notifies the initiator of the request, and all other session participants, by sending a Temporal Maximum Media Stream Bitrate Notification (TMMBN). The TMMBN contains a list of the current applicable restrictions to help the participants to suppress TMMBR requests that wouldn't result in further restrictions for the sender. One usage scenario can be seen as limiting media senders in multiparty conferencing to the slowest receiver's Maximum Media Stream bitrate reception/handling capability. Such a use is helpful, for example, because the receiver's situation may have changed due to computational load, or because the receiver has just joined the conference, and considers it helpful to inform media sender(s) about its constraints, without waiting for congestion induced bitrate reduction. Another application involves graceful bitrate adaptation in scenarios where the upper limit connection bitrate to a receiver changes, but is known in the interval between these dynamic changes. The TMMBR/TMMBN messages are useful for all media types that are not inherently of constant bit rate. However, TMMBR is not a congestion control mechanism and can't replace the need to implement one. The Video Back Channel Message (VBCM) allows conveying bit streams conforming to ITU-T Rec. H.271 [H.271], from a video receiver to video sender. This ITU-T Recommendation defines codepoints for a Wenger, et al. Standards Track [Page 6] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 number of video-specific feedback messages. Examples include messages to signal: - the corruption of reference pictures or parts thereof, - the corruption of decoder state information, e.g. parameter sets, - the suggestion of using a reference picture other than the one typically used, e.g. to support the NEWPRED algorithm [NEWPRED]. The ITU-T has the authority to add codepoints to H.271 every time a need arises, e.g. with the introduction of new video codecs or new tools into existing video codecs. There exists some overlap between VBCM messages and native messages specified in this memo and in AVPF. Examples include the PLI message of [RFC4585] and the FIR message specified herein. As a general rule, the native messages should be preferred over the sending of VBCM messages when all senders and receivers implement this memo. However, if gateways are in the picture, it may be more advisable to utilize VBCM. Similarly, for feedback message types that exist in H.271 but do not exist in this memo or AVPF, there is no other choice but using VBCM. Video Back Channel Messages according to H.271 do not require a notification on a protocol level, because the appropriate reaction of the video encoder and sender can be derived from the forward video bit stream. Finally, the Temporal-Spatial Trade-off Request (TSTR) enables a video receiver to signal to the video sender its preference for spatial quality or high temporal resolution (frame rate). Typically, the receiver of the video stream generates this signal based on input from its user interface, in reaction to explicit requests of the user. However, some implicit use forms are also known. For example, the trade-offs commonly used for live video and document camera content are different. Obviously, this indication is relevant only with respect to video transmission. The message is acknowledged by a notification message indicating the newly chosen tradeoff, so to allow immediate user feedback. 2. Definitions 2.1. Glossary AMID - Additive Increase Multiplicative Decrease ASM - Asynchronous Multicast AVPF - The Extended RTP Profile for RTCP-based Feedback FEC - Forward Error Correction FIR - Full Intra Request Wenger, et al. Standards Track [Page 7] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 MCU - Multipoint Control Unit MPEG - Moving Picture Experts Group PtM - Point to Multipoint PtP - Point to Point TMMBN - Temporary Maximum Media Stream Bitrate Notification TMMBR - Temporary Maximum Media Stream Bitrate Request PLI - Picture Loss Indication TSTN - Temporal Spatial Trade-off Notification TSTR - Temporal Spatial Trade-off Request VBCM - Video Back Channel Message indication. 2.2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. Message: Codepoint defined by this specification, of one of the following types: Request: Message that requires Acknowledgement Command: Message that forces the receiver to an action Indication: Message that reports a situation Notification: See Indication. Note that, with the exception of ''Notification'', this terminology is in alignment with ITU-T Rec. H.245. Decoder Refresh Point: A bit string, packetised in one or more RTP packets, which completely resets the decoder to a known state. Typical examples of Decoder Refresh Points are H.261 Intra pictures and H.264 IDR pictures. However, there are also much more complex decoder refresh points, as discussed below. Examples for "hard" decoder refresh points are Intra pictures in H.261, H.263, MPEG 1, MPEG 2, and MPEG-4 part 2, and IDR pictures in H.264. "Gradual" decoder refresh points may also Wenger, et al. Standards Track [Page 8] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 be used; see for example [AVC]. While both "hard" and "gradual" decoder refresh points are acceptable in the scope of this specification, in most cases the user experience will benefit from using a "hard" decoder refresh point. A decoder refresh point also contains all header information above the picture layer (or equivalent, depending on the video compression standard) that is conveyed in-band. In H.264, for example, a decoder refresh point contains parameter set NAL units that generate parameter sets necessary for the decoding of the following slice/data partition NAL units (and that are not conveyed out of band). Decoding: The operation of reconstructing the media stream. Rendering: The operation of presenting (parts of) the reconstructed media stream to the user. Stream thinning: The operation of removing some of the packets from a media stream. Stream thinning, preferably, is media-aware, implying that media packets are removed in the order of their relevance to the reproductive quality. However even when employing media-aware stream thinning, most media streams quickly lose quality when subject to increasing levels of thinning. Media-unaware stream thinning leads to even worse quality degradation. In contrast to transcoding, stream thinning is typically seen as a computationally lightweight operation Media: Often used (sometimes in conjunction with terms like bitrate, stream, sender, ...) to identify the content of the forward RTP packet stream carrying the codec data to which the codec control message applies to. Media Stream: The stream of packets carrying the media (and in some case also repair information such as retransmission or Forward Error Correction (FEC) information). We further include within this specification the RTP packetization and the usage of additional protocol headers on these packets to carry them from sender to receiver. 2.3. Topologies Wenger, et al. Standards Track [Page 9] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 Please refer to [Topologies] for an in depth discussion. the topologies referred to throughout this memo are labeled (consistent with [Topologies] as follows: Topo-Point-to-Point . . . . . point-to-point communication Topo-Multicast . . . . . . . multicast communication as in RFC 3550 Topo-Translator . . . . . . . translator based as in RFC 3550 Topo-Mixer . . . . . . . . . mixer based as in RFC 3550 Topo-Video-switch-MCU . . . . video switching MCU, Topo-RTCP-terminating-MCU . . mixer but terminating RTCP 3. Motivation (Informative) This section discusses the motivation and usage of the different video and media control messages. The video control messages have been under discussion for a long time, and a requirement draft was drawn up [Basso]. This draft has expired; however we do quote relevant sections of it to provide motivation and requirements. 3.1. Use Cases There are a number of possible usages for the proposed feedback messages. Let's begin with looking through the use cases Basso et al. [Basso] proposed. Some of the use cases have been reformulated and commented: 1. An RTP video mixer composes multiple encoded video sources into a single encoded video stream. Each time a video source is added, the RTP mixer needs to request a decoder refresh point from the video source, so as to start an uncorrupted prediction chain on the spatial area of the mixed picture occupied by the data from the new video source. 2. An RTP video mixer that receives multiple encoded RTP video streams from conference participants, and dynamically selects one of the streams to be included in its output RTP stream. At the time of a bit stream change (determined through means such as voice activation or the user interface), the mixer requests a decoder refresh point from the remote source, in order to avoid using unrelated content as reference data for inter picture prediction. After requesting the decoder refresh point, the video mixer stops the delivery of the current RTP stream and monitors the RTP stream from the new source until it detects data belonging to the decoder refresh point. At that time, the RTP mixer starts Wenger, et al. Standards Track [Page 10] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 forwarding the newly selected stream to the receiver(s). 3. An application needs to signal to the remote encoder a request of change of the desired trade-off in temporal/spatial resolution. For example, one user may prefer a higher frame rate and a lower spatial quality, and another user may prefer the opposite. This choice is also highly content dependent. Many current video conferencing systems offer in the user interface a mechanism to make this selection, usually in the form of a slider. The mechanism is helpful in point-to-point, centralized multipoint and non-centralized multipoint uses. 4. Use case 4 of the Basso draft applies only to AVPF's PLI [RFC4585] and is not reproduced here. 5. Use case 5 of the Basso draft relates to a mechanism known as "freeze picture request". Sending freeze picture requests over a non-reliable forward RTCP channel has been identified as problematic. Therefore, no freeze picture request has been included in this memo, and the use case discussion is not reproduced here. 6. A video mixer dynamically selects one of the received video streams to be sent out to participants and tries to provide the highest bit rate possible to all participants, while minimizing stream transrating. One way of achieving this is to setup sessions with endpoints using the maximum bit rate accepted by that endpoint, and by the call admission method used by the mixer. By means of commands that allow reducing the Maximum Media Stream bitrate beyond what has been negotiated during session setup, the mixer can then reduce the maximum bit rate sent by endpoints to the lowest common denominator of all received streams. As the lowest common denominator changes due to endpoints joining, leaving, or network congestion, the mixer can adjust the limits to which endpoints can send their streams to match the new limit. The mixer then would request a new maximum bit rate, which is equal or less than the maximum bit-rate negotiated at session setup, for a specific media stream, and the remote endpoint can respond with the actual bit-rate that it can support. The picture Basso, et al draws up covers most applications we foresee. However we would like to extend the list with two additional use cases: 7. The used congestion control algorithms (AMID and TFRC [RFC3448]) probe for more available capacity as long as there is something to send. With congestion control using packet-loss as the indication for congestion, this probing does generally result in reduced Wenger, et al. Standards Track [Page 11] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 media quality (often to a point where the distortion is large enough to make the media unusable), due to packet loss and increased delay. In a number of deployment scenarios, especially cellular ones, the bottleneck link is often the last hop link. That cellular link also commonly has some type of QoS negotiation enabling the cellular device to learn the maximal bit-rate available over this last hop. Thus, indicating the maximum available bit-rate to the transmitting part can be beneficial to prevent it from even trying to exceed the known hard limit that exists. For cellular or other mobile devices the available known bit-rate can also quickly change due to handover to another transmission technology, QoS renegotiation due to congestion, etc. To enable minimal disruption of service quick convergence is necessary, and therefore media path signaling is desirable. 8. The use of reference picture selection (RPS) as an error resilience tool has been introduced in 1997 as NEWPRED [NEWPRED], and is now widely deployed. When RPS is in use, simplisticly put, the receiver can send a feedback message to the sender, indicating a reference picture that should be used for future prediction. ([NEWPRED] mentions other forms of feedback as well.) AVPF contains a mechanism for conveying such a message, but did not specify for which codec and according to which syntax the message conforms to. Recently, the ITU-T finalized Rec. H.271 which (among other message types) also includes a feedback message. It is expected that this feedback message will enjoy wide support and fairly quickly. Therefore, a mechanism to convey feedback messages according to H.271 appears to be desirable. 3.2. Using the Media Path There are multiple reasons why we use the media path for the codec control messages. First, systems employing MCUs are often separating the control and media processing parts. As these messages are intended or generated by the media part rather than the signaling part of the MCU, having them on the media path avoids interfaces and unnecessary control traffic between signaling and processing. If the MCU is physically decomposite, the use of the media path avoids the need for media control protocol extensions (e.g. in MEGACO [RFC3525]). Secondly, the signaling path quite commonly contains several signaling entities, e.g. SIP-proxies and application servers. Avoiding going through signaling entities avoids delay for several reasons. Proxies have less stringent delay requirements than media processing and due to their complex and more generic nature may result in significant processing delay. The topological locations of Wenger, et al. Standards Track [Page 12] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 the signaling entities are also commonly not optimized for minimal delay, but rather towards other architectural goals. Thus the signaling path can be significantly longer in both geographical and delay sense. 3.3. Using AVPF The AVPF feedback message framework [RFC4585] provides a simple way of implementing the new messages. Furthermore, AVPF implements rules controlling the timing of feedback messages so to avoid congestion through network flooding by RTCP traffic. We re-use these rules by referencing AVPF. The signaling setup for AVPF allows each individual type of function to be configured or negotiated on a RTP session basis. 3.3.1. Reliability The use of RTCP messages implies that each message transfer is unreliable, unless the lower layer transport provides reliability. The different messages proposed in this specification have different requirements in terms of reliability. However, in all cases, the reaction to an (occasional) loss of a feedback message is specified. 3.4. Multicast The codec control messages might be used with multicast. The RTCP timing rules specified in [RFC3550] and [RFC4585] ensure that the messages do not cause overload of the RTCP connection. The use of multicast may result in the reception of messages with inconsistent semantics. The reaction to inconsistencies depends on the message type, and is discussed for each message type separately. 3.5. Feedback Messages This section describes the semantics of the different feedback messages and how they apply to the different use cases. 3.5.1. Full Intra Request Command A Full Intra Request (FIR) Command, when received by the designated media sender, requires that the media sender sends a Decoder Refresh Point (see 2 .2) at the earliest opportunity. The evaluation of such Wenger, et al. Standards Track [Page 13] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 opportunity includes the current encoder coding strategy and the current available network resources. FIR is also known as an ''instantaneous decoder refresh request'' or ''video fast update request''. Using a decoder refresh point implies refraining from using any picture sent prior to that point as a reference for the encoding process of any subsequent picture sent in the stream. For predictive media types that are not video, the analogue applies. For example, if in MPEG-4 systems scene updates are used, the decoder refresh point consists of the full representation of the scene and is not delta-coded relative to previous updates. Decoder Refresh Points, especially Intra or IDR pictures, are in general several times larger in size than predicted pictures. Thus, in scenarios in which the available bit-rate is small, the use of a Decoder Refresh Point implies a delay that is significantly longer than the typical picture duration. Usage in multicast is possible; however aggregation of the commands is recommended. A receiver that receives a request closely (within 2 times the longest Round Trip Time (RTT) known) after sending a Decoder Refresh Point should await a second request message to ensure that the media receiver has not been served by the previously delivered Decoder Refresh Point. The reason for delaying 2 times the longest known RTT is to avoid sending unnecessary Decoder Refresh Points. A session participant may have sent its own request while another participant's request was in-flight to them. Suppressing those requests that may have been sent without knowledge about the other request avoids this issue. Full Intra Request is applicable in use-case 1, 2, and 5. 3.5.1.1. Reliability The FIR message results in the delivery of a Decoder Refresh Point, unless the message is lost. Decoder Refresh Points are easily identifiable from the bit stream. Therefore, there is no need for protocol-level notification, and a simple command repetition mechanism is sufficient for ensuring the level of reliability required. However, the potential use of repetition does require a mechanism to prevent the recipient from responding to messages already received and responded to. To ensure the best possible reliability, a sender of FIR may repeat the FIR request until a response has been received. The repetition interval is determined by the RTCP timing rules applicable to the Wenger, et al. Standards Track [Page 14] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 session. Upon reception of a complete Decoder Refresh Point or the detection of an attempt to send a Decoder Refresh Point (which got damaged due to a packet loss), the repetition of the FIR must stop. If another FIR is necessary, the request sequence number must be increased. To combat loss of the Decoder Refresh Points sent, the sender that receives repetitions of the FIR 2*RTT after the transmission of the Decoder Refresh Point shall send a new Decoder Refresh Point. Two round trip times allow time for the request to arrive at the media sender and the Decoder Refresh Point to arrive back to the requestor. A FIR sender shall not have more than one FIR request (different request sequence number) outstanding at any time per media sender in the session. An RTP Mixer that receives an FIR from a media receiver is responsible to ensure that a Decoder Refresh Point is delivered to the requesting receiver. It may be necessary for the mixer to generate FIR commands. The two legs (FIR-requesting endpoint to mixer, and mixer to Decoder Refresh Point generating endpoint) are handled independently from each other from a reliability perspective. 3.5.2. Temporal Spatial Trade-off Request and Notification The Temporal Spatial Trade-off Request (TSTR) instructs the video encoder to change its trade-off between temporal and spatial resolution. Index values from 0 to 31 indicate monotonically a desire for higher frame rate. That is, a requester asking for an index of 0 prefers a high quality and is willing to accept a low frame rate, whereas a requester asking for 31 wishes a high frame rate, potentially at the cost of low spatial quality. In general the encoder reaction time may be significantly longer than the typical picture duration. See use case 3 for an example. The encoder decides if the request results in a change of the trade off. The Temporal Spatial Trade-Off Notification message (TSTN) has been defined to provide feedback of the trade-off that is used henceforth. Informative note: TSTR and TSTN have been introduced primarily because it is believed that control protocol mechanisms, e.g. a SIP re-invite, are too heavyweight, and too slow to allow for a reasonable user experience. Consider, for example, a user interface where the remote user selects the temporal/spatial trade- off with a slider (as it is common in state-of-the-art video conferencing systems). An immediate feedback to any slider movement is required for a reasonable user experience. A SIP re- invite [RFC3261] would require at least 2 round-trips more (compared to the TSTR/TSTN mechanism) and may involve proxies and Wenger, et al. Standards Track [Page 15] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 other complex mechanisms. Even in a well-designed system, it may take a second or so until finally the new trade-off is selected. Furthermore the use of RTCP solves very efficiently the multicast use case. The use of TSTR and TSTN in multipoint scenarios is a non-trivial subject, and can be solved in many implementation-specific ways. Problems are stemming from the fact that TSTRs will typically arrive unsynchronized, and may request different trade-off values for the same stream and/or endpoint encoder. This memo does not specify a translator, mixer or endpoint's reaction to the reception of a suggested trade-off as conveyed in the TSTR -- we only require the receiver of a TSTR message to reply to it by sending a TSTN, carrying the new trade-off chosen by its own criteria (which may or may not be based on the trade-off conveyed by TSTR). In other words, the trade- off sent in TSTR is a non-binding recommendation; nothing more. With respect to TSTR/TSTN, four scenarios based on the topologies described in [Topologies] need to be distinguished. The scenarios are described in the following sub-clauses. 3.5.2.1. Point-to-point In this most trivial case (Topo-Point-to-Point), the media sender typically adjusts its temporal/spatial trade-off based on the requested value in TSTR, and within its capabilities. The TSTN message conveys back the new trade-off value (which may be identical to the old one if, for example, the sender is not capable of adjusting its trade-off). 3.5.2.2. Point-to-Multipoint using Multicast or Translators RTCP Multicast is used either with media multicast according to Topo- Multicast, or following RFC 3550's translator model according to Topo-Translator. In these cases, TSTR messages from different receivers may be received unsynchronized, and possibly with different requested trade-offs (because of different user preferences). This memo does not specify how the media sender tunes its trade-off. Possible strategies include selecting the mean, or median, of all trade-off requests received, prioritize certain participants, or continue using the previously selected trade-off (e.g. when the sender is not capable of adjusting it). Again, all TSTR messages need to be acknowledged by TSTN, and the value conveyed back has to reflect the decision made. Wenger, et al. Standards Track [Page 16] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 3.5.2.3. Point-to-Multipoint using RTP Mixer In this scenario (Topo-Mixer) the RTP Mixer receives all TSTR messages, and has the opportunity to act on them based on its own criteria. In most cases, the Mixer should form a ''consensus'' of potentially conflicting TSTR messages arriving from different participants, and initiate its own TSTR message(s) to the media sender(s). The strategy of forming this ''consensus'' is open for the implementation, and can, for example, encompass averaging the participants request values, prioritizing certain participants, or use session default values. If the Mixer changes its trade-off, it needs to request from the media sender(s) the use of the new value, by creating a TSTR of its own. Upon reaching a decision on the used trade-off it includes that value in the acknowledgement. Even if a Mixer or Translator performs transcoding, it is very difficult to deliver media with the requested trade-off, unless the content the Mixer or Translator receives is already close to that trade-off. Only in cases where the original source has substantially higher quality (and bit-rate), it is likely that transcoding can result in the requested trade-off. 3.5.2.4. Reliability A request and reception acknowledgement mechanism is specified. The Temporal Spatial Trade-off Notification (TSTN) message informs the request-sender that its request has been received, and what trade-off is used henceforth. This acknowledgment mechanism is desirable for at least the following reasons: o A change in the trade-off cannot be directly identified from the media bit stream, o User feedback cannot be implemented without information of the chosen trade-off value, according to the media sender's constraints, o Repetitive sending of messages requesting an unimplementable trade- off can be avoided. 3.5.3. H.271 Video Back Channel Message ITU-T Rec. H.271 defines syntax, semantics, and suggested encoder reaction to a video back channel message. The codepoint defined in this memo is used to transparently convey such a message from media receiver to media sender. In this memo, we refrain from an in-depth discussion of the available codepoints within H.271 and refer to the specification text instead [H.271]. Wenger, et al. Standards Track [Page 17] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 However, we note that some H.271 messages bear similarities with native messages of AVPF and this memo. Furthermore, we note that some H.271 message are known to require caution in multicast environments -- or are plainly not usable in multicast or multipoint scenarios. Table 1 provides a brief, oversimplifying overview of the messages currently defined in H.271, their similar AVPF or CCM messages (the latter as specified in this memo), and an indication of our current knowledge of their multicast safety. H.271 msg type AVPF/CCM msg type multicast-safe --------------------------------------------------------------------- 0 (when used for reference picture selection) AVPF RPSI No (positive ACK of pictures) 1 AVPF PLI Yes 2 AVPF SLI Yes 3 N/A Yes (no required sender action) 4 N/A Yes (no required sender action) Table 1: H.271 messages and their AVPF/CCM equivalents Note: H.271 message type 0 is not a strict equivalent to AVPF's RPSI; it is an indication of known-as-correct reference picture(s) at the decoder. It does not command an encoder to use a defined reference picture (the form of control information envisioned to be carried in RPSI). However, it is believed and intended that H.271 message type 0 will be used for the same purpose as AVPF's RPSI -- although other use forms are also possible. In response to the opaqueness of the H.271 messages especially with respect to the multicast safety, the following guidelines MUST be followed when an implementation wishes to employ the H.271 video back channel message: 1. Implementations utilizing the H.271 feedback message MUST stay in compliance with congestion control principles, as outlined in section 5 .. 2. An implementation SHOULD utilize the native messages as defined in [RFC4585] and in this memo instead of similar messages defined in [H.271]. Our current understanding of similar messages is documented in Table 1 above. One good reason to divert from the SHOULD statement above would be if it is clearly understood that, for a given application and video compression standard, the aforementioned ''similarity'' is not given, in contrast to what the table indicates. Wenger, et al. Standards Track [Page 18] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 3. It has been observed that some of the H.271 codepoints currently in existence are not multicast-safe. Therefore, the sensible thing to do is not to use the H.271 feedback message type in multicast environments. It MAY be used only when all the issues mentioned later are fully understood by the implementer, and properly taken into account by all endpoints. In all other cases, the H.271 message type MUST NOT be used in conjunction with multicast. 4. It has been observed that even in centralized multipoint environments, where the mixer should theoretically be able to resolve issues as documented below, the implementation of such a mixer and cooperative endpoints is a very difficult and tedious task. Therefore, H.271 message MUST NOT be used in centralized multipoint scenarios, unless all the issues mentioned below are fully understood by the implementer, and properly taken into account by both mixer and endpoints. Issues to be taken into account when considering the use of H.271 in multipoint environments: 1. Different state on different receivers. In many environments it cannot be guarantied that the decoder state of all media receivers is identical at any given point in time. The most obvious reason for such a possible misalignment of state is a loss that occurs on the link to only one of many media receivers. However, there are other not so obvious reasons, such as recent joins to the multipoint conference (be it by joining the multicast group or through additional mixer output). Different states can lead the media receivers to issue potentially contradicting H.271 messages (or one media receiver issuing an H.271 message that, when observed by the media sender, is not helpful for the other media receivers). A naive reaction of the media sender to these contradicting messages can lead to unpredictable and annoying results. 2. Combining messages from different media receivers in a media sender is a non-trivial task. As reasons, we note that these messages may be contradicting each other, and that their transport is unreliable (there may well be other reasons). In case of many H.271 messages (i.e. types 0, 2, 3, and 4), the algorithm for combining must be both aware of the network/protocol environment (i.e. with respect to congestion) and of the media codec employed, as H.271 messages of a given type can have different semantics for different media codecs. 3. The suppression of requests may need to go beyond the basic mechanism described in AVPF (which are driven exclusively by timing and transport considerations on the protocol level). For example, a receiver is often required to refrain from (or delay) generating requests, based on information it receives from the Wenger, et al. Standards Track [Page 19] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 media stream. For instance, it makes no sense for a receiver to issue a FIR when a transmission of an Intra/IDR picture is ongoing. 4. When using the non-multicast-safe messages (e.g. H.271 type 0 positive ACK of received pictures/slices) in larger multicast groups, the media receiver will likely be forced to delay or even omit sending these messages. For the media sender this looks like data has not been properly received (although it was received properly), and a naively implemented media sender reacts to these perceived problems where it shouldn't. 3.5.3.1. Reliability H.271 Video Back Channel messages do not require reliable transmission, and the reception of a message can be derived from the forward video bit stream. Therefore, no specific reception acknowledgement is specified. With respect to re-sending rules, clause 3.5.1.1. applies. 3.5.4. Temporary Maximum Media Stream Bit-rate Request and Notification A receiver, translator or mixer uses the Temporary Maximum Media Stream Bit-rate Request (TMMBR, "timber") to request a sender to limit the maximum bit-rate for a media stream to, or below, the provided value. The Temporary Maximum Media Stream Bit-rate Notification (TMMBN) advises the media receiver(s) of the changed bitrate it is not going to exceed henceforth. The primary usage for this is a scenario with a MCU or Mixer (use case 6), corresponding to Topo-Translator or Topo-Mixer, but also Topo-Point-to-Point. The temporary limitation on the media stream is expressed as a tuple; one value limiting the bit-rate at the layer for which the overhead is calculated to. A second value provides the per packet header overhead between the layer for which bit-rate is reported and the start of the RTP payload. By having both values the media stream sender can determine the effect of changing the packet rate for the media stream in an environment which contains translators or mixers that affect the amount of per packet overhead. For example a gateway that convert between IPv4 and IPv6 would affect the per packet overhead commonly with 20 bytes. There exist also other mechanisms, like tunnels, that change the amount of headers that are present at a particular bottleneck for which the TMMBR sending entity has knowledge about. The problem with varying overhead is also discussed in [RFC3890]. Wenger, et al. Standards Track [Page 20] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 The above way of measuring allows for one to provide bit-rate and overhead values for different protocol layers, for example on IP level, out part of a tunnel protocol, or the link layer. The level a peer report on, is fully dependent on the level of integration the peer has, as it needs to be able to extract the information from that level. It is expected that peers will be able to report values at least for the IP layer, but in certain implementations link layer may be available to allow for more precise information. The temporary maximum media stream bit-rate messages are generic messages that can be applied to any RTP packet stream. This separates it a bit from the other codec control messages defined in this specification that applies only to specific media types or payload formats. The TMMBR functionality applies to the transport and the requirements it places on the media encoding. The reasoning below assumes that the participants have negotiated a session maximum bit-rate, using a signaling protocol. This value can be global, for example in case of point-to-point, multicast, or translators. It may also be local between the participant and the peer or mixer. In both cases, the bit-rate negotiated in signaling is the one that the participant guarantees to be able to handle (encode and decode). In practice, the connectivity of the participant also bears an influence to the negotiated value -- it does not necessarily make much sense to negotiate a media bit rate that one's network interface does not support. It is also beneficial to have negotiated a maximum packet rate for the session or sender. RFC 3890 provides such a SDP [RFC4566] attribute, however that is not usable in RTP sessions established using offer/answer [RFC3264]. Therefore a max packet rate signaling parameter is specified. An already established temporary limit may be changed at any time (subject to the timing rules of the feedback message sending), and to any values between zero and the session maximum, as negotiated during session establishment signaling. Even if a sender has received a TMMBR message allowing an increase in the bit-rate, all increases must be governed by a congestion control mechanism. TMMBR only indicates known limitations, usually in the local environment, and does not provide any guarantees about the full path. If it is likely that the new value indicated by TMMBR will be valid for the remainder of the session, the TMMBR sender can perform a renegotiation of the session upper limit using the session signaling protocol. Wenger, et al. Standards Track [Page 21] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 3.5.4.1. Behavior for media receivers using TMMBR In multipart scenarios, different receivers likely have different limits for receiving bitrate. Therefore, an algorithm to identify the most restrictive TMMBR requests is specified in section 4 ..2.2.1. The general behavior is explaind in this section and the gist of the algorithm to determine the most restrictive values are explained informally in the next section. Immediately after session setup, the bitrate limit is set to the session limit as established by the session setup signaling (or equivalent). The overhead value is set to 0. When the session setup signaling does not specify a limit, then unlimited bitrate is assumed. Note that many codecs specify their own limits, e.g. through H.264's level concept. At any given time, a media receiver can send a TMMBR with a limit that is lower than the current limit. The media receiver use the algorithm outlined in the below Section 3.5.4.2 to determine if its limit is stricter than already existing ones. The media sender upon receiving the TMMBR request will also excersie the algorithm to determine the set of most restrictive limitations and then send a TMMBN containg that set. Once the media sender has sent the TMMBN message, the receivers indicated in that message becomes ''owners'' of the limitations. Most likely, the owner is the original sender of the TMMBR -- for the handling of corner-cases (i.e. concurrent TMMBRs from different receivers, lost TMMBRs and sender side optimisations) please see the formal specification. ''Owners'' and limits are usually known session wide, as both TMMBR and TMMBN are forwarded to all in the session unless a Mixer or Translator separate the session from RTCP handling point of view. Only a ''owner'' is allowed to raise the bitrate limit to a value higher than the session has been notified of, but not higher than the session limit negotiated by the session setup signaling (see above). A ''owner'' does not need to take into account TMMBR messages sent by anyone else (although that may well be a desirable optimization). If a ''owner'' sets a new session limit that is too high for someone else's liking, other media receivers can react to the situation by emmitting their own TMMBR message (and, in the process, become a ''owner''). Limitations belonging to ''owners'' timing out from the session are removed by the media sender who notifies the session about the event by sending a TMMBN. Obviously, when there is only one media receiver, this receiver becomes ''owner'' once it receives the first TMMBN in response to its own TMMBR, and stays ''owner'' for the rest of the session. Therefore, when it is known that there will always be only a single Wenger, et al. Standards Track [Page 22] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 media receiver, the above algorithm is not required. Media receivers that are aware they are the only ones in a session can send TMMBR messages with bitrate limits both higher and lower than the previously notified limit at any time (subject to AVPF's RTCP RR send timing rules). However, it may be difficult for a session participant to determine if it is the only receiver in the session. Due to that any one implementing TMMBR are required to implement this algorithm. 3.5.4.2. Algorithm for exstablishing current limitations First it is important to consider the implications of using a tuple for limiting the media sender's behavior. The bit-rate and the overhead value results in a 2-dimensional solution space for possible media streams. Fortunately the two variables are linked. The bit-rate available for RTP payloads will be equal to the TMMBR reported bit- rate minus the packet rate used times the TMMBR reported overhead. This has the result in a session with two different participants having set limitations, the used packet rate will determine which of the two that applies. Example: Receiver A: TMMBR_BR = 35 kbps, TMMBR_OH = 40 Receiver B: TMMBR_BR = 40 kbps, TMMBR_OH = 60 For a given packet rate (PR) the bit-rate available for media payloads in RTP will be: Max_media_BR_A = TMMBR_BR_A - PR * TMMBR_OH_A * 8 Max_media_BR_B = TMMBR_BR_B - PR * TMMBR_OH_B * 8 For a PR = 20 these calculations will yield a Max_media_BR_A = 28600 bps and Max_media_BR_B = 30400 bps, which shows that receiver A is the limiting one for this packet rate. However there will be a PR when the difference in bit-rate restriction will be equal to the difference in packet overheads. This can be found by setting Max_media_BR_A equal to Max_media_BR_B and breaking out PR: TMMBR_BR_A - TMMBR_BR_B PR = --------------------------- 8*(TMMBR_OH_A - TMMBR_OH_B) Which, for the numbers above yields 31.25 as the intersection point between the two limits. The implications of this have to be considered by application implementors that are going to control Wenger, et al. Standards Track [Page 23] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 media encoding and its packetization. Because, as exemplified above, there might be multiple TMMBR limits that applies to the trade-off between media bit-rate and packet rate. Which limitation that applies depends on the packet rate considered to be used. This also has implications for how the TMMBR mechanism needs to work. First, there is the possibility that multiple TMMBR tuples are providing limitations on the media sender. Secondly there is a need for any session participant (meda sender and receivers) to be able to determine if a given tuple will become a limitation upon the media sender, or if the set of already given limitations are stricter than the given values. Otherwise the suppression of TMMBR requests would not work. Thus any session participant needs to be able from a given set X of tuples determine which is the minimal set need to express the limitations for all packet rates from 0 to highest possible. Where the highest possible either is application limited and indicated trough session setup signaling or as a result of the given limitations when the available bit-rate is fully consumed by headers. First determine what the highest possible bit-rate given all the limitations is. If there is provided a session maximum packet rate (SMAXPR) then this can be used. In addition one needs to calculate for each tuple in the set what its maximum is by calculating bit-rate (BR) divided by overhead (OH) per packet converted to bits. MaxPR = SMAXPR For i=1 to size(X) { tmp_pr = X(i).BR / 8*X(i).OH; If (tmp_pr < MaxPR) then MaxPR = tmp_pr } For a zero packet rate the TMMBR signaled bit-rate will be the only limiting factor, thus the tuple with the smallest available bit-rate is a limitation at this point of the range and function as a start value in the algorithm. Start by finding the element X(l) in X with the lowest bit-rate value and the highest overhead if there are multiple on the same bit-rate. The set Y that is the minimal set of tuples that provide restrictions initially contain only X(l). Then for each other tuple X(i) calculate if there exist an intersection between the currently selected tuple X(s) (initially s=l) and which of the tuples within the set that has this intersection at the lowest packet rate. Having found the lowest packet rate, compare it with the sessions maximum packet rate. If lower than that limit this tuple provide a session limit and the tuple is added to Y. Update the value of s to the found tuple and Wenger, et al. Standards Track [Page 24] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 repeat search for the tuple that has the intersection at the lowest packet rate, but still higher than the previous intersection. Algorithm has finished when it can't find any new tuple with an intersection at a packet rate lower than the session maximum. // Find the element with the lowest bit-rate in X l=0; for (i=1:size(X)){ if (X(i).BR <= X(l).BR) & (X(i).OH > X(l).OH) then l=I; } tuple_index = l; // The lowest bit-rate tuple Y = X(l); // Initilize Y to X(l) start_pr = 0; // Start from zero bit-rate do { current_low = MaxPr; //Reset packet-rate current_index = tuple_index; // To allow for no intersection For i=each element in X pr = (X(i).BR - X(tuple_index).BR) / (X(i).OH - X(tuple_index).OH) // Calculate packet rate compared to element i If (pr < current_low && pr > start_pr) then { // Update lowest intersection packet rate current_low = pr; current_index = i; } } If (current_index != tuple_index) { // A tuple intersecting below maxpacket rate Y(size(Y)+1) = X(current_index) // Add to Y tuple_index = current_index; // Update which to compare with start_pr = current_low; // Update packet rate to seek from. } } while (current_low < MaxPr) The above algorithm yields the set of applicable restriction Y. 3.5.4.3. Use of TMMBR in a Mixer based Multi-point operation Assume a small mixer-based multiparty conference is ongoing, as depicted in Topo-Mixer of [Topologies]. All participants have negotiated a common maximum bit-rate that this session can use. The conference operates over a number of unicast paths between the participants and the mixer. The congestion situation on each of these paths can be monitored by the participant in question and by the mixer, utilizing, for example, RTCP Receiver Reports or the Wenger, et al. Standards Track [Page 25] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 transport protocol, e.g. DCCP [RFC4340]. However, any given participant has no knowledge of the congestion situation of the connections to the other participants. Worse, without mechanisms similar to the ones discussed in this draft, the mixer (who is aware of the congestion situation on all connections it manages) has no standardized means to inform media senders to slow down, short of forging its own receiver reports (which is undesirable). In principle, a mixer confronted with such a situation is obliged to thin or transcode streams intended for connections that detected congestion. In practice, media-aware stream thinning is unfortunately a very difficult and cumbersome operation and adds undesirable delay. If media-unaware, it leads very quickly to unacceptable reproduced media quality. Hence, means to slow down senders even in the absence of congestion on their connections to the mixer are desirable. To allow the mixer to perform congestion control on the individual links, without performing transcoding, there is a need for a mechanism that enables the mixer to request the participant's media encoders to limit their Maximum Media Stream bit-rate currently used. The mixer handles the detection of a congestion state between itself and a participant as follows: 1. Start thinning the media traffic to the supported bit-rate. 2. Use the TMMBR to request the media sender(s) to reduce the media bit-rate sent by them to the mixer, to a value that is in compliance with congestion control principles for the slowest link. Slow refers here to the available bandwidth/bitrate/capacity and packet rate after congestion control. 3. As soon as the bit-rate has been reduced by the sending part, the mixer stops stream thinning implicitly, because there is no need for it any more as the stream is in compliance with congestion control. Above algorithms may suggest to some that there is no need for the TMMBR - it should be sufficient to solely rely on stream thinning. As much as this is desirable from a network protocol designer's viewpoint, it has the disadvantage that it doesn't work very well - the reproduced media quality quickly becomes unusable. It appears to be a reasonable compromise to rely on stream thinning as an immediate reaction tool to combat congestions, and have a quick control mechanism that instructs the original sender to reduce its bitrate. Note also that the standard RTCP receiver report cannot serve for the purpose mentioned. In an environment with RTP mixers, the RTCP RR is Wenger, et al. Standards Track [Page 26] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 being sent between the RTP receiver in the endpoint and the RTP sender in the mixer only - as there is no multicast transmission. The stream that needs to be bitrate-reduced, however, is the one between the original sending endpoint and the mixer. This endpoint doesn't see the aforementioned RTCP RRs, and hence needs to be explicitly informed about desired bitrate adjustments. In this topology it is the mixer's responsibility to collect, and consider jointly, the different bit-rates which the different links may support, into the bit rate requested. This aggregation may also take into account that the mixer may contain certain transcoding capabilities (as discussed in under Topo-Mixer in [Topologies]), which can be employed for those few of the session participants that have the lowest available bit-rates. 3.5.4.4. Use of TMMBR in Point-to-Multipoint using Multicast or Translators In these topologies, corresponding to Topo-Multicast or Topo- Translator RTCP RRs are transmitted globally which allows for the detection of transmission problems such as congestion, on a medium timescale. As all media senders are aware of the congestion situation of all media receivers, the rationale of the use of TMMBR of section 3 .5.4.3 does not apply. However, even in this case the congestion control response can be improved when the unicast links are employing congestion controlled transport protocols (such as TCP or DCCP). A peer may also report local limitation to the media sender. 3.5.4.5. Use of TMMBR in Point-to-point operation In use case 7 it is possible to use TMMBR to improve the performance at times of changes in the known upper limit of the bit-rate. In this use case the signaling protocol has established an upper limit for the session and media bit-rates. However, at the time of transport link bit-rate reduction, a receiver could avoid serious congestion by sending a TMMBR to the sending side. Thus TMMBR is useful for putting restrictions on the application and thus placing the congestion control mechanism in the right ballpark. However TMMBR is usually unable to have continuously quick feedback loop required for real congestion control. Its semantics is also not a match for congestion control due to its different purpose. Because of these reasons TMMBR SHALL NOT be used for congestion control. Wenger, et al. Standards Track [Page 27] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 3.5.4.6. Reliability The reaction of a media sender to the reception of a TMMBR message is not immediately identifiable through inspection of the media stream. Therefore, a more explicit mechanism is needed to avoid unnecessary re-sending of TMMBR messages. Using a statistically based retransmission scheme would only provide statistical guarantees of the request being received. It would also not avoid the retransmission of already received messages. In addition, it does not allow for easy suppression of other participants requests. For the reasons mentioned, a mechanism based on explicit notification is used, as discussed already in section 3.5.4.1. Upon the reception of a request a media sender sends a notification containing the current applicable limitation of the bit-rate, and which session participants that own that limit. In multicast scenarios, that allows all other participants to suppress any request they may have, with limitation values less strict than the current ones. The identity of the owners allows for small message sizes and media sender states. A media sender only keeps state for the SSRCs of the current owners of the limitations; all other requests and their sources are not saved. Only the owners are allowed to remove or change its limitation. Otherwise, anyone that ever set a limitation would need to remove it to allow the maximum bit-rate to be raised beyond that value. Wenger, et al. Standards Track [Page 28] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 4. RTCP Receiver Report Extensions This memo specifies six new feedback messages. The Full Intra Request (FIR), Temporal-Spatial Trade-off Request (TSTR), Temporal-Spatial Trade-off Notification (TSTN), and Video Back Channel Message (VBCM) are "Payload Specific Feedback Messages" as defined in Section 6.3 of AVPF [RFC4585]. The Temporary Maximum Media Stream Bit-rate Request (TMMBR) and Temporary Maximum Media Stream Bit-rate Notification (TMMBN) are "Transport Layer Feedback Messages" as defined in Section 6.2 of AVPF. In the following subsections, the new feedback messages are defined, following a similar structure as in the AVPF specification's sections 6.2 and 6.3, respectively. 4.1. Design Principles of the Extension Mechanism RTCP was originally introduced as a channel to convey presence, reception quality statistics and hints on the desired media coding. A limited set of media control mechanisms have been introduced in early RTP payload formats for video formats, for example in RFC 2032 [RFC2032]. However, this specification, for the first time, suggests a two-way handshake for some of its messages. There is danger that this introduction could be misunderstood as the precedence for the use of RTCP as an RTP session control protocol. In order to prevent these misunderstandings, this subsection attempts to clarify the scope of the extensions specified in this memo, and strongly suggests that future extensions follow the rationale spelled out here, or compellingly explain why they divert from the rationale. In this memo, and in AVPF [RFC4585], only such messages have been included which a) have comparatively strict real-time constraints, which prevent the use of mechanisms such as a SIP re-invite in most application scenarios. The real-time constraints are explained separately for each message where necessary b) are multicast-safe in that the reaction to potentially contradicting feedback messages is specified, as necessary for each message c) are directly related to activities of a certain media codec, class of media codecs (e.g. video codecs), or a given RTP packet stream. In this memo, a two-way handshake is only introduced for such messages that Wenger, et al. Standards Track [Page 29] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 a) require a notification or acknowledgement due to their nature, which is motivated separately for each message b) the notification or acknowledgement cannot be easily derived from the media bit stream. All messages in AVPF [RFC4585] and in this memo implement their codepoints in a simple, fixed binary format. The reason behind this design principle lies in that media receivers do not always implement higher control protocol functionalities (SDP, XML parsers and such) in their media path. Therefore, simple binary representations are used in the feedback messages and not an (otherwise desirable) flexible format such as, for example, XML. 4.2. Transport Layer Feedback Messages Transport Layer FB messages are identified by the value RTPFB (205) as RTCP packet type (see section 6.1 of RFC 4585 [RFC4585]. In AVPF, one message of this category had been defined. This memo specifies two more messages, for a total of three messages of this type. They are identified by means of the FMT parameter as follows: 0: unassigned 1: Generic NACK (as per AVPF) 2: reserved (see note below) 3: Temporary Maximum Media Stream Bit-rate Request (TMMBR) 4: Temporary Maximum Media Stream Bit-rate Notification (TMMBN) 5-30: unassigned 31: reserved for future expansion of the identifier number space Note: early drafts of AVPF [RFC4585] reserved FMT=2 for a codepoint that has later been removed. It has been pointed out that there may be implementations in the field using this value for according to the expired draft. As there is sufficient numbering space available, we mark FMT=2 as reserved so to avoid possible interoperability problems with implementations that are standard-incompliant with respect to RFC 4585 in this very point. The following subsection defines the formats of the FCI field for this type of FB message. 4.2.1. Temporary Maximum Media Stream Bit-rate Request and Notification The FCI field of a Temporary Maximum Media Stream Bit-Rate Request (TMMBR) message SHALL contain one or more FCI entries. Wenger, et al. Standards Track [Page 30] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 4.2.1.1. Semantics TMMBR is used to indicate the transport related limitation in the form of a tuple. The first value is the highest bit-rate per sender of a media, which the receiver currently supports in this RTP session observed at a particular protocol layer. The second value is the measured header overhead in bytes on the packets received for the stream. Counting from the start of the header on the protocol layer for which the bit-rate is reported until the RTP payload's start. The measurement of the overhead is a running averaging that is updated for each packet received for this particular media source (SSRC). For each packet received the overhead is calculated (pckt_OH) and then added to the average overhead (avg_OH) by calculating: avg_OH = 15/16*avg_OH + 1/16*pckt_OH. The bit-rate values used in this formats are averaged out over a reasonable timescale. What reasonable timescales are, depends on the application. However the goal is be able to ignore any burstiness on very short timescales, below for example 100 ms, introduced by scheduling or link layer packetization effects. The media sender MAY use any combination of packet rate and RTP payload bit-rate to produce a lower media stream bit-rate, as it may need to address a congestion situation or other limiting factors. See section 5 . (congestion control) for more discussion. The ''SSRC of the packet sender'' field indicates the source of the request, and the ''SSRC of media source'' is not used and SHALL be set to 0. The SSRC of media sender in the FCI field denotes the media sender the message applies to. This is useful in the multicast or translator topologies where each media sender may be addressed in a single TMMBR message using multiple FCIs. A TMMBR FCI MAY be repeated in subsequent TMMBR messages if no applicable Temporal Maximum Media Stream Bit-Rate Notification (TMMBN) FCI has been received at the time of transmission of the next RTCP packet. A TMMBN is applicable if it either indicate the sender of the TMMBR as an owner, or contains limitations that are stricter than one sent in the TMMBR message. The bit-rate value of a TMMBR FCI MAY be changed from a previous TMMBR message and the next, regardless of the eventual reception of an applicable TMMBN FCI. The overhead measurement SHALL be updated to the current value of avg_OH. A TMMBN message SHALL be sent by the media sender at the earliest possible point in time, as a result of any TMMBR messages received since the last sending of TMMBN. The TMMBN message indicates the limits and the owners of those limits at the time of the transmission Wenger, et al. Standards Track [Page 31] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 of the message. The limits SHALL be set to the set of the stricts limits of the previous limits and all limits received in TMMBR FCI's since the last TMMBN was transmitted. A media receiver considering sending a TMMBR, who is not a ''owner'' of a limitation, SHOULD request a limitation stricter than their knowledge of the currently established limits for this media sender, or suppress their transmission of the TMMBR. The exception to the above rule is when a receiver either doesn't know the limit or is certain that their local representation of the set of limitations are in error. All received requests for limits equally or less strict compared to the ones currently established MUST BE ignored, with the exception of them resulting in the transmission of a TMMBN containg the current set of limitations. A media receiver who is the owner of a current limitation MAY lower the value further, raise the value or remove the restriction completely by setting the bit-rate part of the limit equal to the session bit-rate limit. A limitation tuple LT can be determined to be stricter or not compared to the current set of limitations if LT is part of the set Y produced by the algorithm described in Section 3.5.4.2. Once a session participant receives the TMMBN in response to its TMMBR, with its own SSRC, it knows that it "owns" the bitrate limitation. Only the "owner" of a bitrate limitation can raise it or reset it to the session limit. Note that, due to the unreliable nature of transport of TMMBR and TMMBN, the above rules may lead to the sending of TMMBR messages disobeying the rules above. Furthermore, in multicast scenarios it can happen that more than one session participants believes it "owns" the current bitrate limitation. This is not critical for a number of reasons: a) If a TMMBR message is lost in transmission, the media sender does not learn about the restrictions imposed on it. However, it also does not send a TMMBN message notifying reception of a request it has never received. Therefore, no new limit is established, the media receiver sending a more restrictive TMMBR is not the owner. Since this media receiver has not seen a notification corresponding to its request, it is free to re-send it. b) Similarly, if a TMMBN message gets lost, the media receiver that has sent the corresponding TMMBR request does not receive the Notification. In that case, it is also not the "owner" of the restriction and is free to re-send the request. c) If multiple competing TMMBR messages are sent by different session participants, then the resulting TMMBN indicates the most restrictive limits requested including its owners. Wenger, et al. Standards Track [Page 32] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 d) If more than one session participant incidently send TMMBR messages at the same time and with the same limit, the media sender selects one of them and addresses it as the ''owner''. Session-wide, the correct limit is thereby established. It is also important to consider the security risks involved with faked TMMBRs. See security considerations in Section 6 . The feedback messages may be used in both multicast and unicast sessions of any of the specified topologies. For sessions with a large number of participants using the lowest common denominator, as required by this mechanism, may not be the most suitable course of action. Large session may need to consider other ways to support adapted bit-rate to participants, such as partitioning the session in different quality tiers, or use some other method of achieving bit-rate scalability. If the value set by a TMMBR message is expected to be permanent, the TMMBR setting party is RECOMMENDED to renegotiate the session parameters to reflect that using session setup signaling, e.g. a SIP re-invite. An SSRC may time out according to the default rules for RTP session participants, i.e. the media sender has not received any RTCP packet from the owner for the last five regular reporting intervals. An SSRC may also leave the session, indicating this through the transmission of an RTCP BYE packet or an external signaling channel. In all of these cases the entity is considered to have left the session. In the case the "owner" leaves the session, the limit SHALL be removed and the transmission of a TMMBN is scheduled indicating the remaining limitations. 4.2.1.2. Message Format The Feedback Control Information (FCI) consists of one or more TMMBR FCI entries with the following syntax: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MMBR Exp | MMBR Mantissa |Measured Overhead| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1 - Syntax for the TMMBR message Wenger, et al. Standards Track [Page 33] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 SSRC: The SSRC value of the media sender that is requested to obey the new maximum bit-rate). MMBR Exp (6 bits): The exponential scaling of the mantissa for the Maximum Media Stream bit-rate value. The value is non signed integer [0..63]. MMBR Mantissa (17 bits): The mantissa of the Maximum Media Stream Bit-rate value as a non-signed integer. Measured Overhead (9 bits): The measured average packet overhead value in bytes. The measurement SHALL be done according to above description in Section 4.2.1.1. The maximum media stream bit-rate (MMBR) value in bits per second is calculated from the MMBR exponent (exp) and mantissa in the following way: MMBR = mantissa * 2^exp This allows for 17 bits of resolution in the range 0 to 131072*2^63 (approximately 1.2*10^24). The length of the FB message is be set to 2+2*N where N is the number of TMMBR FCI entries. 4.2.1.3. Timing Rules The first transmission of the request message MAY use early or immediate feedback in cases when timeliness is desirable. Any repetition of a request message SHOULD use regular RTCP mode for its transmission timing. 4.2.1.4. Handling in Translator and Mixers Media Translators and Mixers will need to receive and respond to TMMBR messages as they are part of the chain that provides a certain media stream to the receiver. The mixer or translator may act locally on the TMMBR request and thus generate a TMMBN to indicate that it has done so. Alternatively it can forward the request in the case of a media translator, or generate one of itself in the case of the mixer. In case it generates a TMMBR, it will need to send a TMMBN back to the original requestor to indicate that it is handling the request. Wenger, et al. Standards Track [Page 34] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 4.2.2. Temporary Maximum Media Stream Bit-rate Notification (TMMBN) The FCI field of the TMMBN Feedback message may contain zero, one or more TMMBN FCI entry. 4.2.2.1. Semantics This feedback message is used to notify the senders of any TMMBR message that one or more TMMBR messages have been received or that a owner has left the session. It indicates to all participants the set of currently employed limitations and the ''owners'' of those. The ''SSRC of the packet sender'' field indicates the source of the notification. The ''SSRC of media source'' is not used and SHALL be set to 0. A TMMBN message SHALL be scheduled for transmission after the reception of a TMMBR message with a FCI identifying this media sender. Only a single TMMBN SHALL be sent, even if more than one TMMBR messages are received between the scheduling of the transmission and the actual transmission of the TMMBN message. The TMMBN message indicates the limits and their owners at the time of transmitting the message. The limits included SHALL be the set of most restrictive values in the previously established set and received TMMBR messages since the last TMMBN was transmitted. The reception of a TMMBR message with a transmission limit equally or less restrictive than the set of current limits SHALL still result in the transmission of a TMMBN message. However the limits and their owners are not changed, unless it was from an owner of a limit within the current set of limitations. This procedure allows session participants that haven't seen the last TMMBN message to get a correct view of this media sender's state. When a media sender determines an ''owner'' of a limitation has left the session, then that limitation is removed, and the media sender SHALL send a TMMBN message indicating the remaining limitations. In case there are no remaining limitations a TMMBN without any FCI SHALL be sent to indicate this. In unicast scenarios (i.e. where a single sender talks to a single receiver), the aforementioned algorithm to determine ownership degenerates to the media receiver becoming the ''owner'' as soon as the media receiver has issued the first TMMBR message. Wenger, et al. Standards Track [Page 35] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 4.2.2.2. Message Format The Feedback Control Information (FCI) consists of zero, one or more TMMBN FCI entries with the following syntax: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MMBR Exp | MMBR Mantissa |Measured Overhead| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2 - Syntax for the TMMBR message SSRC: The SSRC value of the ''owner'' of this limitation. MMBR Exp (6 bits): The exponential scaling of the mantissa for the Maximum Media Stream bit-rate value. The value is non- signed integer [0..63]. MMBR Mantissa (17 bits): The mantissa of the Maximum Media Stream Bit-rate value as non-signed integer. Measured Overhead (9 bits): The measured average packet overhead value in bytes represented as non-signed integer. Thus the FCI contains blocks indicating the applicable limitations as the owner followed by the applicable maximum media stream bit-rate and overhead value. The length of the FB message is be set to 2+2*N where N is the number of TMMBR FCI entries. 4.2.2.3. Timing Rules The acknowledgement SHOULD be sent as soon as allowed by the applied timing rules for the session. Immediate or early feedback mode SHOULD be used for these messages. 4.2.2.4. Handling by Translators and Mixers As discussed in Section 4.2.1.4 mixer or translators may need to issue TMMBN messages as response to TMMBR messages handled by the mixer or translator. Wenger, et al. Standards Track [Page 36] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 4.3. Payload Specific Feedback Messages Payload-Specific FB messages are identified by the value PT=PSFB (206) as RTCP packet type (see section 6.1 of RFC 4585 [RFC4585]). AVPF defines three payload-specific FB messages and one application layer FB message. This memo specifies four additional payload- specific feedback messages. All are identified by means of the FMT parameter as follows: 0: unassigned 1: Picture Loss Indication (PLI) 2: Slice Lost Indication (SLI) 3: Reference Picture Selection Indication (RPSI) 4: Full Intra Request Command (FIR) 5: Temporal-Spatial Trade-off Request (TSTR) 6: Temporal-Spatial Trade-off Notification (TSTN) 7: Video Back Channel Message (VBCM) 8-14: unassigned 15: Application layer FB message 16-30: unassigned 31: reserved for future expansion of the number space The following subsections define the new FCI formats for the payload- specific FB messages. 4.3.1. Full Intra Request (FIR) The FIR message is identified by PT=PSFB and FMT=4. There MUST be one or more FIR entry contained in the FCI field. 4.3.1.1. Semantics Upon reception of FIR, the encoder MUST send a Decoder Refresh Point (see Section 2 ..2) as soon as possible. Note: Currently, video appears to be the only useful application for FIR, as it appears to be the only RTP payloads widely deployed that relies heavily on media prediction across RTP packet boundaries. However, use of FIR could also reasonably be envisioned for other media types that share essential properties with compressed video, namely cross-frame prediction (whatever a frame may be for that media type). One possible example may be the dynamic updates of MPEG-4 scene descriptions. It is suggested that Wenger, et al. Standards Track [Page 37] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 payload formats for such media types refer to FIR and other message types defined in this specification and in AVPF, instead of creating similar mechanisms in the payload specifications. The payload specifications may have to explain how the payload-specific terminologies map to the video-centric terminology used herein. Note: In environments where the sender has no control over the codec (e.g. when streaming pre-recorded and pre-coded content), the reaction to this command cannot be specified. One suitable reaction of a sender would be to skip forward in the video bit stream to the next decoder refresh point. In other scenarios, it may be preferable not to react to the command at all, e.g. when streaming to a large multicast group. Other reactions may also be possible. When deciding on a strategy, a sender could take into account factors such as the size of the receiving group, the ''importance'' of the sender of the FIR message (however ''importance'' may be defined in this specific application), the frequency of Decoder Refresh Points in the content, and others. However a session which predominately handles pre-coded content is not expected to use FIR at all. The sender MUST consider congestion control as outlined in section 5 ., which MAY restrict its ability to send a decoder refresh point quickly. Note: The relationship between the Picture Loss Indication and FIR is as follows. As discussed in section 6.3.1 of AVPF, a Picture Loss Indication informs the decoder about the loss of a picture and hence the likeliness of misalignment of the reference pictures in the encoder and decoder. Such a scenario is normally related to losses in an ongoing connection. In point-to-point scenarios, and without the presence of advanced error resilience tools, one possible option of an encoder consists in sending a Decoder Refresh Point. However, there are other options. One example is that the media sender ignores the PLI, because the embedded stream redundancy is likely to clean up the reproduced picture within a reasonable amount of time. The FIR, in contrast, leaves a (real- time) encoder no choice but to send a Decoder Refresh Point. It disallows the encoder to take into account any considerations such as the ones mentioned above. Note: Mandating a maximum delay for completing the sending of a Decoder Refresh Point would be desirable from an application viewpoint, but may be problematic from a congestion control point of view. ''As soon as possible'' as mentioned above appears to be a reasonable compromise. Wenger, et al. Standards Track [Page 38] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 FIR SHALL NOT be sent as a reaction to picture losses -- it is RECOMMENDED to use PLI instead. FIR SHOULD be used only in such situations where not sending a decoder refresh point would render the video unusable for the users. Note: A typical example where sending FIR is appropriate is when, in a multipoint conference, a new user joins the session and no regular Decoder Refresh Point interval is established. Another example would be a video switching MCU that changes streams. Here, normally, the MCU issues a FIR to the new sender so to force it to emit a Decoder Refresh Point. The Decoder Refresh Point includes normally a Freeze Picture Release (defined outside this specification), which re-starts the rendering process of the receivers. Both techniques mentioned are commonly used in MCU- based multipoint conferences. Other RTP payload specifications such as RFC 2032 [RFC2032] already define a feedback mechanism for certain codecs. An application supporting both schemes MUST use the feedback mechanism defined in this specification when sending feedback. For backward compatibility reasons, such an application SHOULD also be capable to receive and react to the feedback scheme defined in the respective RTP payload format, if this is required by that payload format. The ''SSRC of the packet sender'' field indicates the source of the request, and the ''SSRC of media source'' is not used and SHALL be set to 0. The SSRC of media sender to which the FIR command applies to is in the FCI. 4.3.1.2. Message Format Full Intra Request uses one additional FCI field, the content of which is depicted in Figure 3 The length of the FB message MUST be set to 2+2*N, where N is the number of FCI entries. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seq. nr | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 3 - Syntax for the FIR message Wenger, et al. Standards Track [Page 39] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 SSRC: The SSRC value of the media sender which is requested to send a Decoder Refresh Point. Seq. nr: Command sequence number. The sequence number space is unique for each tuple consisting of the SSRC of command source and the SSRC of the command target. The sequence number SHALL be increased by 1 modulo 256 for each new command. A repetition SHALL NOT increase the sequence number. Initial value is arbitrary. Reserved: All bits SHALL be set to 0 and SHALL be ignored on reception. The semantics of this FB message is independent of the RTP payload type. 4.3.1.3. Timing Rules The timing follows the rules outlined in section 3 of [RFC4585]. FIR commands MAY be used with early or immediate feedback. The FIR feedback message MAY be repeated. If using immediate feedback mode the repetition SHOULD wait at least one RTT before being sent. In early or regular RTCP mode the repetition is sent in the next regular RTCP packet. 4.3.1.4. Handling of message in Mixer and Translators A media translator or a mixer performing media encoding of the content for which the session participant has issued a FIR is responsible for acting upon it. A mixer acting upon a FIR SHOULD NOT forward the message unaltered, instead it SHOULD issue a FIR itself. 4.3.1.5. Remarks In conjunction with video codecs, FIR messages typically trigger the sending of full intra or IDR pictures. Both are several times larger then predicted (inter) pictures. Their size is independent of the time they are generated. In most environments, especially when employing bandwidth-limited links, the use of an intra picture implies an allowed delay that is a significant multitude of the typical frame duration. An example: If the sending frame rate is 10 fps, and an intra picture is assumed to be 10 times as big as an inter picture, then a full second of latency has to be accepted. In such an environment there is no need for a particularly short delay Wenger, et al. Standards Track [Page 40] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 in sending the FIR message. Hence waiting for the next possible time slot allowed by RTCP timing rules as per [RFC4585] may not have an overly negative impact on the system performance. 4.3.2. Temporal-Spatial Trade-off Request (TSTR) The TSTR FB message is identified by PT=PSFB and FMT=5. There MUST be one or more TSTR entry contained in the FCI field. 4.3.2.1. Semantics A decoder can suggest the use of a temporal-spatial trade-off by sending a TSTR message to an encoder. If the encoder is capable of adjusting its temporal-spatial trade-off, it SHOULD take into account the received TSTR message for future coding of pictures. A value of 0 suggests a high spatial quality and a value of 31 suggests a high frame rate. The values from 0 to 31 indicate monotonically a desire for higher frame rate. Actual values do not correspond to precise values of spatial quality or frame rate. The reaction to the reception of more than one TSTR message by a media sender from different media receivers is left open to the implementation. The selected trade-off SHALL be communicated to the media receivers by the means of the TSTN message. The ''SSRC of the packet sender'' field indicates the source of the request, and the ''SSRC of media source'' is not used and SHALL be set to 0. The SSRC of media sender to which the TSTR applies to is in the FCI entries. A TSTR message may contain multiple requests to different media senders, using multiple FCI entries. 4.3.2.2. Message Format The Temporal-Spatial Trade-off Request uses one FCI field, the content of which is depicted in Figure 4. The length of the FB message MUST be set to 2+2*N, where N is the number of FCI entries included. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Wenger, et al. Standards Track [Page 41] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 | SSRC | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seq nr. | Reserved | Index | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 4 - Syntax of the TSTR SSRC: The SSRC of the media sender which is requested to apply the tradeoff value in Index. Seq. nr: Request sequence number. The sequence number space is unique for each tuple consisting of the SSRC of request source and the SSRC of the request target. The sequence number SHALL be increased by 1 modulo 256 for each new command. A repetition SHALL NOT increase the sequence number. Initial value is arbitrary. Index: An integer value between 0 and 31 that indicates the relative trade off that is requested. An index value of 0 index highest possible spatial quality, while 31 indicates highest possible temporal resolution. Reserved: All bits SHALL be set to 0 and SHALL be ignored on reception. 4.3.2.3. Timing Rules The timing follows the rules outlined in section 3 of [RFC4585]. This request message is not time critical and SHOULD be sent using regular RTCP timing. Only if it is known that the user interface requires a quick feedback, the message MAY be sent with early or immediate feedback timing. 4.3.2.4. Handling of message in Mixers and Translators Mixer or Media translators that encodes content sent to the session participant issuing the TSTR SHALL consider the request to determine if it can fulfill it by changing its own encoding parameters. A media translator unable to fulfill the request MAY forward the request unaltered towards the media sender. A Mixer encoding for multiple session participants will need to consider the joint needs before generating a TSTR for itself towards the media sender. See also discussion in Section . 3.5.2. Wenger, et al. Standards Track [Page 42] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 4.3.2.5. Remarks The term "spatial quality" does not necessarily refer to the resolution, measured by the number of pixels the reconstructed video is using. In fact, in most scenarios the video resolution stays constant during the lifetime of a session. However, all video compression standards have means to adjust the spatial quality at a given resolution, often influenced by the Quantizer Parameter or QP. A numerically low QP results in a good reconstructed picture quality, whereas a numerically high QP yields a coarse picture. The typical reaction of an encoder to this request is to change its rate control parameters to use a lower frame rate and a numerically lower (on average) QP, or vice versa. The precise mapping of Index, frame rate, and QP is intentionally left open here, as it depends on factors such as compression standard employed, spatial resolution, content, bit rate, and many more. 4.3.3. Temporal-Spatial Trade-off Notification (TSTN) The TSTN message is identified by PT=PSFB and FMT=6. There SHALL be one or more TSTN contained in the FCI field. 4.3.3.1. Semantics This feedback message is used to acknowledge the reception of a TSTR. A TSTN entry in a TSTN feedback message SHALL be sent for each TSTR entry targeted to this session participant, i.e. each TSTR received that in the SSRC field in the entry has the receiving entities SSRC. A single TSTN message MAY acknowledge multiple requests using multiple FCI entries. The index value included SHALL be the same in all FCI's part of the TSTN message. Including a FCI for each requestor allows each requesting entity to determine that the media sender targeted have received the request. The Notification SHALL be sent also for repetitions received. If the request receiver has received TSTR with several different sequence numbers from a single requestor it SHALL only respond to the request with the highest (modulo 256) sequence number. The TSTN SHALL include the Temporal-Spatial Trade-off index that will be used as a result of the request. This is not necessarily the same index as requested, as media sender may need to aggregate requests from several requesting session participants. It may also have some other policies or rules that limit the selection. Wenger, et al. Standards Track [Page 43] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 The ''SSRC of the packet sender'' field indicates the source of the Notification, and the ''SSRC of media source'' is not used and SHALL be set to 0. The SSRC of the requesting entity to which the Notification applies to is in the FCI. 4.3.3.2. Message Format The Temporal-Spatial Trade-off Notification uses one additional FCI field, the content of which is depicted in Figure 5. The length of the FB message MUST be set to 2+2*N, where N is the number of FCI entries. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seq nr. | Reserved | Index | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5 - Syntax of the TSTN SSRC: The SSRC of the source of the TSTR request which resulted in this Notification. Seq. nr: The sequence number value from the TSTN request that is being acknowledged. Index: The trade-off value the media sender is using henceforth. Reserved: All bits SHALL be set to 0 and SHALL be ignored on reception. Informative note: The returned trade-off value (Index) may differ from the requested one, for example in cases where a media encoder cannot tune its trade-off, or when pre-recorded content is used. 4.3.3.3. Timing Rules The timing follows the rules outlined in section 3 of [RFC4585]. This acknowledgement message is not extremely time critical and SHOULD be sent using regular RTCP timing. Wenger, et al. Standards Track [Page 44] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 4.3.3.4. Handling of message in Mixer and Translators A Mixer or Translator that act upon a TSTR SHALL also send the corresponding TSTN. In cases it needs to forward a TSTR itself the notification message MAY need to be delayed until that request has been responded to. 4.3.3.5. Remarks None 4.3.4. H.271 Video Back Channel Message (VBCM) The VBCM is identified by PT=PSFB and FMT=7. There MUST be one or more VBCM entry contained in the FCI field. 4.3.4.1. Semantics The "payload" of VBCM indication carries codec-specific, different types of feedback information. The type of feedback information can be classified as a 'status report' (such as receiving bit stream without errors, or loss of a partial or complete picture or block) or 'update requests' (such as complete refresh of the bit stream). Note: There are possible overlaps between the VBCM sub- messages and CCM/AVPF feedback messages, such FIR. Please see section 3 ..5.3 for further discussions. The different types of feedback sub-messages carried in the VBCM are indicated by the ''payloadType'' as defined in [VBCM]. The different sub-message types as defined in [VBCM] are re-produced below for convenience. ''payloadType'', in ITU-T Rec. H.271 terminology, refers to the sub-type of the H.271 message and should not be confused with an RTP payload type. Wenger, et al. Standards Track [Page 45] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 Payload Type Message Content --------------------------------------------------------------------- 0 One or more pictures without detected bitstream error mismatch 1 One or more pictures that are entirely or partially lost 2 A set of blocks of one picture that is entirely or partially lost 3 CRC for one parameter set 4 CRC for all parameter sets of a certain type 5 A "reset" request indicating that the sender should completely refresh the video bitstream as if no prior bitstream data had been received > 5 Reserved for future use by ITU-T Table 2: H.271 message types The bit string or the "payload" of VBCM message is of variable length and is self-contained and coded in a variable length, binary format. The media sender necessarily has to be able to parse this optimized binary format to make use of VBCM messages Each of the different types of sub-messages (indicated by payloadType) may have different semantic based on the codec used. The ''SSRC of the packet sender'' field indicates the source of the request, and the ''SSRC of media source'' is not used and SHALL be set to 0. The SSRC of the media sender to which the VBCM message applies to is in the FCI. 4.3.4.2. Message Format The VBCM indication uses one FCI field and the syntax is depicted in Figure 6. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seq. nr |0| Payload Type| Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | VBCM Octet String.... | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 6 - Syntax for VBCM Message Wenger, et al. Standards Track [Page 46] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 SSRC: The SSRC value of the media sender that is requested to instruct its encoder to react to the VBCM message Seq. nr: Command sequence number. The sequence number space is unique for each tuple consisting of the SSRC of command source and the SSRC of the command target. The sequence number SHALL be increased by 1 modulo 256 for each new command. A repetition SHALL NOT increase the sequence number. Initial value is arbitrary. 0: Must be set to 0 and should not be acted upon receiving. Payload: The RTP payload type for which the VBCM bit stream must be interpreted. Length: The length of the VBCM octet string in octets exclusive any padding octets VBCM Octet String: This is the octet string generated by the decoder carrying a specific feedback sub-message. It is of variable length. Padding: Bytes set to 0 to make up a 32 bit boundary. 4.3.4.3. Timing Rules The timing follows the rules outlined in section 3 of [RFC4585]. The different sub-message types may have different properties in regards to the timing of messages that should be used. If several different types are included in the same feedback packet then the sub-message type with the most stringent requirements should be followed. 4.3.4.4. Handling of message in Mixer or Translator The handling of VBCM in a mixer or translator are sub-message type dependent. 4.3.4.5. Remarks Please see section 3.5.3 for the applicability of the VBCM message in relation to messages in both AVPF and this memo with similar functionality. Wenger, et al. Standards Track [Page 47] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 Note: There has been some discussion whether the payload type field in this message is needed. It would be needed if there were potentially more than one VBCM-capable RTP payload types in the same session, and that the semantics of a given VBCM message changes from PT to PT. This appears to be the case. For example, the picture identification mechanism in messages of H.271 type 0 is fundamentally different between H.263 and H.264 (although both use the same syntax. Therefore, the payload field is justified here. It was further commented that for TSTS and FIR such a need does not exist, because the semantics of TSTS and FIR are either loosely enough defined, or generic enough, to apply to all video payloads currently in existence/envisioned. 5. Congestion Control The correct application of the AVPF timing rules prevents the network from being flooded by feedback messages. Hence, assuming a correct implementation, the RTCP channel cannot break its bit-rate commitment and introduce congestion. The reception of some of the feedback messages modifies the behaviour of the media senders or, more specifically, the media encoders. All of these modifications MUST only be performed within the bandwidth limits the applied congestion control provides. For example, when reacting to a FIR, the unusually high number of packets that form the decoder refresh point have to be paced in compliance with the congestion control algorithm, even if the user experience suffers from a slowly transmitted decoder refresh point. A change of the Temporary Maximum Media Stream Bit-rate value can only mitigate congestion, but not cause congestion as long as congestion control is also employed. An increase of the value by a request REQUIRES the media sender to use congestion control when increasing its transmission rate to that value. A reduction of the value results in a reduced transmission bit-rate thus reducing the risk for congestion. 6. Security Considerations The defined messages have certain properties that have security implications. These must be addressed and taken into account by users of this protocol. The defined setup signaling mechanism is sensitive to modification attacks that can result in session creation with sub-optimal configuration, and, in the worst case, session rejection. To prevent Wenger, et al. Standards Track [Page 48] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 this type of attack, authentication and integrity protection of the setup signaling is required. Spoofed or maliciously created feedback messages of the type defined in this specification can have the following implications: a. Severely reduced media bit-rate due to false TMMBR messages that sets the maximum to a very low value. b. The assignment of the ownership of a bit-rate limit with a TMMBN message to the wrong participant. Thus potentially freezing the mechanism until a correct TMMBN message reached the participants. c. Sending TSTR that result in a video quality different from the user's desire, rendering the session less useful. d. Frequent FIR commands will potentially reduce the frame-rate making the video jerky due to the frequent usage of decoder refresh points. To prevent these attacks there is a need to apply authentication and integrity protection of the feedback messages. This can be accomplished against threats external to the current RTP session using the RTP profile that combines SRTP [SRTP] and AVPF into SAVPF [SAVPF]. In the Mixer cases, separate security contexts and filtering can be applied between the Mixer and the participants thus protecting other users on the Mixer from a misbehaving participant. 7. SDP Definitions Section 4 of [RFC4585] defines new SDP [RFC4566] attributes that are used for the capability exchange of the AVPF commands and indications, such as Reference Picture selection, Picture loss indication etc. The defined SDP attribute is known as rtcp-fb and its ABNF is described in section 4.2 of [RFC4585]. In this section we extend the rtcp-fb attribute to include the commands and indications that are described in this document for codec control protocol. We also discuss the Offer/Answer implications for the codec control commands and indications. 7.1. Extension of rtcp-fb attribute As described in [RFC4585], the rtcp-fb attribute is defined to indicate the capability of using RTCP feedback. As defined in AVPF the rtcp-fb attribute must only be used as a media level attribute and must not be provided at session level. All the rules described in [RFC4585] for rtcp-fb attribute relating to payload type and to multiple rtcp-fb attributes in a session description also apply to the new feedback messages defined in this memo. Wenger, et al. Standards Track [Page 49] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 The ABNF for rtcp-fb as defined in [RFC4585] is Rtcp-fb-syntax = "a=rtcp-fb: " rtcp-fb-pt SP rtcp-fb-val CRLF Where rtcp-fb-pt is the payload type and rtcp-fb-val defines the type of the feedback message such as ack, nack, trr-int and rtcp-fb-id. For example to indicate the support of feedback of picture loss indication, the sender declares the following in SDP v=0 o=alice 3203093520 3203093520 IN IP4 host.example.com s=Media with feedback t=0 0 c=IN IP4 host.example.com m=audio 49170 RTP/AVPF 98 a=rtpmap:98 H263-1998/90000 a=rtcp-fb:98 nack pli In this document we define a new feedback value type called "ccm" which indicates the support of codec control using RTCP feedback messages. The "ccm" feedback value should be used with parameters, which indicates the support of which codec commands the session may use. In this draft we define four parameters, which can be used with the ccm feedback value type. o "fir" indicates the support of Full Intra Request o "tmmbr" indicates the support of Temporal Maximum Media Stream Bit-rate. It has an optional sub parameter to indicate the session maximum packet rate to be used. If not included it defaults to infinity. o "tstr" indicates the support of temporal spatial trade-off request. O "vbcm" indicates the support of H.271 video back channel messages. In ABNF for rtcp-fb-val defined in [RFC4585], there is a placeholder called rtcp-fb-id to define new feedback types. The ccm is defined as a new feedback type in this document and the ABNF for the parameters for ccm are defined here (please refer section 4.2 of [RFC4585] for complete ABNF syntax). Rtcp-fb-param = SP "app" [SP byte-string] / SP rtcp-fb-ccm-param / ; empty Wenger, et al. Standards Track [Page 50] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 rtcp-fb-ccm-param = "ccm" SP ccm-param ccm-param = "fir" ; Full Intra Request / "tmmbr" [SP "smaxpr=" MaxPacketRateValue] ; Temporary max media bit rate / "tstr" ; Temporal Spatial Trade Off / "vbcm" *(SP subMessageType] ; H.271 VBCM messages / token [SP byte-string] ; for future commands/indications subMessageType = 1*8DIGIT byte-string = MaxPacketRateValue = 1*15DIGIT 7.2. Offer-Answer The Offer/Answer [RFC3264] implications to codec control protocol feedback messages are similar those described in [RFC4585]. The offerer MAY indicate the capability to support selected codec commands and indications. The answerer MUST remove all ccm parameters which it does not understand or does not wish to use in this particular media session. The answerer MUST NOT add new ccm parameters in addition to what has been offered. The answer is binding for the media session and both offerer and answerer MUST only use feedback messages negotiated in this way. The session maximum packet rate parameter part of the TMMBR indication is declarative and everyone shall use the highest value indicated in a response. If not present in a offer is SHALL NOT be included by the answerer. 7.3. Examples Example 1: The following SDP describes a point-to-point video call with H.263 with the originator of the call declaring its capability to support codec control messages - fir, tstr. The SDP is carried in a high level signaling protocol like SIP v=0 o=alice 3203093520 3203093520 IN IP4 host.example.com s=Point-to-Point call c=IN IP4 172.11.1.124 m=audio 49170 RTP/AVP 0 a=rtpmap:0 PCMU/8000 m=video 51372 RTP/AVPF 98 a=rtpmap:98 H263-1998/90000 Wenger, et al. Standards Track [Page 51] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 a=rtcp-fb:98 ccm tstr a=rtcp-fb:98 ccm fir In the above example the sender when it receives a TSTR message from the remote party can adjust the trade off as indicated in the RTCP TSTN feedback message. Example 2: The following SDP describes a SIP end point joining a video Mixer that is hosting a multiparty video conferencing session. The participant supports only the FIR (Full Intra Request) codec control command and it declares it in its session description. The video Mixer can send an FIR RTCP feedback message to this end point when it needs to send this participants video to other participants of the conference. v=0 o=alice 3203093520 3203093520 IN IP4 host.example.com s=Multiparty Video Call c=IN IP4 172.11.1.124 m=audio 49170 RTP/AVP 0 a=rtpmap:0 PCMU/8000 m=video 51372 RTP/AVPF 98 a=rtpmap:98 H263-1998/90000 a=rtcp-fb:98 ccm fir When the video MCU decides to route the video of this participant it sends an RTCP FIR feedback message. Upon receiving this feedback message the end point is mandated to generate a full intra request. Example 3: The following example describes the Offer/Answer implications for the codec control messages. The Offerer wishes to support "tstr", "fir" and "tmmbr" messages. The offered SDP is -------------> Offer v=0 o=alice 3203093520 3203093520 IN IP4 host.example.com s=Offer/Answer c=IN IP4 172.11.1.124 m=audio 49170 RTP/AVP 0 a=rtpmap:0 PCMU/8000 m=video 51372 RTP/AVPF 98 a=rtpmap:98 H263-1998/90000 a=rtcp-fb:98 ccm tstr a=rtcp-fb:98 ccm fir a=rtcp-fb:* ccm tmmbr smaxpr=120 Wenger, et al. Standards Track [Page 52] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 The answerer only wishes to support FIR and TSTR message as the codec control messages and the answerer SDP is <---------------- Answer v=0 o=alice 3203093520 3203093524 IN IP4 otherhost.example.com s=Offer/Answer c=IN IP4 189.13.1.37 m=audio 47190 RTP/AVP 0 a=rtpmap:0 PCMU/8000 m=video 53273 RTP/AVPF 98 a=rtpmap:98 H263-1998/90000 a=rtcp-fb:98 ccm tstr a=rtcp-fb:98 ccm fir Example 4: The following example describes the Offer/Answer implications for H.271 Video back channel messages (VBCM). The Offerer wishes to support VBCM and the submessages of payloadType 1 (One or more pictures that are entirely or partially lost) and 2 (a set of blocks of one picture that is entirely or partially lost). -------------> Offer v=0 o=alice 3203093520 3203093520 IN IP4 host.example.com s=Offer/Answer c=IN IP4 172.11.1.124 m=audio 49170 RTP/AVP 0 a=rtpmap:0 PCMU/8000 m=video 51372 RTP/AVPF 98 a=rtpmap:98 H263-1998/90000 a=rtcp-fb:98 ccm vbcm 1 2 The answerer only wishes to support sub-messages 1 only <---------------- Answer v=0 o=alice 3203093520 3203093524 IN IP4 otherhost.example.com s=Offer/Answer c=IN IP4 189.13.1.37 m=audio 47190 RTP/AVP 0 a=rtpmap:0 PCMU/8000 m=video 53273 RTP/AVPF 98 Wenger, et al. Standards Track [Page 53] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 a=rtpmap:98 H263-1998/90000 a=rtcp-fb:98 ccm vbcm 1 So in the above example only VBCM indication comprising of only "payloadType" 1 will be supported. 8. IANA Considerations The new value of ccm for the rtcp-fb attribute needs to be registered with IANA. Value name: ccm Long Name: Codec Control Commands and Indications Reference: RFC XXXX For use with "ccm" the following values also needs to be registered. Value name: fir Long name: Full Intra Request Command Usable with: ccm Reference: RFC XXXX Value name: tmmbr Long name: Temporary Maximum Media Stream Bit-rate Usable with: ccm Reference: RFC XXXX Value name: tstr Long name: temporal Spatial Trade Off Usable with: ccm Reference: RFC XXXX Value name: vbcm Long name: H.271 video back channel messages Usable with: ccm Reference: RFC XXXX 9. Acknowledgements The authors would like to thank Andrea Basso, Orit Levin, Nermeen Ismail for their work on the requirement and discussion draft [Basso]. Wenger, et al. Standards Track [Page 54] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 Drafts of this memo were reviewed and extensively commented by Roni Even, Colin Perkins, Randell Jesup, Keith Lantz, Harikishan Desineni, Guido Franceschini and others. The authors appreciate these reviews. Funding for the RFC Editor function is currently provided by the Internet Society. Wenger, et al. Standards Track [Page 55] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 10. References 10.1. Normative references [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., Rey, J., "Extended RTP Profile for Real-Time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, July 2006 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003. [RFC2327] Handley, M. and V. Jacobson, "SDP: Session Description Protocol", RFC 2327, April 1998. [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002. [Topologies] M. Westerlund, and S. Wenger, "RTP Topologies", draft- ietf-avt-topologies-00, work in progress, August 2006 10.2. Informative references [Basso] A. Basso, et. al., "Requirements for transport of video control commands", draft-basso-avt-videoconreq-02.txt, expired Internet Draft, October 2004. [AVC] Joint Video Team of ITU-T and ISO/IEC JTC 1, Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification (ITU-T Rec. H.264 | ISO/IEC 14496-10 AVC), Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, JVT-G050, March 2003. [NEWPRED] S. Fukunaga, T. Nakai, and H. Inoue, "Error Resilient Video Coding by Dynamic Replacing of Reference Pictures," in Proc. Globcom'96, vol. 3, pp. 1503 - 1508, 1996. [SRTP] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. Norrman, "The Secure Real-time Transport Protocol (SRTP)", RFC 3711, March 2004. [RFC2032] Turletti, T. and C. Huitema, "RTP Payload Format for H.261 Video Streams", RFC 2032, October 1996. [SAVPF] J. Ott, E. Carrara, "Extended Secure RTP Profile for RTCP-based Feedback (RTP/SAVPF)," draft-ietf-avt-profile- savpf-02.txt, July, 2005. [RFC3525] Groves, C., Pantaleo, M., Anderson, T., and T. Taylor, "Gateway Control Protocol Version 1", RFC 3525, June 2003. Wenger, et al. Standards Track [Page 56] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 [RFC3448] M. Handley, S. Floyd, J. Padhye, J. Widmer, "TCP Friendly Rate Control (TFRC): Protocol Specification", RFC 3448, Jan 2003 [VBCM] ITU-T Rec. H.271, "Video Back Channel Messages", June 2006 [RFC3890] Westerlund, M., "A Transport Independent Bandwidth Modifier for the Session Description Protocol (SDP)", RFC 3890, September 2004. [RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram Congestion Control Protocol (DCCP)", RFC 4340, March 2006. [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. [RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, June 2002. 11. Authors' Addresses Stephan Wenger Nokia Corporation P.O. Box 100 FIN-33721 Tampere FINLAND Phone: +358-50-486-0637 EMail: stewe@stewe.org Umesh Chandra Nokia Research Center 975, Page Mill Road, Palo Alto,CA 94304 USA Phone: +1-650-796-7502 Email: Umesh.Chandra@nokia.com Magnus Westerlund Ericsson Research Ericsson AB SE-164 80 Stockholm, SWEDEN Phone: +46 8 7190000 EMail: magnus.westerlund@ericsson.com Bo Burman Wenger, et al. Standards Track [Page 57] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 Ericsson Research Ericsson AB SE-164 80 Stockholm, SWEDEN Phone: +46 8 7190000 EMail: bo.burman@ericsson.com Full Copyright Statement Copyright (C) The IETF Trust (2007). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement Wenger, et al. Standards Track [Page 58] INTERNET-DRAFT AVPF RTCP-RR Extensions March 5, 2007 this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Acknowledgement Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). RFC Editor Considerations The RFC editor is requested to replace all occurrences of XXXX with the RFC number this document receives. Wenger, et al. Standards Track [Page 59]