Payload WG T. Kristensen Internet-Draft M. Walters Updates: 6184 (if approved) Cisco Intended status: Standards Track October 15, 2012 Expires: April 18, 2013 Additional H.241 Parameter in the RTP Payload Format for H.264 Video draft-kristensen-payload-rtp-h241param-00 Abstract As systems increasingly are able to run video at 60 frames per second, there is a need to signal desired frame rate to optimise resolution choices and to avoid unnecessary resource or energy usage. This document updates RFC6184 and defines a new optional parameter addressing recent extensions currently supported in H.323 systems: The signalling of the maximum frames per second that can be efficiently received or that can be sent. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on April 18, 2013. Copyright Notice Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must Kristensen & Walters Expires April 18, 2013 [Page 1] Internet-Draft New H.264 RTP parameter October 2012 include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction and Overview . . . . . . . . . . . . . . . . . . 3 1.1. New Parameter max-fps . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Payload Format Parameters . . . . . . . . . . . . . . . . . . 4 3.1. Media Type Registration . . . . . . . . . . . . . . . . . 4 4. SDP Parameters . . . . . . . . . . . . . . . . . . . . . . . . 5 4.1. Mapping of the optional parameter to SDP . . . . . . . . . 5 4.2. Usage with SDP offer/answer . . . . . . . . . . . . . . . 5 4.3. Usage in Declarative Session Descriptions . . . . . . . . 5 4.4. Examples . . . . . . . . . . . . . . . . . . . . . . . . . 6 5. Co-existence . . . . . . . . . . . . . . . . . . . . . . . . . 6 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6 7. Security Considerations . . . . . . . . . . . . . . . . . . . 6 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 6 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 7 9.1. Normative References . . . . . . . . . . . . . . . . . . . 7 9.2. Informative references . . . . . . . . . . . . . . . . . . 7 Appendix A. Change History . . . . . . . . . . . . . . . . . . . 7 A.1. draft-kristensen-avt-rtp-h241param-02 to draft-kristensen-payload-rtp-h241param-00 . . . . . . . . 7 A.2. draft-kristensen-avt-rtp-h241param-01 to -02 . . . . . . . 8 A.3. draft-kristensen-avt-rtp-h241param-00 to -01 . . . . . . . 8 Appendix B. Figures . . . . . . . . . . . . . . . . . . . . . . . 8 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 Kristensen & Walters Expires April 18, 2013 [Page 2] Internet-Draft New H.264 RTP parameter October 2012 1. Introduction and Overview The extended video procedures and control signals for H.300 series terminals in ITU-T H.241 [ITU.H241.2006], associated with the ITU-T H.264 [ITU.H264.2007] codec, continue to evolve. This document describes a new parameter specified in H.241 - Amendment 2 [ITU.H241Amd2.2009] used to indicate the maximum picture rate that can be efficiently received or the maximum picture rate that can be sent. This proposal defines a media type parameter for maximum picture rate signalling and allows use in SDP-based systems, as this is not covered in [RFC6184]. Editorial note: This draft may be used to specify further, new H.264 RTP payload format parameters if applicable and when new parameters are defined. 1.1. New Parameter max-fps The use of MaxMBPS amd MaxFS, either signalled implicitly via supported level or as the optional parameters max-mbps and max-fs [RFC6184], underspecify the video stream and can lead to wasted bandwidth and processing resources, as well as unnecessary energy consumption and, where applicable, shorter battery duration. For example, a decoder signalling support for w1080p30 (243000 MBPS and 8100 MB) implicitly allows the sending of w720p60 (216000 MBPS and 3600 MB). Similarly, if a decoder signalled support for w720p30 (108000 MBPS and 3600 MB) the encoder could choose to send w480p60, w448p60 or even wCIF at 120 frames per second. Graphs visualizing these examples are included in Appendix B, where the existing parameters are used, with the addition of max-fps to constrain the allowed parameter range as preferred. Editorial note: The graphs in Appendix B will most likely be removed in later version, toghether with shortening of the description in this section. If the encoder chooses to send at a higher frame rate than preferred by the receiver side, the decoder will normally discard the additional frames after decoding them. The transmission of the extra frames and the processing of frames to just discard them are wasteful and the bandwidth and processing could be used more effectively. The new parameter MaxFPS is introduced in H.241 - Amendment 2 to remove this issue, by constraining the allowed parameter range for systems supporting this extension. This draft proposes adding the optional parameter max-fps for use in SDP-based signalling. Kristensen & Walters Expires April 18, 2013 [Page 3] Internet-Draft New H.264 RTP parameter October 2012 The specification of max-fps in this draft will also ease the gatewaying between H.323 and SIP systems by making the translation straight-forward for this parameter as well. It is worth emphasizing that systems that does not support and understand the max-fps parameter are free to produce streams as before, within the bounds specified by the existing parameters. 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. In video formats based on NTSC, 30 frames per second and 60 frames per second are used repeatedly. However, the actual values are 30/ 1.001=29.97 and 60/1.001=59.94. Values derived from these fractional values MUST be used in signalling descriptions. 3. Payload Format Parameters The optional H264 media subtype parameter max-fps specified below comes in addition to the list of optional H264 media subtype parameters defined in Section 8.1 of [RFC6184]. 3.1. Media Type Registration New OPTIONAL parameter: max-fps: The value of max-fps is an integer indicating the maximum frame rate in units of hundredths of frames per second that can be efficiently received. The signaled highest level is conveyed in the value of the profile-level-id parameter or the max-recv-level parameter. The max-fps parameter MAY be used to signal that the receiver has a constraint in that it is not capable of decoding video efficiently at the full frame rate that is implied by the signaled highest level and, if present, one or more of the parameters max-mbps, max-smbps or max-fs. Notice that the value of max-fps is not necessarily the frame rate at which the maximum frame size can be sent, it constitutes a constraint on maximum frame rate for all resolutions. Kristensen & Walters Expires April 18, 2013 [Page 4] Internet-Draft New H.264 RTP parameter October 2012 Informational note: The max-fps parameter is semantically different from max-mbps, max-smbps, max-fs, max-cpb, max-dpb, and max-br in that max-fps is used to signal a constraint, lowering the maximum frame rate from what is implied by the signaled MaxMBPS and MaxFS. The encoder MUST use a frame rate equal to or less than this value. In cases where the max-fps parameter is absent, or not understood, the encoder is free to choose any frame rate according to the highest signaled level and any signaled optional parameters. 4. SDP Parameters 4.1. Mapping of the optional parameter to SDP The optional parameter max-fps, when present, MUST be included in the "a=fmtp" line of SDP. The parameters in the "a=fmtp" line are expressed as a media type string, in the form of a semicolon separated list of parameter=value pairs. 4.2. Usage with SDP offer/answer When H.264 is offered over RTP using SDP in an offer/answer exchange [RFC3264] for negotiation of unicast streams, the following limitations and rules applies: The interpretation of the optional parameter max-fps depends on the direction attribute. When the direction attribute is sendonly, then it indicates the maximum picture rate that shall be sent. When the direction attribute is sendrecv or recvonly, the value of this parameter indicates the maximum picture frame rate that the receiver can efficiently handle. Any encoder that understands the parameter semantics shall constrain the frame rate to rates up to that specified. A receiver should have the ability to process video from a sender that does not understand this parameter. The profile-level-id parameter MUST be present in the same receiver capability description that contains this parameter. 4.3. Usage in Declarative Session Descriptions When H.264 over RTP is offered with SDP in a declarative style, the max-fps parameter MUST NOT be used. Kristensen & Walters Expires April 18, 2013 [Page 5] Internet-Draft New H.264 RTP parameter October 2012 4.4. Examples Two simple examples of SDP descriptions with the max-fps parameter. A NTSC-based system supporting maximum 30 frames per second could offer: m=video 49170 RTP/AVP 98 a=rtpmap:98 H264/90000 a=fmtp:98 profile-level-id=42A01E; max-fps=2997 A description for a system supporting a true 60 frames per second: m=video 49170 RTP/AVP 99 a=rtpmap:99 H264/90000 a=fmtp:99 profile-level-id=42A01E; max-fps=6000 5. Co-existence The "a=framerate" limitation, as specified in [RFC4566] allows a global maximum frame rate to be specified for the video media, this limit applies to all codecs in the video media section. In cases where both "a=framerate" and codec specific limits are specified, the maximum frame rate that can be used for a codec is the minimum of the values specified globally and specifically for a codec. 6. IANA Considerations This draft updates RFC6184 by adding a new optional parameter to the H264 media subtype. The parameter is specified in Section Section 3 of this document. 7. Security Considerations No separate considerations introduced in this document. Refer to section 9 of [RFC6184]. 8. Acknowledgements The authors would like to acknowledge Mo Zanaty (mzanaty@cisco.com), Roni Even (even.roni@huawei.com) and Stephen Botzko (stephen.botzko@polycom.com) for helpful input and comments. 9. References Kristensen & Walters Expires April 18, 2013 [Page 6] Internet-Draft New H.264 RTP parameter October 2012 9.1. Normative References [ITU.H241.2006] International Telecommunications Union, "Extended video procedures and control signals for H.300-series terminals", ITU-T Recommendation H.241, May 2006. [ITU.H241Amd2.2009] International Telecommunications Union, "Extended video procedures and control signals for H.300-series terminals", ITU-T Recommendation H.241 - Amendment 2, December 2009. [ITU.H264.2007] International Telecommunications Union, "Advanced video coding for generic audiovisual services", ITU- T Recommendation H.264, November 2007. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002. [RFC6184] Wang, Y., Even, R., Kristensen, T., and R. Jesup, "RTP Payload Format for H.264 Video", RFC 6184, May 2011. 9.2. Informative references [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. Appendix A. Change History A.1. draft-kristensen-avt-rtp-h241param-02 to draft-kristensen-payload-rtp-h241param-00 1. Normative description of max-fps as tentatively concluded on AVT WG mainling list earlier. 2. Shortened text based on previous discussion on AVT WG mailing list. 3. Submitted to Payload WG; draft name changed. Kristensen & Walters Expires April 18, 2013 [Page 7] Internet-Draft New H.264 RTP parameter October 2012 A.2. draft-kristensen-avt-rtp-h241param-01 to -02 1. Note; the media type registration text was added between -00 and -01 - along the lines of sar-supported in [RFC6184]. 2. Changed back to -00 normative language for max-fps media type registration in Section 3.1, use MAY not SHOULD for an OPTIONAL parameter! A.3. draft-kristensen-avt-rtp-h241param-00 to -01 1. Tightened text describing "a=maxprate" and H.26x frame rate signalling parameters, to get the real message clearer through. 2. Emphasized the optimization and energy consumption/battery duration as reasons for specifying max-pfs. 3. Added graphs as an aid visualizing the effect of max-fps, together with macroblock and frame size signalling 4. Clarified the 60/30 and 29.97/59.94 value usage in the document. 5. A better text regarding co-existence with "a=framerate" added. 6. Misc. editorial fixes and cleanup. Appendix B. Figures Graphical representation of the usage of the existing parameters for signalling supported number of macroblocks per second (MaxMBPS) and frame size (MaxFS), with addition of the proposed max-fps parameter in an example scenario. Figure 1 assumes that an encoder can encode at one of three resolutions wCIF, w720p and w1080p, and three different frame rates 30, 60 and 120 frames per second (fps), but not all combinations of these are possible. Kristensen & Walters Expires April 18, 2013 [Page 8] Internet-Draft New H.264 RTP parameter October 2012 | | 20 fps + | | | | 30 fps + * * * | | | | 60 fps + * * | | 120 fps + * | | +----------+----------+----------+----------+----------+-- 0 2000 4000 6000 8000 10000 MaxFS Figure 1: Encoder capabilities The decoder can then signal its capabilities. For example, if it can decode 108000 macroblocks per second, then the encoder can choose any combination of frame size and frame rate within the dotted area as shown in Figure 2. Kristensen & Walters Expires April 18, 2013 [Page 9] Internet-Draft New H.264 RTP parameter October 2012 |. . . . . . . . . . . . . . . . . . |. . . . . . . . . . . . . . . . . 20 fps +. . . . . . . . . . . . . . . . |. . . . . . . . . . . . . . . |. . . . . . . . . . . . . . |. . . . . . . . . . . . . |. . . . . . . . . . . . 30 fps +. * . . . . . . .*. . * |. . . . . . . . . . |. . . . . . . . . |. . . . . . . . |. . . . . . . 60 fps +. * . . . . * |. . . . . |. . . . 120 fps +. * . |. . |. +----------+----------+----------+----------+----------+-- 0 2000 4000 6000 8000 10000 MaxFS Figure 2: MaxMBPS indicated If the decoder signals a maximum frame size (either explicitly as max-fs or via the supported level), then this further constrains the choice of the decoder as shown in Figure 3. Kristensen & Walters Expires April 18, 2013 [Page 10] Internet-Draft New H.264 RTP parameter October 2012 |. . . . . . . . . |. . . . . . . . .| 20 fps +. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . .| 30 fps +. * . . . . . . .* * |. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . | |. . . . . . . | 60 fps +. * . . . . * |. . . . . | |. . . . | 120 fps +. * . | |. . | |. | +---------+----------+----------+----------+----------+-- 0 2000 4000 6000 8000 10000 MaxFS Figure 3: MaxMBPS and MaxFS indicated The parameter max-fps is added as a horizontal limit in Figure 4 according to the decoder's preferences. This restricts the available parameter space for the encoder. Kristensen & Walters Expires April 18, 2013 [Page 11] Internet-Draft New H.264 RTP parameter October 2012 |. . . . . . . . . |. . . . . . . . .| 20 fps +. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . .| 30 fps +. * . . . . . . .* * |. . . . . . . . .| |. . . . . . . . .| |. . . . . . . . | |. . . . . . . | 60 fps +--*--------------*-------------------------------------- | | | | 120 fps + * | | | | | +---------+----------+----------+----------+----------+-- 0 2000 4000 6000 8000 10000 MaxFS Figure 4: Adding max-fps to further constrain Authors' Addresses Tom Kristensen Cisco Philip Pedersens vei 22 N-1366 Lysaker Norway Email: tomkrist@cisco.com, tomkri@ifi.uio.no Malcolm Walters Cisco 14 Waterside Drive Langley, Slough UK Email: malcowal@cisco.com Kristensen & Walters Expires April 18, 2013 [Page 12]