Network Working Group S. Wenger Internet Draft J. Lennox Intended status: Standards Track J. Boyce Expires: March 2014 (Vidyo) 26 September 2013 H.265 Payload Header Extension and Temporal Scalability Support draft-wenger-payload-h265-payload-header-extension-00.txt Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on September 26, 2013 Copyright Notice Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this Wenger Expires March 27, 2014 [Page 1] Internet-Draft H.265 Payload Header Extension September 2013 document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Abstract Proposed are the allocation of a NAL unit type from the "unspecified" set of NAL units of H.265 [HEVC] for the purpose of payload header extensions, a generic syntax for the use of that pseudo NAL unit, and one application for it: the signaling of temporal scalability support information. Two alternative designs are included. The content of this draft is intended for incorporation into the H.265 RTP payload [h265-payload]. Table of Contents 1. Introduction...................................................2 2. Conventions used in this document..............................3 3. Two options for the RTP packet structure.......................3 4. RTP packet structure for an RTP carrying a PACI (Proposal 1)...4 5. RTP packet structure for an RTP carrying a PACI (Proposal 2)...5 6. Rules for PACI.................................................6 6.1. A PACI cannot be fragmented...............................6 6.2. A PACI cannot be aggregated...............................6 6.3. The payload of a PACI can be a fragment...................6 6.4. The payload of a PACI can be an aggregation NAL unit......6 7. Flag[0] == 1: Temporal Scalability Support.....................6 8. Security Considerations........................................8 9. IANA Considerations............................................8 10. References....................................................8 10.1. Normative References.....................................8 10.2. Informative References...................................8 11. Acknowledgments...............................................9 1. Introduction In H.265 and its associated payload format, one key design principle is that the NAL unit header co-serves as the payload header. There Wenger Expires March 25, 2014 [Page 2] Internet-Draft H.265 Payload Header Extension September 2013 is no RTP payload format defined payload header, thereby keeping the header overhead low. H.265 and its payload format share this feature with H.264 [H.264] and its payload format. When, during the development of the payload format for H.264/SVC [RFC6190], a need for a per packet payload header extension became obvious, the so-called PACSI NAL unit was designed for this purpose. This document follows a similar approach for the payload format for H.265: one NAL unit type of the numbering range reserved for that purpose (known as "unspecified" in H.265) is used to signal a Payload Content Information (PACI) NAL-unit like structure. The PACI carries a Payload Header Structure (PHS) and one H.265 NAL unit or one aggregation NAL unit or one fragment (all of which are defined in the H.265 payload spec). So far, one use case for the PACI has been identified, namely the support for the carriage of certain fields supporting temporal scalability that, in the H.264/SVC payload format, were carried in the PACSI NAL unit (and are, with slightly different terminology, also present in other RTP payload formats such as the VP8 payload format. Further uses are to be expected in support of the forthcoming scalable and/or multiview extension of H.265. PACI was used instead of PACSI, as the PACI is useful also for H.265 version 1 (which supports temporal scalability, but not other forms of scalability, and is not commonly recognized as a scalable video codec). 2. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [RFC2119]. In this document, these words will appear with that interpretation only when in ALL CAPS. Lower case uses of these words are not to be interpreted as carrying RFC-2119 significance. 3. Two options for the RTP packet structure We have included two alternatives for the packet structure of the RTP packet. Both carry the PACI and a NAL unit or NAL-unit like structure (such as an aggregation packet or a fragment). The difference between the two proposals is that in proposal 1, the PayloadHdr (with the exception of the NAL unit type) is taken from Wenger Expires March 25, 2014 [Page 3] Internet-Draft H.265 Payload Header Extension September 2013 the PayloadHdr of the NAL unit carried inside the packet, whereas in proposal 2, the bits used for the paylaod header other than the NAL unit type are used for control information. Proposal two saves 16 bits per PACI-carrying packet, but has the disadvantage of introducing a irregularity of the payload header for this packet-- something JCT-VC (and previous standards using NAL units in both ITU and IETF) have carefully avoided. We have not been able to identify a good reason why, in this particular case, this irregularity could be harmful, but that is certainly something that needs careful consideration. 4. RTP packet structure for an RTP carrying a PACI (Proposal 1) An RTP packet carrying a PACI has the following structure: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PayloadHdr |PHSsize | U | Flags[0..6] |X| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Payload Header Structure (PHS) | |=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=| | | | PACI payload: NAL unit | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| The RTP header is followed by the PayloadHdr (16 bit) as defined in the RTP payload format for H.265. Except for the NAL unit type field, the values of the fields of the PayloadHdr are the same as the respective values of PACI Payload, which is the NAL unit or NAL unit structure carried in the packet. The NAL unit type field is set to xxx (TBD, one of the "unspecified" values set aside for non- ITU standardization). The payload header is followed by 6 bits PHSsize, indicating the size (in octets) of the Payload Header Structure (PHS). Two unused bits (U) are for future extensions. Following are seven flags, Flag[0] through Flag [6], each indicating, when set, the presence of an optional field (or set of Wenger Expires March 25, 2014 [Page 4] Internet-Draft H.265 Payload Header Extension September 2013 fields) in the Payload Header Structure (PHS). The X bit, when set, indicates the presence of another octet consisting of seven Flags and another X bit, indicating the presence of more PHS fields) (for future extensions). The total length of all PHSsize, U, Flags, X-bits, and the PHS is limited to xxx (TBD. 32?) octets, to simplify encoder design for MTU size matching. 5. RTP packet structure for an RTP carrying a PACI (Proposal 2) An RTP packet carrying a PACI has the following structure: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = xx | PHSsize |F[0..2]|X| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | Payload Header Structure (PHS) | |=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=| | | | PACI payload: NAL unit | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| The RTP header is followed by the NAL-unit like structure type xxx (TBD) identifying this packet type. The total length of the Payload Header Structure (PHS) is limited to xxx (TBD. 32?) octets, to simplify encoder design for MTU size matching, and is indicated in PHSsize. Three Flags (F[0..2] indicate, when set the presence of the presence of an optional field (or set of fields) in the Payload Header Structure (PHS). The X bit, when set, indicates the presence of another octet consisting of seven Flags and another X bit, indicating the presence of more PHS fields) (for future extensions). Wenger Expires March 25, 2014 [Page 5] Internet-Draft H.265 Payload Header Extension September 2013 6. Rules for PACI Note: described below is the authors' choice of flexibility based on the use cases we are concerned about. From our viewpoint, less flexibility would be undesirable. More flexibility would lead to higher implementation complexity, but should be considered if use cases can be identified. 6.1. A PACI cannot be fragmented. If a PACI could be fragmented, and a fragment other than the first fragment would get lost, we would not have access to the information in the PACI. 6.2. A PACI cannot be aggregated Aggregation of PACIs is inadvisable from a compression viewpoint, as, in many cases, several to be aggregated NAL units would share identical PACI fields and values which would be carried redundantly for no reason. Most, if not all the practical effects of PACI aggregation can be achieved by aggregating NAL units and bundling them with a PACI (see below). 6.3. The payload of a PACI can be a fragment Helps for on-the-fly fragmentation. 6.4. The payload of a PACI can be an aggregation NAL unit Helps for small or unevenly sized NAL unit streams. 7. Flag[0] == 1: Temporal Scalability Support Flag[0] (bit 16) set to 1 in a PACI indicates the presence of temporal scalability information as follows: Wenger Expires March 25, 2014 [Page 6] Internet-Draft H.265 Payload Header Extension September 2013 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PayloadHdr |1|Flags[1..6]|X| TL0REFIDX | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| | IrapPicID |S|E| reserved| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | NALU | | . . . | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | :...OPTIONAL RTP padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ TL0PICIDX (8 bits) When present, the TL0PICIDX field MUST be set to equal to temporal_sub_layer_zero_idx as specified in Section D.3.32 [H.265] for the layer representation containing the NAL unit in the PACI. Note: one of the authors' implementation experience suggests that a larger field for TL0PICIDX may be desirable. The 8 bit field proposed above is aligned with H.265's SEI message. IrapPicID (8 bits) When present, the IrapPicID field MUST be set to equal to irap_pic_id as specified in Section D.3.32 [H.265] for the layer representation containing the NAL unit in the PACI. S (1 bit) The S bit MUST be set to 1, if the NL unit in the payload of the PACI is the first VCL NAL unit, in decoding order, of a layer representation. Otherwise, the S bit MUST be set to 0. Wenger Expires March 25, 2014 [Page 7] Internet-Draft H.265 Payload Header Extension September 2013 E (1 bit) The E bit MUST be set to 1, if the Nal unit in the payload of the PACI is the last VCL NAL unit, in decoding order, of a layer representation. Otherwise, the E bit MUST be set to 0. Note: we have removed the RFC 6190 language regarding aggregation and fragmentation cases. We will likely need to re-insert a -- perhaps modified -- version of that language in both S and E bit definitions. 8. Security Considerations None 9. IANA Considerations None 10. References 10.1. Normative References [HEVC] ITU-T Recommendation H.265, "High Efficiency Video Coding", April 2013 [H.264] ITU-T Recommendation H.264, "Advanced video coding for generic audiovisual services", January 2012 [h265-payload] draft-ietf-payload-h265-01 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 10.2. Informative References [RFC6190] Wenger, S., Wang, Y.-K., Schierl, T., and A. Eleftheriadis, "RTP Payload Format for Scalable Video Coding", RFC 6190, May 2011. Wenger Expires March 25, 2014 [Page 8] Internet-Draft H.265 Payload Header Extension September 2013 [VP8payload] draft-ietf-payload-vp8-09 11. Acknowledgments The authors appreciate the review and constructive comments of an early version of the document by Alex Eleftheriadis, Miska Hannuksela, and Ye-Kui Wang. This document was prepared using 2- Word-v2.0.template.dot. Any spelling errors and typos are solely Stephan's responsibility. Copyright (c) 2013 IETF Trust and the persons identified as authors of the code. All rights reserved. Redistribution and use in source and binary forms, with or without modification, is permitted pursuant to, and subject to the license terms contained in, the Simplified BSD License set forth in Section 4.c of the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info). Wenger Expires March 25, 2014 [Page 9] Internet-Draft H.265 Payload Header Extension September 2013 Authors' Addresses Stephan Wenger Jonathan Lennox Jill Boyce Vidyo, Inc. 433 Hackensack Ave, 7th Floor Hackensack, N.J. 07601 Email: {stephan,jonathan,jill}@vidyo.com Wenger Expires March 25, 2014 [Page 10]