rfc8797xml2.original.xml		rfc8797.xml
	<?xml version="1.0" encoding="US-ASCII"?>		<?xml version='1.0' encoding='utf-8'?>
	<!DOCTYPE rfc SYSTEM "rfc2629.dtd" >		<!DOCTYPE rfc SYSTEM "rfc2629-xhtml.ent">
	<?xml-stylesheet type="text/xsl" href="rfc2629.xslt" ?>
	<?rfc strict="yes" ?>
	<?rfc toc="yes"?>
	<?rfc symrefs="yes"?>
	<?rfc sortrefs="yes" ?>
	<?rfc compact="yes" ?>
	<?rfc subcompact="no" ?>
	<rfc
	category="std"
	docName="draft-ietf-nfsv4-rpcrdma-cm-pvt-data-08"
	ipr="trust200902"
	submissionType="IETF"
	updates="8166"
	xml:lang="en">
	<front>
	<title abbrev="RPC-Over-RDMA CM Private Data">
	RDMA Connection Manager Private Data For RPC-Over-RDMA Version 1
	</title>
	<author initials="C.L." surname="Lever" fullname="Charles Lever">
	<organization abbrev="Oracle">Oracle Corporation</organization>
	<address>
	<postal>
	<street></street>
	<city></city>
	<region></region>
	<code></code>
	<country>United States of America</country>
	</postal>
	<email>chuck.lever@oracle.com</email>
	</address>
	</author>
	<date />
	<area>Transport</area>
	<workgroup>Network File System Version 4</workgroup>
	<keyword>NFS-over-RDMA</keyword>
	<abstract>		<rfc xmlns:xi="http://www.w3.org/2001/XInclude" category="std"
			docName="draft-ietf-nfsv4-rpcrdma-cm-pvt-data-08"
			number="8797" ipr="trust200902" submissionType="IETF" consensus="true"
			updates="8166" xml:lang="en" obsoletes="" tocInclude="true" symRefs="true"
			sortRefs="true" version="3">
			<!-- xml2rfc v2v3 conversion 2.41.0 -->
			<front>
			<title abbrev="RPC-over-RDMA CM Private Data">
			Remote Direct Memory Access - Connection Manager (RDMA-CM) Private Data for
			RPC-over-RDMA Version 1</title>
			<seriesInfo name="RFC" value="8797"/>
			<author initials="C." surname="Lever" fullname="Charles Lever">
			<organization abbrev="Oracle">Oracle Corporation</organization>
			<address>
			<postal>
			<street/>
			<city/>
			<region/>
			<code/>
			<country>United States of America</country>
			</postal>
			<email>chuck.lever@oracle.com</email>
			</address>
			</author>
			<date month="June" year="2020"/>
	<t>		<keyword>NFS-over-RDMA</keyword>
			<abstract>
			<t>
	This document specifies the format of		This document specifies the format of
	Remote Direct Memory Access - Connection Manager (RDMA-CM) Private Data		Remote Direct Memory Access - Connection Manager (RDMA-CM) Private Data
	exchanged between RPC-over-RDMA version 1 peers		exchanged between RPC-over-RDMA version 1 peers
	as part of establishing a connection.		as part of establishing a connection.
	The addition of the private data payload specified in this document		The addition of the Private Data payload specified in this document
	is an optional extension		is an optional extension
	that does not alter the RPC-over-RDMA version 1 protocol.		that does not alter the RPC-over-RDMA version 1 protocol.
	This document updates RFC 8166.		This document updates RFC 8166.
	</t>		</t>
			</abstract>
	</abstract>		</front>
			<middle>
	</front>		<section anchor="section_A4741DA2-FF86-4A9D-907B-8A73D53C7148" numbered="tru
			e" toc="default">
	<middle>		<name>Introduction</name>
			<t>
	<section
	title="Introduction"
	anchor="section:A4741DA2-FF86-4A9D-907B-8A73D53C7148">
	<t>
	The RPC-over-RDMA version 1 transport protocol		The RPC-over-RDMA version 1 transport protocol
	<xref target="RFC8166"/>		<xref target="RFC8166" format="default"/>
	enables payload data transfer using		enables payload data transfer using
	Remote Direct Memory Access (RDMA)		Remote Direct Memory Access (RDMA)
	for upper-layer protocols based on Remote Procedure Calls (RPC)		for upper-layer protocols based on Remote Procedure Calls (RPCs)
	<xref target="RFC5531"/>.		<xref target="RFC5531" format="default"/>.
	The terms "Remote Direct Memory Access" (RDMA) and		The terms "Remote Direct Memory Access" (RDMA) and
	"Direct Data Placement" (DDP) are introduced in		"Direct Data Placement" (DDP) are introduced in
	<xref target="RFC5040"/>.		<xref target="RFC5040" format="default"/>.
	</t>		</t>
	<t>		<t>
	The two most immediate shortcomings		The two most immediate shortcomings
	of RPC-over-RDMA version 1 are:		of RPC-over-RDMA version 1 are as follows:
	<list style="symbols">		</t>
	<t>		<ol spacing="normal">
			<li>
			<t>
	Setting up an RDMA data transfer (via RDMA Read or Write) can be costly.		Setting up an RDMA data transfer (via RDMA Read or Write) can be costly.
	The small default size of messages transmitted using RDMA Send		The small default size of messages transmitted using RDMA Send
	forces the use of RDMA Read or Write operations		forces the use of RDMA Read or Write operations
	even for relatively small messages and data payloads.		even for relatively small messages and data payloads.
	<vspace/>		</t>
			<t>
	The original specification of RPC-over-RDMA version 1 provided		The original specification of RPC-over-RDMA version 1 provided
	an out-of-band protocol for passing inline threshold values		an out-of-band protocol for passing inline threshold values
	between connected peers		between connected peers
	<xref target="RFC5666"/>.		<xref target="RFC5666" format="default"/>.
	However,		However,
	<xref target="RFC8166"/>		<xref target="RFC8166" format="default"/>
	eliminated support for this protocol making it unavailable for this purpose.		eliminated support for this protocol, making it unavailable for this purpose.
	</t>		</t>
	<t>		</li>
			<li>
	Unlike most other contemporary RDMA-enabled storage protocols,		Unlike most other contemporary RDMA-enabled storage protocols,
	there is no facility in RPC-over-RDMA version 1		there is no facility in RPC-over-RDMA version 1
	that enables the use of remote invalidation		that enables the use of remote invalidation
	<xref target="RFC5042"/>.		<xref target="RFC5042" format="default"/>.
	</t>		</li>
	</list>		</ol>
	</t>		<t>
	<t>		Each RPC-over-RDMA version 1 Transport Header follows the
	Each RPC-over-RDMA version 1 transport header follows the		External Data Representation (XDR) definition
	External Data Representation (XDR)		<xref target="RFC4506" format="default"/>
	<xref target="RFC4506"/>		specified in <xref target="RFC8166" format="default"/>.
	definition specified in <xref target="RFC8166"/>.
	However, RPC-over-RDMA version 1		However, RPC-over-RDMA version 1
	has no means of extending this definition		has no means of extending this definition
	in such a way that interoperability with existing implementations is preserved.		in such a way that interoperability with existing implementations is preserved.
	As a result, an out-of-band mechanism is needed		As a result, an out-of-band mechanism is needed
	to help relieve these constraints		to help relieve these constraints
	for existing RPC-over-RDMA version 1 implementations.		for existing RPC-over-RDMA version 1 implementations.
	</t>		</t>
	<t>		<t>
	This document specifies a simple, non-XDR-based message format		This document specifies a simple, non-XDR-based message format
	designed to be passed between RPC-over-RDMA version 1 peers		designed to be passed between RPC-over-RDMA version 1 peers
	at the time each RDMA transport connection is first established.		at the time each RDMA transport connection is first established.
	The mechanism assumes that the underlying RDMA transport has a		The mechanism assumes that the underlying RDMA transport has a
	private data field that is passed between peers at connection time,		Private Data field that is passed between peers at connection time,
	such as is present in the iWARP protocol		such as is present in the Marker PDU Aligned Framing (MPA) protocol
	(described in Section 7.1 of <xref target="RFC5044"/>)		(described in <xref target="RFC5044" sectionFormat="of" section="7.1"/>
	or		and extended in <xref target="RFC6581"/>) or the InfiniBand Connection Manager
	the InfiniBand Connection Manager		<xref target="IBA" format="default"/>.
	<xref target="IBA"/>.
	</t>		</t>
	<t>		<t>
	To enable current RPC-over-RDMA version 1 implementations		To enable current RPC-over-RDMA version 1 implementations
	to interoperate with implementations that support		to interoperate with implementations that support
	the private message format described in this document,		the message format described in this document,
	implementation of the private data message is OPTIONAL.		implementation of the Private Data exchange is <bcp14>OPTIONAL</bcp14>.
	When the private data message has been successfully exchanged,		When Private Data has been successfully exchanged,
	peers may choose to perform extended RDMA semantics.		peers may choose to perform extended RDMA semantics.
	However, the private message format		However, this exchange
	does not alter the XDR definition specified in		does not alter the XDR definition specified in
	<xref target="RFC8166"/>.		<xref target="RFC8166" format="default"/>.
	</t>		</t>
	<t>		<t>
	The message format is intended to be further extensible		The message format is intended to be further extensible
	within the normal scope of such IETF work		within the normal scope of such IETF work
	(see		(see
	<xref target="section:96A18D5A-2AE4-490D-B140-B964302BF1DF"/>		<xref target="section_96A18D5A-2AE4-490D-B140-B964302BF1DF" format="default"/>
	for further details).		for further details).
	<xref target="section:8701C122-F206-4F87-8FA6-62F1CD648A14"/>		<xref target="section_8701C122-F206-4F87-8FA6-62F1CD648A14" format="default"/>
	of this document defines an IANA registry for this purpose.		of this document defines an IANA registry for this purpose.
	In addition, interoperation between		In addition, interoperation between
	implementations of RPC-over-RDMA version 1 that present this message format to p eers		implementations of RPC-over-RDMA version 1 that present this message format to p eers
	and those that do not recognize this message format is guaranteed.		and those that do not recognize this message format is guaranteed.
	</t>		</t>
	</section>		</section>
			<section anchor="section_6E4898D3-A2A9-47FA-9CBD-069328E610E4" numbered="tru
	<section		e" toc="default">
	title="Requirements Language"		<name>Requirements Language</name>
	anchor="section:6E4898D3-A2A9-47FA-9CBD-069328E610E4">		<t>The key words "<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>",
	<t>		"<bcp14>REQUIRED</bcp14>", "<bcp14>SHALL</bcp14>",
	The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",		"<bcp14>SHALL NOT</bcp14>", "<bcp14>SHOULD</bcp14>",
	"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED",		"<bcp14>SHOULD NOT</bcp14>",
	"MAY", and "OPTIONAL" in this document		"<bcp14>RECOMMENDED</bcp14>", "<bcp14>NOT RECOMMENDED</bcp14>",
	are to be interpreted as described in BCP 14		"<bcp14>MAY</bcp14>", and "<bcp14>OPTIONAL</bcp14>" in this document
	<xref target="RFC2119"/>		are to be interpreted as described in BCP&nbsp;14
	<xref target="RFC8174"/>		<xref target="RFC2119"/> <xref target="RFC8174"/> when, and only
	when, and only when, they appear in all capitals, as shown here.		when, they appear in all capitals, as shown here.</t>
	</t>		</section>
	</section>		<section anchor="section_D6ED5D4A-E123-4DDD-B65F-733F2B377679" numbered="tru
			e" toc="default">
	<section		<name>Advertised Transport Properties</name>
	title="Advertised Transport Properties"		<section anchor="section_0F60CBDD-8A7E-4B41-B193-18C04201DB3E" numbered="t
	anchor="section:D6ED5D4A-E123-4DDD-B65F-733F2B377679">		rue" toc="default">
			<name>Inline Threshold Size</name>
	<section		<t>
	title="Inline Threshold Size"		<xref target="RFC8166" sectionFormat="of" section="3.3.2"/>
	anchor="section:0F60CBDD-8A7E-4B41-B193-18C04201DB3E">		defines the term "inline threshold".
	<t>
	Section 3.3.2 of
	<xref target="RFC8166"/>
	defines the term "inline threshold."
	An inline threshold is the maximum number of bytes that		An inline threshold is the maximum number of bytes that
	can be transmitted using one RDMA Send and one RDMA Receive.		can be transmitted using one RDMA Send and one RDMA Receive.
	There are a pair of inline thresholds for a connection:		There are a pair of inline thresholds for a connection:
	a client-to-server threshold and a server-to-client threshold.		a client-to-server threshold and a server-to-client threshold.
	</t>		</t>
	<t>		<t>
	If an incoming RDMA message exceeds the size		If an incoming RDMA message exceeds the size
	of a receiver's inline threshold,		of a receiver's inline threshold,
	the receive operation fails		the Receive operation fails
	and		and
	the RDMA provider typically terminates the connection.		the RDMA provider typically terminates the connection.
	To convey an RPC message larger than the receiver's inline threshold		To convey an RPC message larger than the receiver's inline threshold
	without risking receive failure,		without risking receive failure,
	a sender must use explicit RDMA data transfer operations,		a sender must use explicit RDMA data transfer operations,
	which are more expensive than an RDMA Send.		which are more expensive than an RDMA Send.
	See Sections 3.3 and 3.5 of		See Sections&nbsp;<xref target="RFC8166" section="3.3"
	<xref target="RFC8166"/>		sectionFormat="bare"/> and <xref target="RFC8166" section="3.5"
			sectionFormat="bare"/> of <xref target="RFC8166"/>
	for a complete discussion.		for a complete discussion.
	</t>		</t>
	<t>		<t>
	The default value of inline thresholds for RPC-over-RDMA version 1		The default value of inline thresholds for RPC-over-RDMA version 1
	connections is 1024 bytes (as defined in Section 3.3.3 of		connections is 1024 bytes (as defined in <xref target="RFC8166"
	<xref target="RFC8166"/>).		sectionFormat="of" section="3.3.3"/>).
	This value is adequate for nearly all NFS version 3 procedures.		This value is adequate for nearly all NFS version 3 procedures.
	</t>		</t>
	<t>		<t>
	NFS version 4 COMPOUND operations		NFS version 4 COMPOUND operations
	<xref target="RFC7530"/>		<xref target="RFC7530" format="default"/>
	are larger on average		are larger on average
	than NFS version 3 procedures		than NFS version 3 procedures
	<xref target="RFC1813"/>,		<xref target="RFC1813" format="default"/>,
	forcing clients to use explicit RDMA operations		forcing clients to use explicit RDMA operations
	for frequently-issued requests such as LOOKUP and GETATTR.		for frequently issued requests such as LOOKUP and GETATTR.
	The use of RPCSEC_GSS security also increases the average size		The use of RPCSEC_GSS security also increases the average size
	of RPC messages,		of RPC messages,
	due to the larger size of RPCSEC_GSS credential material		due to the larger size of RPCSEC_GSS credential material
	included in RPC headers		included in RPC headers
	<xref target="RFC7861"/>.		<xref target="RFC7861" format="default"/>.
	</t>		</t>
	<t>		<t>
	If a sender and receiver could somehow agree on larger inline thresholds,		If a sender and receiver could somehow agree on larger inline thresholds,
	frequently-used RPC transactions avoid the cost of explicit RDMA operations.		frequently used RPC transactions avoid the cost of explicit RDMA operations.
	</t>		</t>
	</section>		</section>
			<section anchor="section_F82BDC05-F8A6-478C-A02C-BC308EE75A18" numbered="t
	<section		rue" toc="default">
	title="Remote Invalidation"		<name>Remote Invalidation</name>
	anchor="section:F82BDC05-F8A6-478C-A02C-BC308EE75A18">		<t>
	<t>
	After an RDMA data transfer operation completes,		After an RDMA data transfer operation completes,
	an RDMA consumer can request		an RDMA consumer can request
	that its peer's RDMA network interface card (RNIC)		that its peer's RDMA Network Interface Card (RNIC)
	invalidate the Steering Tag (STag)		invalidate the Steering Tag (STag)
	associated with the data transfer		associated with the data transfer
	<xref target="RFC5042"/>.		<xref target="RFC5042" format="default"/>.
	</t>		</t>
	<t>		<t>
	An RDMA consumer requests remote invalidation by posting		An RDMA consumer requests remote invalidation by posting
	an RDMA Send With Invalidate Work Request		an RDMA Send with Invalidate operation
	in place of an RDMA Send Work Request.		in place of an RDMA Send operation.
	Each RDMA Send With Invalidate carries one STag to invalidate.		Each RDMA Send with Invalidate carries one STag to invalidate.
	The receiver of an RDMA Send With Invalidate performs the		The receiver of an RDMA Send with Invalidate performs the
	requested invalidation and then reports that invalidation		requested invalidation and then reports that invalidation
	as part of the completion of a waiting Receive Work Request.		as part of the completion of a waiting Receive operation.
	</t>		</t>
	<t>		<t>
	If both peers support remote invalidation,		If both peers support remote invalidation,
	an RPC-over-RDMA responder might use remote invalidation		an RPC-over-RDMA responder might use remote invalidation
	when replying to an RPC request that provided chunks.		when replying to an RPC request that provided chunks.
	Because one of the chunks has already been invalidated,		Because one of the chunks has already been invalidated,
	finalizing the results of the RPC is made simpler and faster.		finalizing the results of the RPC is made simpler and faster.
	</t>		</t>
	<t>		<t>
	However, there are some important caveats which contraindicate		However, there are some important caveats that contraindicate
	the blanket use of remote invalidation:		the blanket use of remote invalidation:
	<list style="symbols">
	<t>
	Remote invalidation is not supported by all RNICs.
	</t>		</t>
	<t>		<ul spacing="normal">
			<li>
			Remote invalidation is not supported by all RNICs.
			</li>
			<li>
	Not all RPC-over-RDMA responder implementations can generate		Not all RPC-over-RDMA responder implementations can generate
	RDMA Send With Invalidate Work Requests.		RDMA Send with Invalidate operations.
	</t>		</li>
	<t>		<li>
	Not all RPC-over-RDMA requester implementations can recognize		Not all RPC-over-RDMA requester implementations can recognize
	when remote invalidation has occurred.		when remote invalidation has occurred.
	</t>		</li>
	<t>		<li>
	On one connection in different RPC-over-RDMA transactions,		On one connection in different RPC-over-RDMA transactions,
	or in a single RPC-over-RDMA transaction,		or in a single RPC-over-RDMA transaction,
	an RPC-over-RDMA requester can expose a mixture of STags		an RPC-over-RDMA requester can expose a mixture of STags
	that may be invalidated remotely		that may be invalidated remotely
	and some that must not be.		and some that must not be.
	No indication is provided at the RDMA layer as to which is which.		No indication is provided at the RDMA layer as to which is which.
	</t>		</li>
	</list>		</ul>
			<t>
	A responder therefore must not employ remote invalidation unless it is		A responder therefore must not employ remote invalidation unless it is
	aware of support for it in its own RDMA stack, and on the requester.		aware of support for it in its own RDMA stack, and on the requester.
	And, without altering the XDR structure of RPC-over-RDMA version 1 messages,		And, without altering the XDR structure of RPC-over-RDMA version 1 messages,
	it is not possible to support remote invalidation with requesters		it is not possible to support remote invalidation with requesters
	that mix STags that may and must not be invalidated remotely		that include an STag that must not be invalidated
	in a single RPC or on the same connection.		remotely in an RPC with STags that may be invalidated. Likewise, it
			is not possible to support remote invalidation with requesters that
			mix RPCs with STags that may be invalidated with RPCs with STags that
			must not be invalidated on the same connection.
	</t>		</t>
	<t>		<t>
	There are some NFS/RDMA client implementations whose STags		There are some NFS/RDMA client implementations whose STags
	are always safe to invalidate remotely.		are always safe to invalidate remotely.
	For such clients, indicating to the responder that remote		For such clients, indicating to the responder that remote
	invalidation is always safe can enable such invalidation		invalidation is always safe can enable such invalidation
	without the need for additional protocol elements to be defined.		without the need for additional protocol elements to be defined.
	</t>		</t>
	</section>		</section>
			</section>
	</section>		<section anchor="section_99541DF3-E579-42B6-9C7A-D020926798F6" numbered="tru
			e" toc="default">
	<section		<name>Private Data Message Format</name>
	title="Private Data Message Format"		<t>
	anchor="section:99541DF3-E579-42B6-9C7A-D020926798F6">
	<t>
	With an InfiniBand lower layer, for example,		With an InfiniBand lower layer, for example,
	RDMA connection setup uses a Connection Manager		RDMA connection setup uses a Connection Manager (CM)
	when establishing a Reliable Connection		when establishing a Reliable Connection
	<xref target="IBA"/>.		<xref target="IBA" format="default"/>.
	When an RPC-over-RDMA version 1 transport connection is established,		When an RPC-over-RDMA version 1 transport connection is established,
	the client (which actively establishes connections)		the client (which actively establishes connections)
	and the server (which passively accepts connections)		and the server (which passively accepts connections)
	populate the CM Private Data field exchanged		populate the CM Private Data field exchanged
	as part of CM connection establishment.		as part of CM connection establishment.
	</t>		</t>
	<t>		<t>
	The transport properties exchanged via this mechanism		The transport properties exchanged via this mechanism
	are fixed for the life of the connection.		are fixed for the life of the connection.
	Each new connection presents an opportunity		Each new connection presents an opportunity
	for a fresh exchange.		for a fresh exchange.
	An implementation of the extension described in this document		An implementation of the extension described in this document
	MUST be prepared for the settings to change upon a reconnection.		<bcp14>MUST</bcp14> be prepared for the settings to change upon a reconnection.
	</t>		</t>
	<t>		<t>
	For RPC-over-RDMA version 1, the CM Private Data field		For RPC-over-RDMA version 1, the CM Private Data field
	is formatted as described in the following subsection.		is formatted as described below. RPC clients and servers use the same format.
	RPC clients and servers use the same format.
	If the capacity of the Private Data field is too small		If the capacity of the Private Data field is too small
	to contain this message format		to contain this message format
	or		or
	the underlying RDMA transport is not managed by a Connection Manager,		the underlying RDMA transport is not managed by a CM,
	the CM Private Data field cannot be used on behalf of RPC-over-RDMA version 1.		the CM Private Data field cannot be used on behalf of RPC-over-RDMA version 1.
	</t>		</t>
	<t>		<t>
	The first 8 octets of the CM Private Data field		The first eight octets of the CM Private Data field
	is to be formatted as follows:		are to be formatted as follows:
	</t>		</t>
	<figure align="left">		<artwork align="left" name="" type="" alt=""><![CDATA[
	<artwork xml:space="preserve" align="left">		0 1 2 3
	0 1 2 3		0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
	0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1		+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
	+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+		| Format Identifier |
	| Format Identifier |		+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
	+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+		| Version | Reserved |R| Send Size | Receive Size |
	| Version | Reserved |R| Send Size | Receive Size |		+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+]]></artwork>
	+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+		<dl newline="false" spacing="normal">
	</artwork>		<dt>Format Identifier:</dt>
	</figure>		<dd>
	<t>
	<list style="hanging">
	<t hangText="Format Identifier:">
	This field contains a fixed 32-bit value that identifies		This field contains a fixed 32-bit value that identifies
	the content of the Private Data field as an RPC-over-RDMA		the content of the Private Data field as an RPC-over-RDMA
	version 1 CM Private Data message.		version 1 CM Private Data message.
	In RPC-over-RDMA version 1 Private Data,		In RPC-over-RDMA version&nbsp;1 Private Data,
	the value of this field is always 0xf6ab0e18, in network byte order.		the value of this field is always 0xf6ab0e18, in network byte order.
	The use of this field is further expanded upon in		The use of this field is further expanded upon in
	<xref target="section:0BD80E15-13FF-4210-A1D8-E0AE531821A4"/>.		<xref target="section_0BD80E15-13FF-4210-A1D8-E0AE531821A4" format="default"/>.
	</t>		</dd>
	<t hangText="Version:">		<dt>Version:</dt>
			<dd>
	This 8-bit field contains a message format version number.		This 8-bit field contains a message format version number.
	The value "1" in this field indicates that exactly eight octets are present,		The value "1" in this field indicates that exactly eight octets are present,
	that they appear in the order described in this section,		that they appear in the order described in this section,
	and that each has the meaning defined in this section.		and that each has the meaning defined in this section.
	Further considerations about the use of this field are discussed in		Further considerations about the use of this field are discussed in
	<xref target="section:96A18D5A-2AE4-490D-B140-B964302BF1DF"/>.		<xref target="section_96A18D5A-2AE4-490D-B140-B964302BF1DF" format="default"/>.
	</t>		</dd>
	<t hangText="Reserved:">		<dt>Reserved:</dt>
			<dd>
	This 7-bit field is unused.		This 7-bit field is unused.
	Senders MUST set these bits to zero		Senders <bcp14>MUST</bcp14> set these bits to zero,
	and		and
	receivers MUST ignore their value.		receivers <bcp14>MUST</bcp14> ignore their value.
	</t>		</dd>
	<t hangText="R:">		<dt>R:</dt>
			<dd>
	This 1-bit field indicates that		This 1-bit field indicates that
	the sender supports remote invalidation.		the sender supports remote invalidation.
	The field is set and interpreted as described in		The field is set and interpreted as described in
	<xref target="section:1D624044-49CC-4464-9BEF-59EA2473ACBF"/>.		<xref target="section_1D624044-49CC-4464-9BEF-59EA2473ACBF" format="default"/>.
	</t>		</dd>
	<t hangText="Send Size:">		<dt>Send Size:</dt>
			<dd>
	This 8-bit field contains an encoded value		This 8-bit field contains an encoded value
	corresponding to the maximum number of bytes		corresponding to the maximum number of bytes
	this peer is prepared to transmit in a single RDMA Send		this peer is prepared to transmit in a single RDMA Send
	on this connection.		on this connection.
	The value is encoded as described in		The value is encoded as described in
	<xref target="section:689BD9D4-9C3C-41C7-81F3-8BBE283D252B"/>.		<xref target="section_689BD9D4-9C3C-41C7-81F3-8BBE283D252B" format="default"/>.
	</t>		</dd>
	<t hangText="Receive Size:">		<dt>Receive Size:</dt>
			<dd>
	This 8-bit field contains an encoded value		This 8-bit field contains an encoded value
	corresponding to the maximum number of bytes		corresponding to the maximum number of bytes
	this peer is prepared to receive with a single RDMA Receive		this peer is prepared to receive with a single RDMA Receive
	on this connection.		on this connection.
	The value is encoded as described in		The value is encoded as described in
	<xref target="section:689BD9D4-9C3C-41C7-81F3-8BBE283D252B"/>.		<xref target="section_689BD9D4-9C3C-41C7-81F3-8BBE283D252B" format="default"/>.
	</t>		</dd>
	</list>		</dl>
	</t>		<section anchor="section_1D624044-49CC-4464-9BEF-59EA2473ACBF" numbered="t
			rue" toc="default">
	<section		<name>Using the R Field</name>
	title="Using the R Field"		<t>
	anchor="section:1D624044-49CC-4464-9BEF-59EA2473ACBF">		The R field indicates limited support for remote invalidation
	<t>
	The R field indicates limited support for remote invalidate
	as described in		as described in
	<xref target="section:F82BDC05-F8A6-478C-A02C-BC308EE75A18"/>.		<xref target="section_F82BDC05-F8A6-478C-A02C-BC308EE75A18"
			format="default"/>.
	When both connection peers have set this bit flag in their CM Private Data,		When both connection peers have set this bit flag in their CM Private Data,
	the responder MAY use RDMA Send With Invalidate		the responder <bcp14>MAY</bcp14> use RDMA Send with Invalidate operations
	when transmitting RPC Replies.		when transmitting RPC Replies.
	Each RDMA Send With Invalidate MUST invalidate an STag		Each RDMA Send with Invalidate <bcp14>MUST</bcp14> invalidate an STag
	associated only with the XID in the rdma_xid field		associated only with the Transaction ID (XID) in the rdma_xid field
	of the RPC-over-RDMA Transport Header it carries.		of the RPC-over-RDMA Transport Header it carries.
	</t>		</t>
	<t>		<t>
	When either peer on a connection clears this flag,		When either peer on a connection clears this flag,
	the responder MUST use only RDMA Send when transmitting RPC Replies.		the responder <bcp14>MUST</bcp14> use only RDMA Send when transmitting RPC Repli es.
	</t>		</t>
	</section>		</section>
			<section anchor="section_689BD9D4-9C3C-41C7-81F3-8BBE283D252B" numbered="t
	<section		rue" toc="default">
	title="Send and Receive Size Values"		<name>Send and Receive Size Values</name>
	anchor="section:689BD9D4-9C3C-41C7-81F3-8BBE283D252B">		<t>
	<t>
	Inline threshold sizes from 1024 to 262144 octets		Inline threshold sizes from 1024 to 262144 octets
	can be represented in the Send Size and Receive Size fields.		can be represented in the Send Size and Receive Size fields.
	The inline threshold values provide a pair of		The inline threshold values provide a pair of
	1024-octet-aligned maximum message lengths that		1024-octet-aligned maximum message lengths that
	guarantee Send and Receive operations		guarantee that Send and Receive operations
	do not fail due to length errors.		do not fail due to length errors.
	</t>		</t>
	<t>		<t>
	The minimum inline threshold for RPC-over-RDMA version 1		The minimum inline threshold for RPC-over-RDMA version 1
	is 1024 octets (see Section 3.3.3 of		is 1024 octets (see <xref target="RFC8166" sectionFormat="of" section="3.3.3"/>)
	<xref target="RFC8166"/>).		.
	The values in the Send Size and Receive Size fields represent		The values in the Send Size and Receive Size fields represent
	the unsigned number of additional kilo-octets of length		the unsigned number of additional kilo-octets of length
	beyond the first 1024 octets.		beyond the first 1024 octets.
	Thus, a sender computes the encoded value by		Thus, a sender computes the encoded value by
	dividing its actual buffer size, in octets, by 1024		dividing its actual buffer size, in octets, by 1024
	and		and
	subtracting one from the result.		subtracting one from the result.
	A receiver decodes an incoming Size value by performing		A receiver decodes an incoming Size value by performing
	the inverse set of operations:		the inverse set of operations:
	it adds one to the encoded value		it adds one to the encoded value
	and then		and then
	multiplies that result by 1024.		multiplies that result by 1024.
	</t>		</t>
	<t>		<t>
	The client uses the smaller of its own send size and		The client uses the smaller of its own send size and
	the server's reported receive size		the server's reported receive size
	as the client-to-server inline threshold.		as the client-to-server inline threshold.
	The server uses the smaller of its own send size and		The server uses the smaller of its own send size and
	the clients's reported receive size		the client's reported receive size
	as the server-to-client inline threshold.		as the server-to-client inline threshold.
	</t>		</t>
	</section>		</section>
			</section>
	</section>		<section anchor="section_58229D50-C3E7-48D1-A82F-7720AC8A9F8B" numbered="tru
			e" toc="default">
	<section		<name>Interoperability Considerations</name>
	title="Interoperability Considerations"		<t>
	anchor="section:58229D50-C3E7-48D1-A82F-7720AC8A9F8B">
	<t>
	The extension described in this document is designed to allow		The extension described in this document is designed to allow
	RPC-over-RDMA version implementations that use CM Private Data		RPC-over-RDMA version implementations that use CM Private Data
	to interoperate fully with		to interoperate fully with
	RPC-over-RDMA version 1 implementations that do not exchange this information.		RPC-over-RDMA version 1 implementations that do not exchange this information.
	Implementations that use this extension must also interoperate		Implementations that use this extension must also interoperate
	fully with RDMA implementations that use CM Private Data for other purposes.		fully with RDMA implementations that use CM Private Data for other purposes.
	Realizing these goals requires that implementations of this extension		Realizing these goals requires that implementations of this extension
	follow the practices described in the rest of this section.		follow the practices described in the rest of this section.
	</t>		</t>
			<section anchor="section_9A3B285D-D557-4028-ACB0-60EA1C9C93BE" numbered="t
	<section		rue" toc="default">
	title="Interoperability with RPC-over-RDMA Version 1 Implementations"		<name>Interoperability with RPC-over-RDMA Version 1 Implementations</nam
	anchor="section:9A3B285D-D557-4028-ACB0-60EA1C9C93BE">		e>
	<t>		<t>
	When a peer does not receive a CM Private Data message		When a peer does not receive a CM Private Data message
	which conforms to		that conforms to
	<xref target="section:99541DF3-E579-42B6-9C7A-D020926798F6"/>,		<xref target="section_99541DF3-E579-42B6-9C7A-D020926798F6" format="default"/>,
	it needs to act as if the remote peer supports only the		it needs to act as if the remote peer supports only the
	default RPC-over-RDMA version 1 settings,		default RPC-over-RDMA version 1 settings,
	as defined in		as defined in
	<xref target="RFC8166"/>.		<xref target="RFC8166" format="default"/>.
	In other words, the peer MUST behave as if a Private Data		In other words, the peer <bcp14>MUST</bcp14> behave as if a Private Data
	message was received in which bit 15 of the Flags field is zero,		message was received in which (1)&nbsp;bit 15 of the Flags field is zero
	and both Size fields contain the value zero.		and (2)&nbsp;both Size fields contain the value zero.
	</t>		</t>
	</section>		</section>
			<section anchor="section_0BD80E15-13FF-4210-A1D8-E0AE531821A4" numbered="t
	<section		rue" toc="default">
	title="Interoperability Amongst RDMA Transports"		<name>Interoperability amongst RDMA Transports</name>
	anchor="section:0BD80E15-13FF-4210-A1D8-E0AE531821A4">		<t>
	<t>
	The Format Identifier field defined in		The Format Identifier field defined in
	<xref target="section:99541DF3-E579-42B6-9C7A-D020926798F6"/>		<xref target="section_99541DF3-E579-42B6-9C7A-D020926798F6" format="default"/>
	is provided to enable implementations		is provided to enable implementations to distinguish the Private Data defined
	to distinguish RPC-over-RDMA version 1 Private Data		in this document from Private Data inserted at other layers, such as the
	from private data inserted at other layers,		additional Private Data defined by the MPAv2 protocol described in
	such as the private data inserted by		<xref target="RFC6581"/>, and others.
	the iWARP MPAv2 enhancement described in
	<xref target="RFC6581"/>.
	</t>		</t>
	<t>		<t>
	As part of connection establishment,		As part of connection establishment,
	the received private data buffer is searched for the Format Identifier word.		the buffer containing the received Private Data is searched for the Format Ident ifier word.
	The offset of the Format Identifier is not restricted to any alignment.		The offset of the Format Identifier is not restricted to any alignment.
	If the RPC-over-RDMA version 1 CM Private Data Format Identifier		If the RPC-over-RDMA version 1 CM Private Data Format Identifier
	is not present,		is not present,
	an RPC-over-RDMA version 1 receiver MUST		an RPC-over-RDMA version 1 receiver <bcp14>MUST</bcp14>
	behave as if no RPC-over-RDMA version 1 CM Private Data		behave as if no RPC-over-RDMA version 1 CM Private Data
	has been provided.		has been provided.
	</t>		</t>
	<t>		<t>
	Once the RPC-over-RDMA version 1 CM Private Data Format Identifier		Once the RPC-over-RDMA version 1 CM Private Data Format Identifier
	is found,		is found,
	the receiver parses the subsequent octets as		the receiver parses the subsequent octets as
	RPC-over-RDMA version 1 CM Private Data.		RPC-over-RDMA version 1 CM Private Data.
	As additional assurance that the private data content is valid		As additional assurance that the content is valid
	RPC-over-RDMA version 1 CM Private Data,		RPC-over-RDMA version 1 CM Private Data,
	the receiver should check that		the receiver should check that
	the format version number field contains a valid and recognized version number		the format version number field contains a valid and recognized version number
	and		and
	the size of the private data does not overrun the length of the buffer.		the size of the content does not overrun the length of the buffer.
	</t>		</t>
	</section>		</section>
			</section>
	</section>		<section anchor="section_96A18D5A-2AE4-490D-B140-B964302BF1DF" numbered="tru
			e" toc="default">
	<section		<name>Updating the Message Format</name>
	title="Updating the Message Format"		<t>
	anchor="section:96A18D5A-2AE4-490D-B140-B964302BF1DF">
	<t>
	Although the message format described in this document		Although the message format described in this document
	provides the ability for the client and server		provides the ability for the client and server
	to exchange particular information about		to exchange particular information about
	the local RPC-over-RDMA implementation,		the local RPC-over-RDMA implementation,
	it is possible that there will be a future need		it is possible that there will be a future need
	to exchange additional properties.		to exchange additional properties.
	This would make it necessary to extend or otherwise modify		This would make it necessary to extend or otherwise modify
	the format described in this document.		the format described in this document.
	</t>		</t>
	<t>		<t>
	Any modification faces the problem of interoperating properly		Any modification faces the problem of interoperating properly
	with implementations of RPC-over-RDMA version 1		with implementations of RPC-over-RDMA version 1
	that are unaware of the existence of the new format.		that are unaware of the existence of the new format.
	These include implementations that that do not recognize		These include implementations that do not recognize
	the exchange of CM Private Data		the exchange of CM Private Data
	as well as		as well as
	those that recognize only the format described in this document.		those that recognize only the format described in this document.
	</t>		</t>
	<t>		<t>
	Given the message format described in this document,		Given the message format described in this document,
	these interoperability constraints could be met by the following		these interoperability constraints could be met by the following
	sorts of new message formats:		sorts of new message formats:
	<list style="symbols">
	<t>
	A format which uses a different value for the first four bytes of the format,
	as provided for in the registry described in
	<xref target="section:8701C122-F206-4F87-8FA6-62F1CD648A14"/>.
	</t>		</t>
	<t>		<ul spacing="normal">
	A format which uses the same value for the Format Identifier field		<li>
			A format that uses a different value for the first four bytes of the format,
			as provided for in the registry described in
			<xref target="section_8701C122-F206-4F87-8FA6-62F1CD648A14" format="default"/>.
			</li>
			<li>
			A format that uses the same value for the Format Identifier field
	and a value other than one (1) in the Version field.		and a value other than one (1) in the Version field.
	</t>		</li>
	</list>		</ul>
			<t>
	Although it is possible to reorganize		Although it is possible to reorganize
	the last three of the eight bytes in the existing format,		the last three of the eight&nbsp;bytes in the existing format,
	extended formats are unlikely to do so.		extended formats are unlikely to do so.
	New formats would take the form of extensions		New formats would take the form of extensions
	of the format described in this document with added fields		of the format described in this document with added fields
	starting at byte eight of the format		starting at byte eight of the format
	or changes to the definition of bits in the Reserved field.		or changes to the definition of bits in the Reserved field.
	</t>		</t>
	</section>		</section>
			<section anchor="section_19F962AD-CDE5-4216-BE3A-44BD238458EC" numbered="tru
	<section		e" toc="default">
	title="Security Considerations"		<name>Security Considerations</name>
	anchor="section:19F962AD-CDE5-4216-BE3A-44BD238458EC">		<t>
	<t>
	The reader is directed to the Security Considerations section of		The reader is directed to the Security Considerations section of
	<xref target="RFC8166"/>		<xref target="RFC8166" format="default"/>
	for background and further discussion.		for background and further discussion.
	</t>		</t>
	<t>		<t>
	The RPC-over-RDMA version 1 protocol framework depends		The RPC-over-RDMA version 1 protocol framework depends
	on the semantics of the Reliable Connected (RC) queue pair (QP)		on the semantics of the Reliable Connected (RC) queue pair (QP)
	type, as defined in Section 9.7.7 of		type, as defined in
	<xref target="IBA"/>.		Section&nbsp;9.7.7 of <xref target="IBA"/>.
	The integrity of CM Private Data		The integrity of CM Private Data
	and		and
	the authenticity of its source		the authenticity of its source
	are ensured by the exclusive use of RC queue pairs.		are ensured by the exclusive use of RC QPs.
	Any attempt to interfere with or hijack data in transit		Any attempt to interfere with or hijack data in transit
	on an RC connection		on an RC connection
	results in the RDMA provider terminating the connection.		results in the RDMA provider terminating the connection.
	</t>		</t>
	<t>		<t>
	The Security Considerations section of		The Security Considerations section of
	<xref target="RFC5042"/>		<xref target="RFC5042" format="default"/>
	refers the reader to further relevant discussion		refers the reader to further relevant discussion
	of generic RDMA transport security.		of generic RDMA transport security.
	That document recommends IPsec as		That document recommends IPsec as
	the default transport layer security solution.		the default transport-layer security solution.
	When deployed with iWARP, IPsec establishes a protected channel		When deployed with the Remote Direct Memory Access Protocol (RDMAP) <xref
	before any iWARP operations are exchanged,		target="RFC5040"/>, DDP <xref target="RFC5041"/>, and MPA <xref
	thus it protects the exchange of Private Data		target="RFC5044"/>, IPsec establishes a protected channel before any
	that occurs as each QP is established.		operations are exchanged; thus, it protects the exchange of Private Data.
	However, IPsec is not available for InfiniBand or RoCE deployments.		However, IPsec is not available for InfiniBand or RDMA over Converged Ethernet (
			RoCE) deployments.
	Those fabrics rely on		Those fabrics rely on
	physical security		physical security
	and		and
	cyclic redundancy checks		cyclic redundancy checks
	to protect network traffic.		to protect network traffic.
	</t>		</t>
	<t>		<t>
	Exchanging the information		Exchanging the information
	contained in the Private Message format defined in this document		contained in the message format defined in this document
	does not expose upper-layer payloads to an attacker.		does not expose upper-layer payloads to an attacker.
	Furthermore, the behavior changes that occur		Furthermore, the behavior changes that occur
	as a result of processing the CM Private Data		as a result of exchanging the Private Data
	format described in the current document		described in the current document
	do not introduce any new risk of exposure		do not introduce any new risk of exposure
	of upper-layer payload data.		of upper-layer payload data.
	</t>		</t>
	<t>		<t>
	Improperly setting one of the fields in a version 1		Improperly setting one of the fields in version 1
	Private Message can result in an increased risk of disconnection		Private Data can result in an increased risk of disconnection
	(i.e., self-imposed Denial of Service).		(i.e., self-imposed Denial of Service).
	A similar risk can arise		A similar risk can arise
	if non-RPC-over-RDMA CM Private Data		if non-RPC-over-RDMA CM Private Data
	inadvertently contains the Format Identifier that		inadvertently contains the Format Identifier that
	identifies this protocol's data structure.		identifies this protocol's data structure.
	Additional checking of incoming Private Data,		Additional checking of incoming Private Data,
	as described in		as described in
	<xref target="section:0BD80E15-13FF-4210-A1D8-E0AE531821A4"/>,		<xref target="section_0BD80E15-13FF-4210-A1D8-E0AE531821A4" format="default"/>,
	can help reduce this risk.		can help reduce this risk.
	</t>		</t>
	<t>		<t>
	In addition to describing the structure of a new format version,		In addition to describing the structure of a new format version,
	any document that extends the Private Data format described		any document that extends the Private Data format described
	in the current document must discuss security considerations		in the current document must discuss security considerations
	of new data items exchanged between connection peers.		of new data items exchanged between connection peers.
	Such documents should also explore the risks		Such documents should also explore the risks
	of erroneously identifying non-RPC-over-RDMA CM Private Data		of erroneously identifying non-RPC-over-RDMA CM Private Data
	as the new format.		as the new format.
	</t>		</t>
	</section>		</section>
			<section anchor="section_8701C122-F206-4F87-8FA6-62F1CD648A14" numbered="tru
	<section		e" toc="default">
	title="IANA Considerations"		<name>IANA Considerations</name>
	anchor="section:8701C122-F206-4F87-8FA6-62F1CD648A14">		<t>
	<t>		IANA has created the "RDMA-CM Private Data Identifiers" subregistry within the
	In accordance with		"Remote Direct Data Placement" protocol category group.
	<xref target="RFC8126"/>,		This is a subregistry of 32-bit numbers that identify
	the author requests that IANA create a new registry in the
	"Remote Direct Data Placement"
	Protocol Category Group.
	The new registry is to be called the
	"RDMA-CM Private Data Identifier Registry".
	This is a registry of 32-bit numbers that identify
	the upper-layer protocol associated with data that appears in		the upper-layer protocol associated with data that appears in
	the application-specific RDMA-CM Private Data area.		the application-specific RDMA-CM Private Data area.
	The fields in this registry include:		The fields in this subregistry include the following:
	Format Identifier,		Format Identifier,
	Format Length (in octets),		Length (format length, in octets),
	Description,		Description,
	and		and
	Reference.		Reference.
	</t>		</t>
	<t>		<t>
	The initial contents of this registry are a single entry:		The initial contents of this registry are a single entry:
	</t>		</t>
	<texttable		<table align="left" anchor="table_336556D8-B52A-476E-8E9E-FDF4A4C7ED66">
	align="left"		<name>New &quot;RDMA-CM Private Data Identifiers&quot; Registry</name>
	style="full"		<thead>
	title="RDMA-CM Private Data Identifier Registry"		<tr>
	anchor="table:336556D8-B52A-476E-8E9E-FDF4A4C7ED66">		<th align="left">Format Identifier</th>
	<ttcol align="left">Format Identifier</ttcol>		<th align="left">Length</th>
	<ttcol align="left">Length</ttcol>		<th align="left">Description</th>
	<ttcol align="left">Description</ttcol>		<th align="left">Reference</th>
	<ttcol align="left">Reference</ttcol>		</tr>
	<c>0xf6ab0e18</c>		</thead>
	<c>8</c>		<tbody>
	<c>RPC-over-RDMA version 1 CM Private Data</c>		<tr>
	<c>[RFC-TBD]</c>		<td align="left">0xf6ab0e18</td>
	</texttable>		<td align="left">8</td>
			<td align="left">RPC-over-RDMA version 1 CM Private Data</td>
	<!-- RFC Editor:		<td align="left">RFC 8797</td>
	Please replace RFC-TBD with the RFC number assigned to this document		</tr>
			</tbody>
			</table>
	<t>		<t>
	IANA is to assign subsequent new entries in this registry using		IANA is to assign subsequent new entries in this registry using
	the Specification Required policy as defined in Section 4.6 of		the Specification Required policy as defined in <xref target="RFC8126" sectionFo
	<xref target="RFC8126"/>.		rmat="of" section="4.6"/>.
	</t>		</t>
			<section anchor="section_DB1C2628-A2D2-4EE7-8CD6-592F832C8C86" numbered="t
	<section		rue" toc="default">
	title="Guidance for Designated Experts"		<name>Guidance for Designated Experts</name>
	anchor="section:DB1C2628-A2D2-4EE7-8CD6-592F832C8C86">		<t>
	<t>
	The Designated Expert (DE), appointed by the IESG,		The Designated Expert (DE), appointed by the IESG,
	should ascertain the existence of suitable documentation that		should ascertain the existence of suitable documentation that
	defines the semantics and format of the private data,		defines the semantics and format of the Private Data,
	and verify that the document is permanently and publicly available.		and verify that the document is permanently and publicly available.
	Documentation produced outside the IETF must not conflict		Documentation produced outside the IETF must not conflict
	with work that is active or already published within the IETF.		with work that is active or already published within the IETF.
	The new Reference field should contain		The new Reference field should contain
	a reference to that documentation.		a reference to that documentation.
	</t>		</t>
	<t>		<t>
	The Description field should contain the name of the		The Description field should contain the name of the
	upper-layer protocol		upper-layer protocol
	that generates and uses the private data.		that generates and uses the Private Data.
	</t>		</t>
	<t>		<t>
	The DE should assign a new Format Identifier so that		The DE should assign a new Format Identifier so that
	it does not conflict with existing entries in this registry,		it does not conflict with existing entries in this registry
	and so that		and so that
	it is not likely to be mistaken		it is not likely to be mistaken
	as part of the payload of other registered formats.		as part of the payload of other registered formats.
	</t>		</t>
	<t>		<t>
	The DE shall post the request to the nfsv4 WG mailing list		The DE shall post the request to the NFSV4 Working Group mailing list
	(or a successor to that list, if such a list exists),		(or a successor to that list, if such a list exists)
	for comment and review.		for comment and review.
	The DE shall approve or deny the request and publish notice		The DE shall approve or deny the request and publish notice
	of the decision within 30 days.		of the decision within 30 days.
	</t>		</t>
	</section>		</section>
			</section>
	</section>		</middle>
			<back>
	</middle>		<references>
			<name>References</name>
	<back>		<references>
			<name>Normative References</name>
	<references title="Normative References">		<reference anchor="IBA" target="https://www.infinibandta.org/">
			<front>
	<reference		<title>InfiniBand Architecture Specification Volume 1</title>
	anchor="IBA">		<seriesInfo name="Release" value="1.3"/>
	<front>		<author>
	<title>InfiniBand Architecture Specification Volume 1</title>		<organization>InfiniBand Trade Association</organization>
	<author>		</author>
	<organization>InfiniBand Trade Association</organization>		<date month="March" year="2015"/>
	</author>		</front>
	<date month="March" year="2015"/>		</reference>
	</front>
	<seriesInfo name="Release" value="1.3"/>
	<annotation>
	Available from https://www.infinibandta.org/
	</annotation>
	</reference>
	<?rfc include="reference.RFC.2119.xml"?>
	<?rfc include="reference.RFC.4506.xml"?>
	<?rfc include="reference.RFC.5040.xml"?>
	<?rfc include="reference.RFC.5042.xml"?>
	<?rfc include="reference.RFC.8126.xml"?>
	<?rfc include="reference.RFC.8166.xml"?>
	<?rfc include="reference.RFC.8174.xml"?>
	</references>
	<references title="Informative References">
	<?rfc include="reference.RFC.1813.xml"?>
	<?rfc include="reference.RFC.5044.xml"?>
	<?rfc include="reference.RFC.5531.xml"?>
	<?rfc include="reference.RFC.5666.xml"?>
	<?rfc include="reference.RFC.6581.xml"?>
	<?rfc include="reference.RFC.7530.xml"?>
	<?rfc include="reference.RFC.7861.xml"?>
	</references>		<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2119.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4506.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5040.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5042.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8126.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8166.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8174.
			xml"/>
			</references>
			<references>
			<name>Informative References</name>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.1813.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5041.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5044.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5531.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5666.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6581.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7530.
			xml"/>
			<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7861.
			xml"/>
			</references>
			</references>
	<section		<section anchor="section_3910182B-831B-4CE3-9A51-B23F0C26A43B" numbered="fal
	title="Acknowledgments"		se" toc="default">
	anchor="section:3910182B-831B-4CE3-9A51-B23F0C26A43B"		<name>Acknowledgments</name>
	numbered="no">		<t>
	<t>
	Thanks to		Thanks to
	Christoph Hellwig		<contact fullname="Christoph Hellwig"/>
	and		and
	Devesh Sharma		<contact fullname="Devesh Sharma"/>
	for suggesting this approach,		for suggesting this approach,
	and to		and to
	Tom Talpey		<contact fullname="Tom Talpey"/>
	and		and
	Dave Noveck		<contact fullname="David Noveck"/>
	for their expert comments and review.		for their expert comments and review.
	The author also wishes to thank		The author also wishes to thank
	Bill Baker		<contact fullname="Bill Baker"/>
	and		and
	Greg Marsden		<contact fullname="Greg Marsden"/>
	for their support of this work.		for their support of this work.
	Also, thanks to expert reviewers		Also, thanks to expert reviewers
	Sean Hefty		<contact fullname="Sean Hefty"/>
	and		and
	Dave Minturn.		<contact fullname="Dave Minturn"/>.
	</t>		</t>
	<t>		<t>
	Special thanks go to		Special thanks go to
	document shepherd		document shepherd
	Brian Pawlowski,		<contact fullname="Brian Pawlowski"/>,
	Transport Area Director		Transport Area Director
	Magnus Westerlund,		<contact fullname="Magnus Westerlund"/>,
	NFSV4 Working Group Chairs		NFSV4 Working Group Chairs
	David Noveck		<contact fullname="David Noveck"/>
	and		and
	Spencer Shepler,		<contact fullname="Spencer Shepler"/>,
	and		and
	NFSV4 Working Group Secretary Thomas Haynes.		NFSV4 Working Group Secretary <contact fullname="Thomas Haynes"/>.
	</t>		</t>
	</section>		</section>
			</back>
	</back>
	</rfc>		</rfc>
End of changes. 146 change blocks.
	450 lines changed or deleted		425 lines changed or added
This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/