rfc9732xml2.original.xml   rfc9732.xml 
<?xml version="1.0" encoding="US-ASCII"?> <?xml version='1.0' encoding='UTF-8'?>
<?xml-stylesheet type="text/xsl" href="rfc2629.xslt" ?>
<!-- generated by https://github.com/cabo/kramdown-rfc2629 version 1.2.3 --> <!DOCTYPE rfc [
<!DOCTYPE rfc SYSTEM "rfc2629.dtd"> <!ENTITY nbsp "&#160;">
<?rfc toc="yes"?> <!ENTITY zwsp "&#8203;">
<?rfc sortrefs="yes"?> <!ENTITY nbhy "&#8209;">
<?rfc symrefs="yes"?> <!ENTITY wj "&#8288;">
<?rfc comments="yes"?> ]>
<rfc category="info" docName="draft-ietf-teas-enhanced-vpn-20"
ipr="trust200902"> <rfc xmlns:xi="http://www.w3.org/2001/XInclude" category="info" docName="draft-i
<front> etf-teas-enhanced-vpn-20" number="9732" consensus="true" ipr="trust200902" obsol
<title abbrev="Enhanced VPN Framework">A Framework for Network Resource etes="" updates="" submissionType="IETF" xml:lang="en" tocInclude="true" sortRef
Partition (NRP) based Enhanced Virtual Private Networks</title> s="true" symRefs="true" version="3">
<front>
<title abbrev="Enhanced VPN Framework">A Framework for NRP-Based Enhanced Vi
rtual Private Networks</title>
<seriesInfo name="RFC" value="9732"/>
<author fullname="Jie Dong" initials="J." surname="Dong"> <author fullname="Jie Dong" initials="J." surname="Dong">
<organization>Huawei</organization> <organization>Huawei</organization>
<address> <address>
<email>jie.dong@huawei.com</email> <email>jie.dong@huawei.com</email>
</address> </address>
</author> </author>
<author fullname="Stewart Bryant" initials="S." surname="Bryant"> <author fullname="Stewart Bryant" initials="S." surname="Bryant">
<organization>University of Surrey</organization> <organization>University of Surrey</organization>
<address> <address>
<email>stewart.bryant@gmail.com</email> <email>stewart.bryant@gmail.com</email>
</address> </address>
</author> </author>
<author fullname="Zhenqiang Li" initials="Z." surname="Li"> <author fullname="Zhenqiang Li" initials="Z." surname="Li">
<organization>China Mobile</organization> <organization>China Mobile</organization>
<address> <address>
<email>lizhenqiang@chinamobile.com</email> <email>lizhenqiang@chinamobile.com</email>
</address> </address>
</author> </author>
<author fullname="Takuya Miyasaka" initials="T." surname="Miyasaka"> <author fullname="Takuya Miyasaka" initials="T." surname="Miyasaka">
<organization>KDDI Corporation</organization> <organization>KDDI Corporation</organization>
<address> <address>
<email>ta-miyasaka@kddi.com</email> <email>ta-miyasaka@kddi.com</email>
</address> </address>
</author> </author>
<author fullname="Young Lee" initials="Y." surname="Lee"> <author fullname="Young Lee" initials="Y." surname="Lee">
<organization>Samsung</organization> <organization>Samsung</organization>
<address> <address>
<email>younglee.tx@gmail.com</email> <email>younglee.tx@gmail.com</email>
</address> </address>
</author> </author>
<date month="March" year="2025"/>
<area>RTG</area>
<workgroup>teas</workgroup>
<date day="14" month="June" year="2024"/> <keyword>network slicing</keyword>
<keyword>traffic engineering</keyword>
<workgroup>TEAS Working Group</workgroup> <keyword>performance guarantee</keyword>
<abstract> <abstract>
<t>This document describes the framework for Network Resource Partition <t>This document describes a framework for Enhanced Virtual Private
(NRP) based Enhanced Virtual Private Networks (VPNs) to support the Networks (VPNs) based on Network Resource Partitions (NRPs) in order to
needs of applications with specific traffic performance requirements support the needs of applications with specific traffic performance
(e.g., low latency, bounded jitter). An NRP represents a subset of requirements (e.g., low latency, bounded jitter). NRP-based enhanced
network resources and associated policies in the underlay network. VPNs leverage the VPN and Traffic Engineering (TE) technologies and add
NRP-based Enhanced VPNs leverage the VPN and Traffic Engineering (TE) characteristics that specific services require beyond those provided by
technologies and add characteristics that specific services require conventional VPNs. Typically, an NRP-based enhanced VPN will be used to
beyond those provided by conventional VPNs. Typically, an NRP-based underpin network slicing, but it could also be of use in its own right
enhanced VPN will be used to underpin network slicing, but could also be providing enhanced connectivity services between customer sites. This
of use in its own right providing enhanced connectivity services between document also provides an overview of relevant technologies in different
customer sites. This document also provides an overview of relevant network layers and identifies some areas for potential new work.</t>
technologies in different network layers, and identifies some areas for
potential new work.</t>
</abstract> </abstract>
</front> </front>
<middle> <middle>
<section anchor="introduction" title="Introduction"> <section anchor="introduction" numbered="true" toc="default">
<name>Introduction</name>
<t>Virtual Private Networks (VPNs) have served the industry well as a <t>Virtual Private Networks (VPNs) have served the industry well as a
means of providing different groups of users with logically isolated means of providing different groups of users with logically isolated
connectivity over a common network. The common (base) network that is connectivity over a common network. The common (base) network that is
used to provide the VPNs is often referred to as the underlay, and the used to provide the VPNs is often referred to as the "underlay", and the
VPN is often called an overlay.</t> VPN is often called an "overlay".</t>
<t>Customers of a network operator may request connectivity services <t>Customers of a network operator may request connectivity services
with advanced characteristics, such as low latency guarantees, bounded with advanced characteristics, such as low-latency guarantees, bounded
jitter, or isolation from other services or customers so that changes in jitter, or isolation from other services or customers, so that changes in
other services (e.g., changes in network load, or events such as other services (e.g., changes in network load, or events such as
congestion or outages) have no effect or only acceptable effects on the congestion or outages) have no effect or only acceptable effects on the
observed throughput or latency of the services delivered to the observed throughput or latency of the services delivered to the
customer. These services are referred to as "enhanced VPNs", as they are customer. These services are referred to as "enhanced VPNs", as they are
similar to VPN services providing the customer with the required similar to VPN services, providing the customer with the required
connectivity, but in addition, they also provide enhanced connectivity, but they also provide enhanced characteristics.</t>
characteristics.</t>
<t>This document describes a framework for delivering VPN services with <t>This document describes a framework for delivering VPN services with
enhanced characteristics, such as guaranteed resources, latency, jitter, enhanced characteristics, such as guaranteed resources, latency, jitter,
etc. This list is not exhaustive. It is expected that other enhanced etc. This list is not exhaustive. It is expected that other enhanced
features may be added to VPN over time, and it is expected this features may be added to VPN over time and that this
framework will support these additions with necessary changes or framework will support these additions with necessary changes or
enhancements in some network layers and network planes (data plane, enhancements in some network layers and network planes (data plane,
control plane, and management plane).</t> control plane, and management plane).</t>
<t>The concept of network slicing has gained traction, driven largely by
<t>The concept of network slicing has gained traction driven largely by needs surfacing from 5G (see <xref target="NGMN-NS-Concept" format="defaul
needs surfacing from 5G <xref target="NGMN-NS-Concept"/> <xref t"/>, <xref target="TS23501" format="default"/>, and <xref target="TS28530" form
target="TS23501"/> <xref target="TS28530"/>. According to <xref at="default"/>). According to <xref target="TS28530" format="default"/>, a 5G en
target="TS28530"/>, a 5G end-to-end network slice consists of three d-to-end network slice consists of three
major types of network segments: Radio Access Network (RAN), Transport major types of network segments: Radio Access Network (RAN), Transport
Network (TN), and Mobile Core Network (CN). The transport network Network (TN), and mobile Core Network (CN). The transport network
provides the connectivity between different entities in RAN and CN provides the connectivity between different entities in RAN and CN
segments of a 5G end-to-end network slice, with specific performance segments of a 5G end-to-end network slice, with specific performance
commitments.</t> commitments.</t>
<t><xref target="RFC9543" format="default"/> discusses the general framewo
<t><xref target="RFC9543"/> discusses the general framework, components, rk, components,
and interfaces for requesting and operating network slices using IETF and interfaces for requesting and operating network slices using IETF
technologies. These network slices may be referred to as RFC 9543 technologies. These network slices may be referred to as "RFC 9543
Network Slices, but in this document (which is solely about IETF Network Slices", but in this document (which is solely about IETF
technologies) we simply use the term "network slice" to refer to this technologies), we simply use the term "network slice" to refer to this
concept. A network slice service enables connectivity between a set of concept. A network slice service enables connectivity between a set of
Service Demarcation Points (SDPs) with specific Service Level Objectives Service Demarcation Points (SDPs) with specific Service Level Objectives
(SLOs) and Service Level Expectations (SLEs) over a common underlay (SLOs) and Service Level Expectations (SLEs) over a common underlay
network. A network slice can be realized as a logical network connecting network. A network slice can be realized as a logical network connecting
a number of endpoints and is associated with a set of shared or a number of endpoints and is associated with a set of shared or
dedicated network resources that are used to satisfy the SLOs and SLEs dedicated network resources that are used to satisfy the SLO and SLE
requirements. A network slice is considered as one target use case of requirements. A network slice is considered to be one target use case of
enhanced VPNs.</t> enhanced VPNs.</t>
<t><xref target="RFC9543" format="default"/> also introduces the concept o
<t><xref target="RFC9543"/> also introduces the concept of Network f NRP, which is a subset of the
Resource Partition (NRP), which is a subset of the
buffer/queuing/scheduling resources and associated policies on each of a buffer/queuing/scheduling resources and associated policies on each of a
connected set of links in the underlay network. An NRP can be associated connected set of links in the underlay network. An NRP can be associated
with a dedicated or shared network topology to select or specify the set with a dedicated or shared network topology to select or specify the set
of links and nodes involved.</t> of links and nodes involved.</t>
<t>The requirements of enhanced VPN services cannot simply be met by <t>The requirements of enhanced VPN services cannot simply be met by
overlay networks, as enhanced VPN services require tighter coordination overlay networks: enhanced VPN services require tighter coordination
and integration between the overlay and the underlay networks.</t> and integration between the overlay and the underlay networks.</t>
<t>In the overlay network, the VPN has been defined as the network <t>In the overlay network, the VPN has been defined as the network
construct to provide the required connectivity for different services or construct to provide the required connectivity for different services or
customers. Multiple VPN flavors can be considered to create that customers. Multiple VPN flavors can be considered to create that
construct <xref target="RFC4026"/>. In the underlay network, the NRP is construct <xref target="RFC4026" format="default"/>. In the underlay netwo rk, the NRP is
used to represent a subset of the network resources and associated used to represent a subset of the network resources and associated
policies in the underlay network. An NRP can be associated with a policies in the underlay network. An NRP can be associated with a
dedicated or shared network topology to select or specify the set of dedicated or shared network topology to select or specify the set of
links and nodes involved.</t> links and nodes involved.</t>
<t>An enhanced VPN service can be realized by integrating a VPN in the <t>An enhanced VPN service can be realized by integrating a VPN in the
overlay and an NRP in the underlay. This is called an NRP-based enhanced overlay and an NRP in the underlay. This is called an "NRP-based enhanced
VPN. In doing so, an enhanced VPN service can provide enhanced VPN". In doing so, an enhanced VPN service can provide enhanced
properties, such as guaranteed resources and assured or predictable properties, such as guaranteed resources and assured or predictable
performance. An enhanced VPN service may also involve a set of service performance. An enhanced VPN service may also involve a set of service
functions (see Section 1.4 of <xref target="RFC7665"/> for the functions (see <xref target="RFC7665" sectionFormat="of" section="1.4"/> f or the
definition of service function). The techniques for delivering an definition of service function). The techniques for delivering an
NRP-based enhanced VPN can be used to instantiate a network slice NRP-based enhanced VPN can be used to instantiate a network slice
service (as described in <xref target="app-ns-realization"/>), and they service (as described in <xref target="app-ns-realization" format="default "/>), and they
can also be of use in general cases to provide enhanced connectivity can also be of use in general cases to provide enhanced connectivity
services between customer sites or service endpoints.</t> services between customer sites or service endpoints.</t>
<t>This document describes a framework for using existing, modified, and <t>This document describes a framework for using existing, modified, and
potential new technologies as components to provide NRP-based enhanced potential new technologies as components to provide NRP-based enhanced
VPN services. Specifically, this document provides:</t> VPN services. Specifically, this document provides:</t>
<ul spacing="normal">
<t><list style="symbols"> <li>
<t>The functional requirements and service characteristics of an <t>The functional requirements and service characteristics of an
enhanced VPN service.</t> enhanced VPN service.</t>
</li>
<li>
<t>The design of the data plane for NRP-based enhanced VPNs.</t> <t>The design of the data plane for NRP-based enhanced VPNs.</t>
</li>
<li>
<t>The necessary control and management protocols in both the <t>The necessary control and management protocols in both the
underlay and the overlay of enhanced VPNs.</t> underlay and the overlay of enhanced VPNs.</t>
</li>
<li>
<t>The mechanisms to achieve integration between the overlay network <t>The mechanisms to achieve integration between the overlay network
and the underlay network.</t> and the underlay network.</t>
</li>
<t>The necessary Operation, Administration, and Management (OAM) <li>
<t>The necessary Operation, Administration, and Maintenance (OAM)
methods to instrument an enhanced VPN to make sure that the required methods to instrument an enhanced VPN to make sure that the required
Service Level Agreement (SLA) between the customer and the network Service Level Agreement (SLA) between the customer and the network
operator is met, and to take any corrective action (such as operator is met and to take any corrective action (such as
switching traffic to an alternate path) to avoid SLA violation.</t> switching traffic to an alternate path) to avoid SLA violation.</t>
</list></t> </li>
</ul>
<t>One possible layered network structure to achieve these objectives is <t>One possible layered network structure to achieve these objectives is
shown in <xref target="COMLAY"/>.</t> shown in <xref target="COMLAY" format="default"/>.</t>
<t>It is not envisaged that enhanced VPN services will replace <t>It is not envisaged that enhanced VPN services will replace
conventional VPN services. VPN services will continue to be delivered conventional VPN services. VPN services will continue to be delivered
using existing mechanisms and can co-exist with enhanced VPN services. using existing mechanisms and can coexist with enhanced VPN services.
Whether enhanced VPN features are added to an active VPN service is Whether enhanced VPN features are added to an active VPN service is
deployment-specific.</t> deployment specific.</t>
</section> </section>
<section numbered="true" toc="default">
<section title="Terminology"> <name>Terminology</name>
<t>In this document, the relationship of the four terms "VPN", "enhanced <t>In this document, the relationship of the four terms "VPN", "enhanced
VPN", "NRP", and "Network Slice" are as follows:</t> VPN", "NRP", and "Network Slice" are as follows:</t>
<ul spacing="normal">
<li>
<t><list style="symbols">
<t>A Virtual Private Network (VPN) refers to the overlay network <t>A Virtual Private Network (VPN) refers to the overlay network
service that provides connectivity between different customer sites, service that provides connectivity between different customer sites
and that maintains traffic separation between different customers. and that maintains traffic separation between different customers.
Examples of technologies to provide VPN services are: IPVPN <xref Examples of technologies to provide VPN services are as follows: IP VP
target="RFC2764"/>, L2VPN <xref target="RFC4664"/>, L3VPN <xref N <xref target="RFC2764" format="default"/> <xref
target="RFC4364"/>, and EVPN <xref target="RFC7432"/>.</t> target="RFC4364" format="default"/>, L2VPN <xref target="RFC4664" format="defaul
t"/>, and EVPN <xref target="RFC7432" format="default"/>.</t>
</li>
<li>
<t>An enhanced VPN service is an evolution of the VPN service that <t>An enhanced VPN service is an evolution of the VPN service that
makes additional service-specific commitments. An NRP-based enhanced makes additional service-specific commitments. An NRP-based enhanced
VPN is made by integrating a VPN with a set of network resources VPN is made by integrating a VPN with a set of network resources
allocated in the underlay network (i.e. an NRP).</t> allocated in the underlay network (i.e., an NRP).</t>
</li>
<t>A Network Resource Partition (NRP) as defined in <xref <li>
target="RFC9543"/> is a subset of the buffer/queuing/scheduling <t>An NRP, as defined in <xref target="RFC9543" format="default"/>, is
a subset of the buffer/queuing/scheduling
resources and associated policies on each of a connected set of resources and associated policies on each of a connected set of
links in the underlay network. An NRP can be associated with a links in the underlay network. An NRP can be associated with a
dedicated or shared network topology to select or specify the set of dedicated or shared network topology to select or specify the set of
links and nodes involved. An NRP is designed to meet the network links and nodes involved. An NRP is designed to meet the network
resources and performance characteristics required by the enhanced resources and performance characteristics required by the enhanced
VPN services.</t> VPN services.</t>
</li>
<li>
<t>A network slice service could be delivered by provisioning one or <t>A network slice service could be delivered by provisioning one or
more NRP-based enhanced VPNs in the network. Other mechanisms for more NRP-based enhanced VPNs in the network. Other mechanisms for
realizing network slices may exist but are not in the scope of this realizing network slices may exist but are not in the scope of this
document.</t> document.</t>
</list></t> </li>
</ul>
<t>The term "tenant" is used in this document to refer to a customer of <t>The term "tenant" is used in this document to refer to a customer of
the enhanced VPN services.</t> the enhanced VPN services.</t>
<t>The following terms, defined in other documents, are also used in <t>The following terms, defined in other documents, are also used in
this document. <list style="hanging"> this document. </t>
<t hangText="SLA:">Service Level Agreement. See <xref <dl newline="false" spacing="normal">
target="RFC9543"/>.</t> <dt>SLA:</dt>
<dd>Service Level Agreement (see <xref target="RFC9543" format="default"
<t hangText="SLO:">Service Level Objective. See <xref />)</dd>
target="RFC9543"/>.</t> <dt>SLO:</dt>
<dd>Service Level Objective (see <xref target="RFC9543" format="default"
<t hangText="SLE:">Service Level Expectation. See <xref />)</dd>
target="RFC9543"/>.</t> <dt>SLE:</dt>
<dd>Service Level Expectation (see <xref target="RFC9543" format="defaul
<t hangText="ACTN:">Abstraction and Control of Traffic Engineered t"/>)</dd>
Networks <xref target="RFC8453"/>.</t> <dt>ACTN:</dt>
<dd>Abstraction and Control of TE Networks (see <xref target="RFC8453" f
<t hangText="DetNet:">Deterministic Networking. See <xref ormat="default"/>)</dd>
target="RFC8655"/>.</t> <dt>DetNet:</dt>
<dd>Deterministic Networking (see <xref target="RFC8655" format="default
<t hangText="FlexE:">Flexible Ethernet <xref target="FLEXE"/>.</t> "/>)</dd>
<dt>FlexE:</dt>
<t hangText="TSN:">Time Sensitive Networking <xref <dd>Flex Ethernet (see <xref target="FLEXE" format="default"/>)</dd>
target="TSN"/>.</t> <dt>TSN:</dt>
<dd>Time-Sensitive Networking (see <xref target="TSN" format="default"/>
<t hangText="VN:">Virtual Network. See <xref target="RFC8453"/>.</t> )</dd>
</list></t> <dt>VN:</dt>
<dd>Virtual Network (see <xref target="RFC8453" format="default"/>)</dd>
</dl>
</section> </section>
<section anchor="overview-of-the-requirements" numbered="true" toc="default"
<section anchor="overview-of-the-requirements" >
title="Overview of the Requirements"> <name>Overview of the Requirements</name>
<t>This section provides an overview of the requirements of an enhanced <t>This section provides an overview of the requirements of an enhanced
VPN service.</t> VPN service.</t>
<section anchor="diverse-performance-guarantees" numbered="true" toc="defa
<section anchor="diverse-performance-guarantees" ult">
title="Performance Guarantees"> <name>Performance Guarantees</name>
<t>Performance guarantees are committed by network operators to their <t>Performance guarantees are committed by network operators to their
customers in relation to the services delivered to the customers. They customers in relation to the services delivered to the customers. They
are usually expressed in SLAs as a set of SLOs.</t> are usually expressed in SLAs as a set of SLOs.</t>
<t>There are several kinds of performance guarantees, including <t>There are several kinds of performance guarantees, including
guaranteed maximum packet loss, guaranteed maximum delay, and guaranteed maximum packet loss, guaranteed maximum delay, and
guaranteed delay variation. Note that these guarantees apply to guaranteed delay variation. Note that these guarantees apply to
conformance traffic; out-of-profile traffic will be handled according conformance traffic; out-of-profile traffic will be handled according
to a separate agreement with the customer (see, for example, Section to a separate agreement with the customer (see, for example,
3.6 of <xref target="RFC7297"/>).</t> <xref target="RFC7297" sectionFormat="of" section="3.6"/>).</t>
<t>Guaranteed maximum packet loss is usually addressed by setting <t>Guaranteed maximum packet loss is usually addressed by setting
packet priorities, queue sizes, and discard policies. However, this packet priorities, queue sizes, and discard policies. However, this
becomes more difficult when the requirement is combined with latency becomes more difficult when the requirement is combined with latency
requirements. The limiting case is zero congestion loss, and that is requirements. The limiting case is zero congestion loss, and that is
the goal of Deterministic Networking (DetNet) <xref target="RFC8655"/> the goal of DetNet <xref target="RFC8655" format="default"/>
and Time-Sensitive Networking (TSN) <xref target="TSN"/>. In modern and Time-Sensitive Networking (TSN) <xref target="TSN" format="default"/
>. In modern
optical networks, loss due to transmission errors already approaches optical networks, loss due to transmission errors already approaches
zero, but there is the possibility of failure of the interface or the zero, but there is the possibility of failure of the interface or the
fiber itself. This type of fault can be addressed by some form of fiber itself. This type of fault can be addressed by some form of
signal duplication and transmission over diverse paths.</t> signal duplication and transmission over diverse paths.</t>
<t>Guaranteed maximum latency is required by a number of applications, <t>Guaranteed maximum latency is required by a number of applications,
particularly real-time control applications and some types of particularly real-time control applications and some types of
augmented reality and virtual reality (AR/VR) applications. DetNet augmented reality and virtual reality (AR/VR) applications. DetNet
techniques may be considered <xref target="RFC8655"/>, however techniques may be considered <xref target="RFC8655" format="default"/>; however,
additional methods of enhancing the underlay to better support the additional methods of enhancing the underlay to better support the
delay guarantees may be needed, and these methods will need to be delay guarantees may be needed. These methods will need to be
integrated with the overall service provisioning mechanisms.</t> integrated with the overall service provisioning mechanisms.</t>
<t>Guaranteed maximum delay variation is a performance guarantee that <t>Guaranteed maximum delay variation is a performance guarantee that
may also be needed. <xref target="RFC8578"/> calls up a number of may also be needed. <xref target="RFC8578" format="default"/> calls up a
cases that need this guarantee, for example in electrical utilities. number of
Time transfer is an example service that needs a performance cases that need this guarantee, for example, in electrical utilities.
Time transfer is an example service that needs this performance
guarantee, although it is in the nature of time that the service might guarantee, although it is in the nature of time that the service might
be delivered by the underlay as a shared service and not provided be delivered by the underlay as a shared service and not provided
through different enhanced VPNs. Alternatively, a dedicated enhanced through different enhanced VPNs. Alternatively, a dedicated enhanced
VPN might be used to provide time transfer as a shared service.</t> VPN might be used to provide time transfer as a shared service.</t>
<t>This suggests that a spectrum of service guarantees needs to be <t>This suggests that a spectrum of service guarantees needs to be
considered when designing and deploying an enhanced VPN. For considered when designing and deploying an enhanced VPN. For
illustration purposes and without claiming to be exhaustive, four illustration purposes and without claiming to be exhaustive, four
types of services are considered:</t> types of services are considered:</t>
<ul spacing="normal">
<t><list style="symbols"> <li>
<t>Best effort</t> <t>Best effort</t>
</li>
<li>
<t>Assured bandwidth</t> <t>Assured bandwidth</t>
</li>
<li>
<t>Guaranteed latency</t> <t>Guaranteed latency</t>
</li>
<li>
<t>Enhanced delivery</t> <t>Enhanced delivery</t>
</list></t> </li>
</ul>
<t>It is noted that some services may have mixed requirements from <t>It is noted that some services may have mixed requirements from
this list, e.g., both assured bandwidth and guaranteed latency can be this list, e.g., both assured bandwidth and guaranteed latency can be
required.</t> required.</t>
<t>The best-effort service is the basic connectivity service that can
<t>The best effort service is the basic connectivity service that can
be provided by current VPNs.</t> be provided by current VPNs.</t>
<t>An assured bandwidth service is a connectivity service in which the <t>An assured bandwidth service is a connectivity service in which the
bandwidth over some period of time is assured. This could be achieved bandwidth over some period of time is assured. This could be achieved
either simply based on a best effort service with over-capacity either simply based on a best-effort service with over-capacity
provisioning, or it can be based on MPLS traffic engineered label provisioning or based on MPLS TE Label
switching paths (TE-LSPs) with bandwidth reservations. Depending on Switching Paths (TE-LSPs) with bandwidth reservations. Depending on
the technique used, however, the bandwidth is not necessarily assured the technique used, however, the bandwidth is not necessarily assured
at any instant. Providing assured bandwidth to VPNs, for example by at any instant. Providing assured bandwidth to VPNs, for example, by
using per-VPN TE-LSPs, is not widely deployed at least partially due using per-VPN TE-LSPs, is not widely deployed at least partially due
to scalability concerns. The more common approach of aggregating to scalability concerns. The more common approach of aggregating
multiple VPNs onto common TE-LSPs results in shared bandwidth and so multiple VPNs onto common TE-LSPs results in shared bandwidth and so
may reduce the assurance of bandwidth to any one service. Enhanced may reduce the assurance of bandwidth to any one service. Enhanced
VPNs aim to provide a more scalable approach for such services.</t> VPNs aim to provide a more scalable approach for such services.</t>
<t>A guaranteed latency service has an upper bound to edge-to-edge <t>A guaranteed latency service has an upper bound to edge-to-edge
latency. Assuring the upper bound is sometimes more important than latency. Assuring the upper bound is sometimes more important than
minimizing latency. There are several new technologies that provide minimizing latency. There are several new technologies that provide
some assistance with this performance guarantee. Firstly, the IEEE TSN some assistance with this performance guarantee:</t>
project <xref target="TSN"/> introduces the concept of scheduling of <ul>
delay- and loss-sensitive packets. FlexE <xref target="FLEXE"/> is <li>the IEEE TSN
also useful to help provide a guaranteed upper bound to latency. project <xref target="TSN" format="default"/> introduces the concept of
DetNet is also of relevance in assuring an upper bound of end-to-end scheduling of
packet latency in the network layer. The use of these technologies to delay-sensitive and loss-sensitive packets.</li>
<li>FlexE <xref target="FLEXE" format="default"/> is
useful to help provide a guaranteed upper bound to latency.</li>
<li>DetNet is of relevance in assuring an upper bound of end-to-end
packet latency in the network layer.</li>
</ul>
<t>The use of these technologies to
deliver enhanced VPN services needs to be considered when a guaranteed deliver enhanced VPN services needs to be considered when a guaranteed
latency service is required.</t> latency service is required.</t>
<t>An enhanced delivery service is a connectivity service in which the <t>An enhanced delivery service is a connectivity service in which the
underlay network (at Layer 3) needs to ensure to eliminate or minimize underlay network (at Layer 3) needs to ensure to eliminate or minimize
packet loss in the event of equipment or media failures. This may be packet loss in the event of equipment or media failures. This may be
achieved by delivering a copy of the packet through multiple paths. achieved by delivering a copy of the packet through multiple paths.
Such a mechanism may need to be used for enhanced VPN services.</t> Such a mechanism may need to be used for enhanced VPN services.</t>
</section> </section>
<section anchor="interaction-between-vpn-services" numbered="true" toc="de
<section anchor="interaction-between-vpn-services" fault">
title="Interaction between Enhanced VPN Services"> <name>Interaction Between Enhanced VPN Services</name>
<t>There is a fine distinction between how a customer requests limits <t>There is a fine distinction between how a customer requests limits
on interaction between an enhanced VPN service and other services on interaction between an enhanced VPN service and other services
(whether they are other enhanced VPN services or any other network (whether they are other enhanced VPN services or any other network
service), and how that is delivered by the service provider. This service) and how that is delivered by the service provider. This
section examines the requirements and realization of limited section examines the requirements and realization of limited
interaction between an enhanced VPN service and other services.</t> interaction between an enhanced VPN service and other services.</t>
<section anchor="requirements-on-traffic-isolation" numbered="true" toc=
<section anchor="requirements-on-traffic-isolation" "default">
title="Requirements on Traffic Isolation"> <name>Requirements on Traffic Isolation</name>
<t>Traffic isolation is a generic term that can be used to describe <t>"Traffic isolation" is a generic term that can be used to describe
the requirements for separating the services of different customers the requirements for separating the services of different customers
or different service types in the network. In the context of network or different service types in the network. In the context of network
slicing, traffic isolation is defined as an SLE of the network slice slicing, traffic isolation is defined as an SLE of the network slice
service (Section 8.1 of <xref target="RFC9543"/>), which is one service (<xref target="RFC9543" sectionFormat="of" section="8.1"/>), w hich is one
element of the SLA. A customer may care about disruption caused by element of the SLA. A customer may care about disruption caused by
other services, contamination by other traffic, or delivery of their other services, contamination by other traffic, or delivery of their
traffic to the wrong destinations.</t> traffic to the wrong destinations.</t>
<t>A customer may want to specify (and thus pay for) the traffic <t>A customer may want to specify (and thus pay for) the traffic
isolation provided by the service provider. Some customers (banking, isolation provided by the service provider. Some customers (banking,
for example) may have strict requirements on how their flows are for example) may have strict requirements on how their flows are
handled when delivered over a shared network. Some professional handled when delivered over a shared network. Some professional
services are used to relying on specific certifications and audits services are used to relying on specific certifications and audits
to ensure the compliancy of a network with traffic isolation to ensure the compliancy of a network with traffic-isolation
requirements, and specifically to prevent data leaks.</t> requirements and, specifically, to prevent data leaks.</t>
<t>With traffic isolation, a customer expects that the service <t>With traffic isolation, a customer expects that the service
traffic cannot be received by other customers in the same network. traffic cannot be received by other customers in the same network.
In <xref target="RFC4176"/>, traffic isolation is mentioned as one In <xref target="RFC4176" format="default"/>, traffic isolation is men tioned as one
of the requirements of VPN customers. Traffic isolation is also of the requirements of VPN customers. Traffic isolation is also
described in Section 3.8 of <xref target="RFC7297"/>.</t> described in <xref target="RFC7297" sectionFormat="of" section="3.8"/>
.</t>
<t>There can be different expectations of traffic isolation. For <t>There can be different expectations of traffic isolation. For
example, a customer may further request the protection of their example, a customer may further request the protection of their
traffic by requesting specific encryption schemes at the enhanced traffic by requesting specific encryption schemes at the enhanced
VPN network access and also when transported between Provider Edge VPN access and also when transported between Provider Edge
(PE) Nodes.</t> (PE) nodes.</t>
<t>An enhanced VPN service customer may request traffic isolation <t>An enhanced VPN service customer may request traffic isolation
together with other operator defined service characteristics. The together with other operator-defined service characteristics. The
exact details about the expected behavior need to be specified in exact details about the expected behavior need to be specified in
the service request, so that meaningful service assurance and the service request so that meaningful service assurance and
fulfillment feedback can be exposed to the customers. It is out of fulfillment feedback can be exposed to the customer. It is out of
the scope of this document to elaborate the service modeling the scope of this document to elaborate the service-modeling
considerations.</t> considerations.</t>
</section> </section>
<section anchor="limited-interaction-with-other-services" numbered="true
<section anchor="limited-interaction-with-other-services" " toc="default">
title="Limited Interaction with Other Services"> <name>Limited Interaction with Other Services</name>
<t><xref target="RFC2211"/> describes the Controlled Load Service. <t><xref target="RFC2211" format="default"/> describes the controlled-
load service.
In that document, the end-to-end behavior provided to an application In that document, the end-to-end behavior provided to an application
by a series of network elements providing controlled-load service is by a series of network elements providing controlled-load service is
described as closely approximating to the behavior visible to described as closely approximating to the behavior visible to
applications receiving best-effort service when those network applications receiving best-effort service when those network
elements are not carrying substantial traffic from other elements are not carrying substantial traffic from other
services.</t> services.</t>
<t>Thus, a consumer of a controlled-load service may assume
<t>Thus, a consumer of a Controlled Load Service may assume
that:</t> that:</t>
<ul spacing="normal">
<t><list style="symbols"> <li>
<t>A very high percentage of transmitted packets will be <t>A very high percentage of transmitted packets will be
successfully delivered by the network to the receiving successfully delivered by the network to the receiving
end-nodes.</t> end nodes.</t>
</li>
<li>
<t>The transit delay experienced by a very high percentage of <t>The transit delay experienced by a very high percentage of
the delivered packets will not greatly exceed the minimum the delivered packets will not greatly exceed the minimum
transmit delay experienced by any successfully delivered transmit delay experienced by any successfully delivered
packet.</t> packet.</t>
</list></t> </li>
</ul>
<t>An enhanced VPN customer may request a Controlled Load Service in <t>An enhanced VPN customer may request a controlled-load service in
one of two ways:</t> one of two ways:</t>
<ol spacing="normal" type="1"><li>
<t><list style="numbers">
<t>It may configure a set of SLOs (for example, for delay and <t>It may configure a set of SLOs (for example, for delay and
loss) such that the delivered enhanced VPN meets the behavioral loss) such that the delivered enhanced VPN meets the behavioral
objectives of the customer.</t> objectives of the customer.</t>
</li>
<t>As described in <xref target="RFC2211"/>, a customer may <li>
request the Controlled Load Service without reference to or <t>As described in <xref target="RFC2211" format="default"/>, a cu
stomer may
request the controlled-load service without reference to or
specification of specific target values for control parameters specification of specific target values for control parameters
such as delay or loss. Instead, acceptance of a request for such as delay or loss. Instead, acceptance of a request for
Controlled Load Service is defined to imply a commitment by the controlled-load service is defined to imply a commitment by the
network element to provide the requestor with service closely network element to provide the requestor with service closely
equivalent to that provided to uncontrolled (best-effort) equivalent to that provided to uncontrolled (best-effort)
traffic under lightly loaded conditions. This way of requesting traffic under lightly loaded conditions. This way of requesting
the service is an SLE.</t> the service is an SLE.</t>
</list></t> </li>
</ol>
<t>Limited interaction between enhanced VPN services does not cover <t>Limited interaction between enhanced VPN services does not cover
service degradation due to non-interaction-related causes, such as service degradation due to non-interaction-related causes, such as
link errors.</t> link errors.</t>
</section> </section>
<section anchor="realization-of-limited-interaction-between-vpn-services
<section anchor="realization-of-limited-interaction-between-vpn-services " numbered="true" toc="default">
" <name>Realization of Limited Interaction with Enhanced VPN Services</n
title="Realization of Limited Interaction with Enhanced VPN Ser ame>
vices">
<t>A service provider may translate the requirements related to <t>A service provider may translate the requirements related to
limited interaction into distinct engineering rules in its network. limited interaction into distinct engineering rules in its network.
Honoring the service requirement may involve tweaking a set of QoS, Honoring the service requirement may involve tweaking a set of QoS,
TE, security, and planning tools, while traffic isolation will TE, security, and planning tools, while traffic isolation will
involve adequately configuring routing and authorization involve adequately configuring routing and authorization
capabilities.</t> capabilities.</t>
<t>Concretely, there are many existing techniques that can be used
<t>Concretely, there are many existing techniques which can be used
to provide traffic isolation, such as IP and MPLS VPNs or other to provide traffic isolation, such as IP and MPLS VPNs or other
multi- tenant virtual network techniques. Controlled Load Services multi-tenant virtual network techniques. Controlled-load services
may be realized as described in <xref target="RFC2211"/>. Other may be realized as described in <xref target="RFC2211" format="default
"/>. Other
tools may include various forms of resource management and tools may include various forms of resource management and
reservation techniques, such as network capacity planning, reservation techniques, such as network capacity planning,
allocating dedicated network resources, traffic policing or shaping, allocating dedicated network resources, traffic policing or shaping,
prioritizing in using shared network resources etc., so that a prioritizing in using shared network resources, etc., so that a
subset of bandwidth, buffers, and queueing resources can be subset of bandwidth, buffers, and queueing resources can be
available in the underlay network to support the enhanced VPN available in the underlay network to support the enhanced VPN
services.</t> services.</t>
<t>To provide the required traffic isolation, or to reduce the <t>To provide the required traffic isolation, or to reduce the
interaction with other enhanced VPN services, network resources may interaction with other enhanced VPN services, network resources may
need to be reserved in the data plane of the underlay network and need to be reserved in the data plane of the underlay network and
dedicated to traffic from a specific enhanced VPN service or a dedicated to traffic from a specific enhanced VPN service or a
specific group of enhanced VPN services. This may introduce specific group of enhanced VPN services.
scalability concerns both in the implementation (as each enhanced
VPN may need to be tracked in the network) and in how many resources This may introduce scalability concerns in the implementation, as each
need to be reserved and how the services are mapped to the resources enhanced VPN may need to be tracked in the network. It may also introduce
(Section 4.4). Thus, some trade-off needs to be considered to scalability concerns in deployment, such as how many resources need to be
provide the traffic isolation and limited interaction between an reserved and how the services are mapped to the resources (<xref
enhanced VPN services and other services.</t> target="scalable-mapping"/>). Thus, some trade-off needs to be considered to
provide the traffic isolation and limited interaction between an enhanced VPN
service and other services.</t>
<t>A dedicated physical network can be used to meet stricter SLO and <t>A dedicated physical network can be used to meet stricter SLO and
SLE requests, at the cost of allocating resources on a long-term and SLE requests, at the cost of allocating resources on a long-term and
end- to-end basis. On the other hand, where adequate traffic end- to-end basis. On the other hand, where adequate traffic
isolation and limited interaction can be achieved at the packet isolation and limited interaction can be achieved at the packet
layer, this permits the resources to be shared amongst a group of layer, this permits the resources to be shared amongst a group of
services and only dedicated to a service on a temporary basis. By services and only dedicated to a service on a temporary basis. By
combining conventional VPNs with TE/QoS/security techniques, an combining conventional VPNs with TE, QoS, and security techniques,
enhanced VPN offers a variety of means to honor customer's an enhanced VPN offers a variety of means to honor customer's
requirements.</t> requirements.</t>
</section> </section>
</section> </section>
<section anchor="integration" numbered="true" toc="default">
<name>Integration with Network Resources and Service Functions</name>
<section anchor="integration" <t>The way to meet the enhanced VPN service's demand for certain
title="Integration with Network Resources and Service Functions"> characteristics (such as guaranteed or predictable performance) is by
<t>The way to achieve the characteristics demand of an enhanced VPN
service (such as guaranteed or predictable performance) is by
integrating the overlay VPN with a particular set of resources in the integrating the overlay VPN with a particular set of resources in the
underlay network which are allocated to meet the service requirements. underlay network that are allocated to meet the service requirements.
This needs to be done in a flexible and scalable way so that it can be This needs to be done in a flexible and scalable way so that it can be
widely deployed in operators' networks to support a good number of widely deployed in operators' networks to support a good number of
enhanced VPN services.</t> enhanced VPN services.</t>
<t>Taking mobile networks and, in particular, 5G into consideration, the
<t>Taking mobile networks and in particular 5G into consideration, the
integration of the network with service functions is likely a integration of the network with service functions is likely a
requirement. The IETF's work on service function chaining (SFC) <xref requirement. The IETF's work on Service Function Chaining (SFC) <xref ta
target="RFC7665"/> provides a foundation for this. Service functions rget="RFC7665" format="default"/> provides a foundation for this. Service functi
in the underlay network can be considered as part of the enhanced VPN ons
in the underlay network can be considered to be part of the enhanced VPN
services, which means the service functions may need to be an integral services, which means the service functions may need to be an integral
part of the corresponding NRP. The details of the integration between part of the corresponding NRP. The details of the integration between
service functions and enhanced VPNs are out of the scope of this service functions and enhanced VPNs are out of the scope of this
document.</t> document.</t>
<section numbered="true" toc="default">
<section title="Abstraction"> <name>Abstraction</name>
<t>Integration of the overlay VPN and the underlay network resources <t>Integration of the overlay VPN and the underlay network resources
and service functions does not always need to be a direct mapping. and service functions does not always need to be a direct mapping.
As described in <xref target="RFC7926"/>, abstraction is the process As described in <xref target="RFC7926" format="default"/>, abstraction
of applying policy to a set of information about a traffic is the process
engineered (TE) network to produce selective information that of applying policy to a set of information about a traffic-engineered
network to produce selective information that
represents the potential ability to connect across the network. The represents the potential ability to connect across the network. The
process of abstraction presents the connectivity graph in a way that process of abstraction presents the connectivity graph in a way that
is independent of the underlying network technologies, capabilities, is independent of the underlying network technologies, capabilities,
and topology so that the graph can be used to plan and deliver and topology so that the graph can be used to plan and deliver
network services in a uniform way.</t> network services in a uniform way.</t>
<t>With the approach of abstraction, an enhanced VPN may be built on <t>Using the abstraction approach, an enhanced VPN may be built on
top of an abstracted topology that represents the connectivity top of an abstracted topology that represents the connectivity
capabilities of the underlay TE based network as described in the capabilities of the underlay TE-based network as described in the
framework for Abstraction and Control of TE Networks (ACTN) <xref framework for Abstraction and Control of TE Networks (ACTN) <xref targ
target="RFC8453"/> as discussed further in <xref et="RFC8453" format="default"/> as discussed further in <xref target="management
target="management-plane"/>.</t> -plane" format="default"/>.</t>
</section> </section>
</section> </section>
<section anchor="dynamic-configuration" numbered="true" toc="default">
<section anchor="dynamic-configuration" title="Dynamic Changes"> <name>Dynamic Changes</name>
<t>Enhanced VPNs need to be created, modified, and removed from the <t>Enhanced VPNs need to be created, modified, and removed from the
network according to service demands (including scheduled requests). network according to service demands (including scheduled requests).
An enhanced VPN that requires limited interaction with other services An enhanced VPN that requires limited interaction with other services
(<xref target="limited-interaction-with-other-services"/>) must not be (<xref target="limited-interaction-with-other-services" format="default" />) must not be
disrupted by the instantiation or modification of another enhanced VPN disrupted by the instantiation or modification of another enhanced VPN
service. As discussed in Section 3.1 of <xref target="RFC4176"/>, the service. As discussed in <xref target="RFC4176" sectionFormat="of" secti on="3.1"/>, the
assessment of traffic isolation is part of the management of a VPN assessment of traffic isolation is part of the management of a VPN
service. Determining whether modification of an enhanced VPN can be service. Determining whether modification of an enhanced VPN can be
disruptive to that enhanced VPN and whether the traffic in flight will disruptive to that enhanced VPN and whether the traffic in flight will
be disrupted can be a difficult problem.</t> be disrupted can be a difficult problem.</t>
<t>Dynamic changes both to the enhanced VPN and to the underlay <t>Dynamic changes both to the enhanced VPN and to the underlay
network need to be managed to avoid disruption to services that are network need to be managed to avoid disruption to services that are
sensitive to changes in network performance.</t> sensitive to changes in network performance.</t>
<t>In addition to non-disruptively managing the network during changes <t>In addition to managing the network without disruption during changes
such as the inclusion of a new enhanced VPN service endpoint or a such as the inclusion of a new enhanced VPN service endpoint or a
change to a link, enhanced VPN traffic might need to be moved because change to a link, enhanced VPN traffic might need to be moved because
of changes to traffic patterns and volumes. This means that during the of changes to traffic patterns and volume. This means that during the
lifetime of an enhanced VPN service, closed-loop optimization is lifetime of an enhanced VPN service, closed-loop optimization is
needed so that the delivered service always matches the ordered needed so that the delivered service always matches the ordered
service SLA.</t> SLA.</t>
<t>The data plane aspects of this problem are discussed further in <t>The data plane aspects of this problem are discussed further in
<xref target="L2-DP"/>, <xref target="NW-DP"> </xref>, and <xref Sections <xref target="L2-DP" format="counter"/>, <xref target="NW-DP" f
target="Non-Packet-DP"/>.</t> ormat="counter"> </xref>, and <xref target="Non-Packet-DP" format="counter"/>.</
t>
<t>The control plane aspects of this problem are discussed further in <t>The control plane aspects of this problem are discussed further in
<xref target="control-plane"/>.</t> <xref target="control-plane" format="default"/>.</t>
<t>The management plane aspects of this problem are discussed further <t>The management plane aspects of this problem are discussed further
in <xref target="management-plane"/>.</t> in <xref target="management-plane" format="default"/>.</t>
</section> </section>
<section anchor="customized-control-plane" numbered="true" toc="default">
<section anchor="customized-control-plane" title="Customized Control"> <name>Customized Control</name>
<t>In many cases enhanced VPN services are delivered to customers <t>In many cases enhanced VPN services are delivered to customers
without information about the underlying NRPs. However, depending on without information about the underlying NRPs. However, in some cases, d
the agreement between the operator and the customer, in some cases the epending on
the agreement between the operator and the customer, the
customer may also be provided with some information about the customer may also be provided with some information about the
underlying NRPs. Such information can be filtered or aggregated underlying NRPs. Such information can be filtered or aggregated
according to the operator's policy. This allows the customer of an according to the operator's policy. This allows the customer of an
enhanced VPN service to have some visibility and even control over how enhanced VPN service to have some visibility and even control over how
the underlying topology and resources of the NRP are used. For the underlying topology and resources of the NRP are used. For
example, the customers may be able to specify the path or path example, the customer may be able to specify the path or path
constraints within the NRP for specific traffic flows of their constraints within the NRP for specific traffic flows of their
enhanced VPN service. Depending on the requirements, an enhanced VPN enhanced VPN service. Depending on the requirements, an enhanced VPN
customer may have their own network controller, which may be provided customer may have their own network controller, which may be provided
with an interface to the control or management system run by the with an interface to the control or management system run by the
network operator. Note that such a control is within the scope of the network operator. Note that such a control is within the scope of the
customer's enhanced VPN service; any additional changes beyond this customer's enhanced VPN service; any additional changes beyond this
would require some intervention by the network operator.</t> would require some intervention by the network operator.</t>
<t>A description of the control plane aspects of this problem are <t>A description of the control plane aspects of this problem are
discussed further in <xref target="control-plane"/>. A description of discussed further in <xref target="control-plane" format="default"/>. A
the management plane aspects of this feature can be found in <xref description of
target="management-plane"/>.</t> the management plane aspects of this feature can be found in <xref targe
t="management-plane" format="default"/>.</t>
</section> </section>
<section anchor="applicability" numbered="true" toc="default">
<section anchor="applicability" <name>Applicability to Overlay Technologies</name>
title="Applicability to Overlay Technologies">
<t>The concept of an enhanced VPN can be applied to any existing and <t>The concept of an enhanced VPN can be applied to any existing and
future multi-tenancy overlay technologies including but not limited future multi-tenancy overlay technologies including but not limited
to:</t> to:</t>
<ul spacing="normal">
<t><list style="symbols"> <li>
<t>Layer-2 point-to-point services, such as pseudowires <xref <t>Layer 2 point-to-point (P2P) services, such as pseudowires (see <
target="RFC3985"/></t> xref target="RFC3985" format="default"/>)</t>
</li>
<t>Layer-2 VPNs <xref target="RFC4664"/></t> <li>
<t>Layer 2 VPNs (see <xref target="RFC4664" format="default"/>)</t>
<t>Ethernet VPNs <xref target="RFC7209"/>, <xref </li>
target="RFC7432"/></t> <li>
<t>Ethernet VPNs (see <xref target="RFC7209" format="default"/> and
<t>Layer-3 VPNs <xref target="RFC4364"/>, <xref <xref target="RFC7432" format="default"/>)</t>
target="RFC2764"/></t> </li>
</list></t> <li>
<t>Layer 3 VPNs (see <xref target="RFC4364" format="default"/> and <
xref target="RFC2764" format="default"/>)</t>
</li>
</ul>
<t>Where such VPN service types need enhanced isolation and delivery <t>Where such VPN service types need enhanced isolation and delivery
characteristics, the technologies described in <xref target="SDDC"/> characteristics, the technologies described in <xref target="SDDC" forma t="default"/>
can be used to tweak the underlay to provide the required enhanced can be used to tweak the underlay to provide the required enhanced
performance.</t> performance.</t>
</section> </section>
<section numbered="true" toc="default">
<section title="Inter-Domain and Inter-Layer Network"> <name>Inter-Domain and Inter-Layer Network</name>
<t>In some scenarios, an enhanced VPN service may span multiple <t>In some scenarios, an enhanced VPN service may span multiple
network domains. A domain is considered to be any collection of network domains. A domain is considered to be any collection of
network elements under the responsibility of the same administrative network elements under the responsibility of the same administrative
entity, for example, an Autonomous System (AS). In some domains, the entity, for example, an Autonomous System (AS). In some domains, the
network operator may manage a multi-layered network, for example, a network operator may manage a multi-layered network, for example, a
packet network over an optical network. When enhanced VPN services are packet network over an optical network. When enhanced VPN services are
provisioned in such network scenarios, the technologies used in provisioned in such network scenarios, the technologies used in
different network planes (data plane, control plane, and management different network planes (the data plane, control plane, and management
plane) need to provide mechanisms to support multi-domain and plane) need to provide mechanisms to support multi-domain and
multi-layer coordination and integration, so as to provide the multi-layer coordination and integration; this is to provide the
required service characteristics for different enhanced VPN services, required service characteristics for different enhanced VPN services
and improve network efficiency and operational simplicity. The and improve network efficiency and operational simplicity. The
mechanisms for multi-domain VPNs <xref target="RFC4364"/> may be mechanisms for multi-domain VPNs (see <xref target="RFC4364" format="def ault"/>) may be
reused, and some enhancement may be needed to meet the additional reused, and some enhancement may be needed to meet the additional
requirements of enhanced VPN services.</t> requirements of enhanced VPN services.</t>
</section> </section>
</section> </section>
<section anchor="architecture-and-components-of-vpn" numbered="true" toc="de
<section anchor="architecture-and-components-of-vpn" fault">
title="The Architecture of NRP-based Enhanced VPNs"> <name>The Architecture of NRP-Based Enhanced VPNs</name>
<t>Multiple NRP-based enhanced VPN services can be provided by a common <t>Multiple NRP-based enhanced VPN services can be provided by a common
network infrastructure. Each NRP-based enhanced VPN service is network infrastructure. Each NRP-based enhanced VPN service is
provisioned with an overlay VPN and mapped to a corresponding NRP, which provisioned with an overlay VPN and mapped to a corresponding NRP, which
has a specific set of network resources and service functions allocated has a specific set of network resources and service functions allocated
in the underlay to satisfy the needs of the customer. One NRP may in the underlay to satisfy the needs of the customer. One NRP may
support one or more NRP-based enhanced VPN services. The integration support one or more NRP-based enhanced VPN services. The integration
between the overlay connectivity and the underlay resources ensures the between the overlay connectivity and the underlay resources ensures the
required isolation between different enhanced VPN services, and achieves required isolation between different enhanced VPN services and achieves
the guaranteed performance for different customers.</t> the guaranteed performance for different customers.</t>
<t>The NRP-based enhanced VPN architecture needs to be designed with <t>The NRP-based enhanced VPN architecture needs to be designed with
consideration given to:</t> consideration given to:</t>
<ul spacing="normal">
<t><list style="symbols"> <li>
<t>An enhanced data plane.</t> <t>An enhanced data plane.</t>
</li>
<li>
<t>A control plane to create enhanced VPNs and NRPs, making use of <t>A control plane to create enhanced VPNs and NRPs, making use of
the data plane isolation and performance guarantee techniques.</t> the data plane isolation and performance guarantee techniques.</t>
</li>
<t>A management plane for enhanced VPN service life-cycle <li>
management.</t> <t>A management plane to manage enhanced VPN service life cycles.</t>
</li>
<li>
<t>The OAM mechanisms for enhanced VPNs and the underlying NRPs.</t> <t>The OAM mechanisms for enhanced VPNs and the underlying NRPs.</t>
</li>
<li>
<t>Telemetry mechanisms for enhanced VPNs and the underlying <t>Telemetry mechanisms for enhanced VPNs and the underlying
NRPs.</t> NRPs.</t>
</list> These topics are expanded below.</t> </li>
</ul>
<t><list style="symbols"> <t> These topics are expanded below.</t>
<t>The enhanced data plane provides:<list style="symbols"> <ul spacing="normal">
<t>The required packet latency and jitter characteristics.</t> <li>
<t>The enhanced data plane provides:</t>
<t>The required packet loss characteristics.</t> <ul spacing="normal">
<li>
<t>The required resource isolation capability, e.g., bandwidth <t>The required packet-latency and jitter characteristics.</t>
</li>
<li>
<t>The required packet-loss characteristics.</t>
</li>
<li>
<t>The required resource-isolation capability, e.g., bandwidth
guarantee.</t> guarantee.</t>
</li>
<li>
<t>The mechanism to associate a packet with the set of resources <t>The mechanism to associate a packet with the set of resources
allocated to an NRP which the enhanced VPN service packet is allocated to an NRP to which the enhanced VPN service packet is
mapped to.</t> mapped.</t>
</list></t> </li>
</ul>
<t>The control plane:<list style="symbols"> </li>
<li>
<t>The control plane:</t>
<ul spacing="normal">
<li>
<t>Collects information about the underlying network topology <t>Collects information about the underlying network topology
and network resources, and exports this to network nodes and/or and network resources and exports this to network nodes and/or
a centralized controller as required.</t> a centralized controller as required.</t>
</li>
<li>
<t>Creates NRPs with the network resource and topology <t>Creates NRPs with the network resource and topology
properties needed by the enhanced VPN services.</t> properties needed by the enhanced VPN services.</t>
</li>
<t>Distributes the attributes of NRPs to network nodes which <li>
<t>Distributes the attributes of NRPs to network nodes that
participate in the NRPs and/or a centralized controller.</t> participate in the NRPs and/or a centralized controller.</t>
</li>
<li>
<t>Computes and sets up network paths in each NRP.</t> <t>Computes and sets up network paths in each NRP.</t>
</li>
<li>
<t>Maps enhanced VPN services to an appropriate NRP.</t> <t>Maps enhanced VPN services to an appropriate NRP.</t>
</li>
<li>
<t>Determines the risk of SLA violation and takes appropriate <t>Determines the risk of SLA violation and takes appropriate
avoiding/correction actions.</t> avoidance/correction actions.</t>
</li>
<li>
<t>Considers the right balance of per-packet and per-node state <t>Considers the right balance of per-packet and per-node state
according to the needs of the enhanced VPN services to scale to according to the needs of the enhanced VPN services to scale to
the required size.</t> the required size.</t>
</list></t> </li>
</ul>
</li>
<li>
<t>The management plane includes management interfaces, the <t>The management plane includes management interfaces, the
Operations, Administration, and Maintenance (OAM) and Telemetry OAM and telemetry
mechanisms. More specifically, it provides:<list style="symbols"> mechanisms. More specifically, it provides:</t>
<ul spacing="normal">
<li>
<t>An interface between the enhanced VPN service provider (e.g., <t>An interface between the enhanced VPN service provider (e.g.,
operator's network management system) and the enhanced VPN the operator's network management system) and the enhanced VPN
customer (e.g., an organization or a service with enhanced VPN customer (e.g., an organization or service with an enhanced VPN
requirement) such that the operation requests and the related requirement) such that the requests for specific operations and th
e related
parameters can be exchanged without the awareness of other parameters can be exchanged without the awareness of other
enhanced VPN customers.</t> enhanced VPN customers.</t>
</li>
<li>
<t>An interface between the enhanced VPN service provider and <t>An interface between the enhanced VPN service provider and
the enhanced VPN customers to expose the network capability the enhanced VPN customers to expose the network capability
information toward the customer.</t> information toward the customer.</t>
</li>
<li>
<t>The service life-cycle management and operation of enhanced <t>The service life-cycle management and operation of enhanced
VPN services (e.g., creation, modification, VPN services (e.g., creation, modification,
assurance/monitoring, and decommissioning).</t> assurance/monitoring, and decommissioning).</t>
</li>
<li>
<t>The OAM tools to verify whether the underlay network <t>The OAM tools to verify whether the underlay network
resources (i.e. NRPs) are correctly allocated and operating resources (i.e., NRPs) are correctly allocated and operating
properly.</t> properly.</t>
</li>
<li>
<t>The OAM tools to verify the connectivity and monitor the <t>The OAM tools to verify the connectivity and monitor the
performance of the enhanced VPN service.</t> performance of the enhanced VPN service.</t>
</li>
<li>
<t>Telemetry of information in the underlay network for overall <t>Telemetry of information in the underlay network for overall
performance evaluation and the planning of the enhanced VPN performance evaluation and the planning of the enhanced VPN
services.</t> services.</t>
</li>
<li>
<t>Telemetry of information of enhanced VPN services for <t>Telemetry of information of enhanced VPN services for
monitoring and analytics of the characteristics and SLA monitoring and analytics of the characteristics and SLA
fulfillment of the enhanced VPN services.</t> fulfillment of the enhanced VPN services.</t>
</list></t> </li>
</list></t> </ul>
</li>
<section anchor="COMLAY" title="Layered Architecture"> </ul>
<section anchor="COMLAY" numbered="true" toc="default">
<name>Layered Architecture</name>
<t>The layered architecture of NRP-based enhanced VPNs is shown in <t>The layered architecture of NRP-based enhanced VPNs is shown in
<xref target="LAFIG"/>.</t> <xref target="LAFIG" format="default"/>.</t>
<t>Underpinning everything is the physical network infrastructure <t>Underpinning everything is the physical network infrastructure
layer which provides the underlying resources used to provision the layer, which provides the underlying resources used to provision the
separate NRPs. This layer is responsible for the partitioning of link separate NRPs. This layer is responsible for the partitioning of link
and/or node resources for different NRPs. Each subset of link or node and/or node resources for different NRPs. Each subset of a link or node
resource can be considered as a virtual link or virtual node used to resource can be considered to be a virtual link or virtual node used to
build the NRPs.</t> build the NRPs.</t>
<figure anchor="LAFIG">
<figure anchor="LAFIG" <name>The Layered Architecture of Enhanced VPNs</name>
title="The Layered Architecture of Enhanced VPNs"> <artwork align="center" name="" type="" alt=""><![CDATA[
<artwork align="center"><![CDATA[
/\ /\
|| ||
+-------------------+ Centralized +-------------------+ Centralized
| Network Controller| Control & Management | Network Controller| Control & Management
+-------------------+ +-------------------+
|| ||
\/ \/
o---------------------------o Enhanced VPN #1 o---------------------------o Enhanced VPN #1
/-------------o /-------------o
o____________/______________o Enhanced VPN #2 o____________/______________o Enhanced VPN #2
skipping to change at line 808 skipping to change at line 794
+--+===+--+===+--+===+--+===+--+ +--+===+--+===+--+===+--+===+--+
++++ ++++ ++++ ++++ ++++ ++++ ++++ ++++ ++++ ++++
o Virtual Node ++++ o Virtual Node ++++
+--+ Physical Node with resource partition +--+ Physical Node with resource partition
-- Virtual Link +--+ -- Virtual Link +--+
++++ ++++
== Physical Link with resource partition == Physical Link with resource partition
]]></artwork> ]]></artwork>
</figure> </figure>
<t>Various components and techniques discussed in <xref target="SDDC" fo
<t>Various components and techniques discussed in <xref rmat="default"/> can be used to enable resource partitioning of the
target="SDDC"/> can be used to enable resource partitioning of the
physical network infrastructure, such as FlexE, TSN, dedicated queues, physical network infrastructure, such as FlexE, TSN, dedicated queues,
etc. These partitions may be physical or virtual so long as the SLA etc. These partitions may be physical or virtual so long as the SLA
required by the higher layers is met.</t> required by the higher layers is met.</t>
<t>Based on the set of network resource partitions provided by the <t>Based on the set of NRPs provided by the
physical network infrastructure, multiple NRPs can be created, each physical network infrastructure, multiple NRPs can be created. Each of t
with a set of dedicated or shared network resources allocated from the hese NRPs:</t><ul>
physical underlay network, and each can be associated with a <li>has a set of dedicated or shared network resources allocated from the
customized logical network topology, so as to meet the requirements of physical underlay network,</li>
different enhanced VPN services or different groups of enhanced VPN <li>can be associated with a
services. According to the associated logical network topology, each customized logical network topology, and</li>
NRP needs to be instantiated on a set of network nodes and links which <li>meets the requirements of a specific enhanced VPN service or
are involved in the logical topology. And on each node or link, each a specific group of enhanced VPN services.</li>
NRP is associated with a set of local resources which are allocated </ul>
<t>According to the associated logical network topology, each
NRP needs to be instantiated on a set of network nodes and links that
are involved in the logical topology. On each node or link, each
NRP is associated with a set of local resources that are allocated
for the processing of traffic in the NRP. The NRP provides the for the processing of traffic in the NRP. The NRP provides the
integration between the logical network topology and the required integration between the logical network topology and the required
underlying network resources.</t> underlying network resources.</t>
<t>According to the service requirements of connectivity, performance <t>According to the service requirements of connectivity, performance,
and isolation, etc., enhanced VPN services can be mapped to the isolation, etc., enhanced VPN services can be mapped to the
appropriate NRPs in the network. Different enhanced VPN services can appropriate NRPs in the network. Different enhanced VPN services can
be mapped to different NRPs, while it is also possible that multiple be mapped to different NRPs; it is also possible that multiple
enhanced VPN services are mapped to the same NRP. Thus, the NRP is an enhanced VPN services are mapped to the same NRP. Thus, the NRP is an
essential scaling technique, as it has the potential of eliminating essential scaling technique as it has the potential of eliminating
per-service per-path state from the network. In addition, when a group per-service per-path state from the network. In addition, when a group
of enhanced VPN services are mapped to a single NRP, only the network of enhanced VPN services is mapped to a single NRP, only the network
state of the single NRP needs to be maintained in the network (see state of the single NRP needs to be maintained in the network (see
<xref target="scalable-mapping"/> for more information).</t> <xref target="scalable-mapping" format="default"/> for more information)
.</t>
<t>The network controller is responsible for creating an NRP, <t>The network controller is responsible for creating an NRP,
instructing the involved network nodes to allocate network resources instructing the involved network nodes to allocate network resources
to the NRP, and provisioning the enhanced VPN services on the NRP. A to the NRP, and provisioning the enhanced VPN services on the NRP. A
distributed control plane may be used for distributing the NRP distributed control plane may be used for distributing the NRP
resource and topology attributes among nodes in the NRP. Extensions to resource and topology attributes among nodes in the NRP. Extensions to
distributed control protocols (if any) are out of the scope of this distributed control protocols (if any) are out of the scope of this
document.</t> document.</t>
<t>The process used to create NRPs and to allocate network resources <t>The process used to create NRPs and to allocate network resources
for use by the NRPs needs to take a holistic view of the needs of all for use by the NRPs needs to take a holistic view of the needs of all
of the service provider's customers and to partition the resources of the service provider's customers and to partition the resources
accordingly. However, within an NRP these resources can, if required, accordingly. However, within an NRP, these resources can
be managed via a dynamic control plane. This provides the required be managed via a dynamic control plane if required. This provides the re
quired
scalability and isolation with some flexibility.</t> scalability and isolation with some flexibility.</t>
</section> </section>
<section anchor="multi-point-to-multi-point" numbered="true" toc="default"
<section anchor="multi-point-to-multi-point" title="Connectivity Types"> >
<t>At the VPN service level, the required connectivity for an MP2MP <name>Connectivity Types</name>
<t>At the VPN service level, the required connectivity for a Multipoint-
to-Multipoint (MP2MP)
VPN service is usually full or partial mesh. To support such VPN VPN service is usually full or partial mesh. To support such VPN
services, the corresponding NRP also needs to provide MP2MP services, the corresponding NRP also needs to provide MP2MP
connectivity among the end points.</t> connectivity among the endpoints.</t>
<t>Other service requirements may be expressed at different <t>Other service requirements may be expressed at different
granularities, some of which can be applicable to the whole service, granularities, some of which can be applicable to the whole service
while some others may only be applicable to some pairs of end points. while others may only be applicable to some pairs of endpoints.
For example, when a particular level of performance guarantee is For example, when a particular level of performance guarantee is
required, the point-to-point path through the underlying NRP of the required, the point-to-point path through the underlying NRP of the
enhanced VPN service may need to be specifically engineered to meet enhanced VPN service may need to be specifically engineered to meet
the required performance guarantee.</t> the required performance guarantee.</t>
</section> </section>
<section anchor="application-specific-network-types" numbered="true" toc="
<section anchor="application-specific-network-types" default">
title="Application-Specific Data Types"> <name>Application-Specific Data Types</name>
<t>Although a lot of the traffic that will be carried over enhanced <t>Although a lot of the traffic that will be carried over enhanced
VPN will likely be IP-based, the design must be capable of carrying VPN will likely be IP based, the design must be capable of carrying
other traffic types, in particular Ethernet traffic. This is easily other traffic types, in particular Ethernet traffic. This is easily
accomplished through the various pseudowire (PW) techniques <xref accomplished through various pseudowire (PW) techniques <xref target="RF
target="RFC3985"/>.</t> C3985" format="default"/>.</t>
<t>Where the underlay is MPLS, Ethernet traffic can be carried over an <t>Where the underlay is MPLS, Ethernet traffic can be carried over an
enhanced VPN encapsulated according to the method specified in <xref enhanced VPN encapsulated according to the method specified in <xref tar
target="RFC4448"/>. Where the underlay is IP, Layer Two Tunneling get="RFC4448" format="default"/>. Where the underlay is IP, L2 Tunneling
Protocol - Version 3 (L2TPv3) <xref target="RFC3931"/> can be used Protocol - Version 3 (L2TPv3) <xref target="RFC3931" format="default"/>
with Ethernet traffic carried according to <xref target="RFC4719"/>. can be used
Encapsulations have been defined for most of the common layer-2 types with Ethernet traffic carried according to <xref target="RFC4719" format
="default"/>.
Encapsulations have been defined for most of the common L2 types
for both PW over MPLS and for L2TPv3.</t> for both PW over MPLS and for L2TPv3.</t>
</section> </section>
<section anchor="scalable-mapping" numbered="true" toc="default">
<section anchor="scalable-mapping" title="Scalable Service Mapping"> <name>Scalable Service Mapping</name>
<t>VPNs are instantiated as overlays on top of an operator's network <t>VPNs are instantiated as overlays on top of an operator's network
and offered as services to the operator's customers. An important and offered as services to the operator's customers. An important
feature of overlays is that they can deliver services without placing feature of overlays is that they can deliver services without placing
per-service state in the core of the underlay network.</t> per-service state in the core of the underlay network.</t>
<t>An enhanced VPN may need to install some additional state within <t>An enhanced VPN may need to install some additional state within
the network to achieve the features that they require. Solutions need the network to achieve the features that they require. Solutions need
to take the scale of such state into consideration, and deployment to take the scale of such state into consideration, and deployment
architectures should constrain the number of enhanced VPN services so architectures should constrain the number of enhanced VPN services so
that the additional state introduced to the network is acceptable and that the additional state introduced to the network is acceptable and
under control. It is expected that the number of enhanced VPN services under control. It is expected that the number of enhanced VPN services
will be small at the beginning, and even in the future the number of will be small at the beginning: even in the future, the number of
enhanced VPN services will be fewer than conventional VPNs because enhanced VPN services will be fewer than conventional VPNs because
existing VPN techniques are good enough to meet the needs of most existing VPN techniques are good enough to meet the needs of most
existing VPN-type services.</t> existing VPN-type services.</t>
<t>In general, it is not required that the state in the network be <t>In general, it is not required that the state in the network be
maintained in a 1:1 relationship with the enhanced VPN services. It maintained in a 1:1 relationship with the enhanced VPN services. It
will usually be possible to aggregate a set or group of enhanced VPN will usually be possible to aggregate a set or group of enhanced VPN
services so that they share the same NRP and the same set of network services so that they share the same NRP and the same set of network
resources (much in the same way that current VPNs are aggregated over resources (much in the same way that current VPNs are aggregated over
transport tunnels) so that collections of enhanced VPN services that transport tunnels) so that collections of enhanced VPN services that
require the same behavior from the network in terms of resource require the same behavior from the network in terms of resource
reservation, latency bounds, resiliency, etc. can be grouped together. reservation, latency bounds, resiliency, etc. can be grouped together.
This is an important feature to assist with the scaling This is an important feature to assist with the scaling
characteristics of NRP-based enhanced VPN deployments.</t> characteristics of NRP-based enhanced VPN deployments.</t>
<t><xref target="I-D.ietf-teas-nrp-scalability" format="default"/> provi
<t><xref target="I-D.ietf-teas-nrp-scalability"/> provides more des more
details of scalability considerations for the NRPs used to instantiate details of scalability considerations for the NRPs used to instantiate
NRPs, and <xref target="scalability-considerations"/> includes a NRPs, and <xref target="scalability-considerations" format="default"/> i ncludes a
greater discussion of scalability considerations.</t> greater discussion of scalability considerations.</t>
</section> </section>
</section> </section>
<section anchor="SDDC" numbered="true" toc="default">
<name>Candidate Technologies</name>
<section anchor="SDDC" title="Candidate Technologies"> <t>A VPN is created by applying a demultiplexing technique to the
<t>A VPN is a virtual network created by applying a demultiplexing underlying network (the underlay) to distinguish the traffic of one VPN
technique to the underlying network (the underlay) to distinguish the from that of another. The connections of a VPN are supported by a set of
traffic of one VPN from that of another. The connections of a VPN are underlay paths. Any path other than the shortest path through the
supported by a set of underlay paths. A path that travels by other than underlay normally requires state to specify that path. The state of the
the shortest path through the underlay normally requires state to paths could be applied to the underlay through the use of the RSVP-TE
specify that path. The state of the paths could be applied to the signaling protocol or directly through the use of a Software-Defined
underlay through the use of the RSVP-TE signaling protocol, or directly Networking (SDN) controller. Based on Segment Routing (SR), state could
through the use of an SDN controller. Based on Segment Routing, state be maintained at the ingress node of the path and carried in the data
could be maintained at the ingress node of the path, and carried in the packet. Other techniques may emerge as this problem is studied. This
data packet. Other techniques may emerge as this problem is studied. state gets harder to manage as the number of paths increases.
This state gets harder to manage as the number of paths increases.
Furthermore, as we increase the coupling between the underlay and the Furthermore, as we increase the coupling between the underlay and the
overlay to support the enhanced VPN service, this state is likely to overlay to support the enhanced VPN service, this state is likely to
increase further. Through the use of NRP, a subset of underlay network increase further. Through the use of NRP, a subset of underlay network
resource can be either dedicated for a particular enhanced VPN service resources can be either dedicated for a particular enhanced VPN service
or shared among a group of enhanced VPN services. A group of underlay or shared among a group of enhanced VPN services. A group of underlay
paths can be established using the common set of network resources of paths can be established using the common set of network resources of
the NRP.</t> the NRP.</t>
<t>This section describes the candidate technologies and examines them
<t>This section describes the candidate technologies, and examines them
in the context of the different network planes that may be used together in the context of the different network planes that may be used together
to build NRPs. <xref target="L2-DP"/> discusses the layer-2 packet-based to build NRPs. <xref target="L2-DP" format="default"/> discusses the L2 pa
or frame-based forwarding plane mechanisms for resource partitioning. cket-based
<xref target="NW-DP"/> discusses the corresponding encapsulation and or frame-based forwarding-plane mechanisms for resource partitioning.
<xref target="NW-DP" format="default"/> discusses the corresponding encaps
ulation and
forwarding mechanisms in the network layer. Non-packet data plane forwarding mechanisms in the network layer. Non-packet data plane
mechanisms are briefly discussed in <xref target="Non-Packet-DP"/>. The mechanisms are briefly discussed in <xref target="Non-Packet-DP" format="d
control plane and management plane mechanisms are discussed in <xref efault"/>. The
target="control-plane"/> and <xref target="management-plane"/> control plane and management plane mechanisms are discussed in Sections <x
respectively.</t> ref target="control-plane" format="counter"/> and <xref target="management-plane
" format="counter"/>, respectively.</t>
<section anchor="L2-DP" <section anchor="L2-DP" numbered="true" toc="default">
title="Underlay Forwarding Resource Partitioning"> <name>Underlay Forwarding Resource Partitioning</name>
<t>Several candidate layer-2 packet-based or frame-based forwarding <t>Several candidate L2 packet-based or frame-based forwarding-plane mec
plane mechanisms which can provide the required traffic isolation and hanisms that can provide the required traffic isolation and
performance guarantees are described in the following sections.</t> performance guarantees are described in the following sections.</t>
<section anchor="flexe" numbered="true" toc="default">
<name>Flex Ethernet</name>
<section anchor="flexe" title="Flexible Ethernet"> <t>FlexE <xref target="FLEXE" format="default"/> provides the
<t>FlexE <xref target="FLEXE"/> provides the ability to multiplex ability to multiplex channels over an Ethernet link to create
channels over an Ethernet link to create point-to-point point-to-point fixed-bandwidth connections in a way that provides
fixed-bandwidth connections in a way that provides separation separation between enhanced VPN services. FlexE also supports
between enhanced VPN services. FlexE also supports bonding links to bonding multiple low-capacity links to create larger
create larger links out of multiple low-capacity links.</t> links.</t>
<t>However, FlexE is only a link-level technology. When packets are
<t>However, FlexE is only a link-level technology. When packets are received by the downstream node, they need to be processed in a way
received by the downstream node, they need to be processed in a way that preserves traffic isolation. In turn, this requires a queuing
that preserves that traffic isolation in the downstream node. This and forwarding implementation that preserves the end-to-end
in turn requires a queuing and forwarding implementation that separation of enhanced VPNs.</t> <t>If different FlexE channels are
preserves the end-to-end separation of enhanced VPNs.</t> used for different services, then no sharing is possible between the
FlexE channels. Thus, it may be difficult to dynamically
<t>If different FlexE channels are used for different services, then redistribute unused bandwidth to lower priority services in another
no sharing is possible between the FlexE channels. This means that FlexE channel. If one FlexE channel is used by one customer, the
it may be difficult to dynamically redistribute unused bandwidth to customer can use some methods to manage the relative priority of
lower priority services in another FlexE channel. If one FlexE their own traffic in the FlexE channel.</t>
channel is used by one customer, the customer can use some methods
to manage the relative priority of their own traffic in the FlexE
channel.</t>
</section> </section>
<section anchor="dedicated-queues" numbered="true" toc="default">
<section anchor="dedicated-queues" title="Dedicated Queues"> <name>Dedicated Queues</name>
<t>DiffServ-based queuing systems are described in <xref <t>Diffserv-based queuing systems are described in <xref target="RFC24
target="RFC2475"/> and <xref target="RFC4594"/>. This approach is 75" format="default"/> and <xref target="RFC4594" format="default"/>. This appro
ach is
not sufficient to provide separation of enhanced VPN services not sufficient to provide separation of enhanced VPN services
because DiffServ does not provide enough markers to differentiate because Diffserv does not provide enough markers to differentiate
between traffic of a large number of enhanced VPN services. between traffic of a large number of enhanced VPN services.
Additionally, DiffServ does not offer the range of service classes Additionally, Diffserv does not offer the range of service classes
that each enhanced VPN service needs to provide to its tenants. This that each enhanced VPN service needs to provide to its tenants. This
problem is particularly acute with an MPLS underlay, because MPLS problem is particularly acute with an MPLS underlay because MPLS
only provides eight traffic classes.</t> only provides eight traffic classes.</t>
<t>In addition, Diffserv, as currently implemented, mainly provides
<t>In addition, DiffServ, as currently implemented, mainly provides
per- hop priority-based scheduling, and it is difficult to use it to per- hop priority-based scheduling, and it is difficult to use it to
achieve quantitative resource reservation for different enhanced VPN achieve quantitative resource reservation for different enhanced VPN
services.</t> services.</t>
<t>To address these problems and to reduce the potential <t>To address these problems and to reduce the potential
interactions between enhanced VPN services, it would be necessary to interactions between enhanced VPN services, it would be necessary to
steer traffic to dedicated input and output queues per enhanced VPN steer traffic to dedicated input and output queues per enhanced VPN
service or per group of enhanced VPN services: some routers have a service or per group of enhanced VPN services: some routers have a
large number of queues and sophisticated queuing systems which could large number of queues and sophisticated queuing systems that could
support this, while some routers may struggle to provide the support this while some routers may struggle to provide the
granularity and level of separation required by the applications of granularity and level of separation required by the applications of
an enhanced VPN.</t> an enhanced VPN.</t>
</section> </section>
<section anchor="time-sensitive-networking" numbered="true" toc="default
<section anchor="time-sensitive-networking" ">
title="Time Sensitive Networking"> <name>Time-Sensitive Networking</name>
<t>Time-Sensitive Networking (TSN) <xref target="TSN"/> is an IEEE <t><xref target="TSN" format="default"/> is an IEEE
project to provide a method of carrying time-sensitive information project to provide a method of carrying time-sensitive information
over Ethernet. It introduces the concept of packet scheduling where over Ethernet. It introduces the concept of packet scheduling where
a packet stream may be given a time slot guaranteeing that it a packet stream may be given a time slot guaranteeing that it will
experiences no queuing delay or increase in latency beyond the very experience no queuing delay or increase in latency beyond a very
small scheduling delay. The mechanisms defined in TSN can be used to small scheduling delay. The mechanisms defined in TSN can be used to
meet the requirements of time-sensitive traffic flows of enhanced meet the requirements of time-sensitive traffic flows of enhanced
VPN service.</t> VPN service.</t>
<t>Ethernet can be emulated over a L3 network using an IP or
<t>Ethernet can be emulated over a layer-3 network using an IP or
MPLS pseudowire. However, a TSN Ethernet payload would be opaque to MPLS pseudowire. However, a TSN Ethernet payload would be opaque to
the underlay and thus not treated specifically as time-sensitive the underlay; thus, it would not be treated specifically as time-sensi
data. The preferred method of carrying TSN over a layer-3 network is tive
through the use of deterministic networking as explained in <xref data. The preferred method of carrying TSN over a L3 network is
target="deterministic-networking"/>.</t> through the use of DetNet as explained in <xref target="deterministic-
networking" format="default"/>.</t>
</section> </section>
</section> </section>
<section anchor="NW-DP" numbered="true" toc="default">
<section anchor="NW-DP" <name>Network Layer Encapsulation and Forwarding</name>
title="Network Layer Encapsulation and Forwarding">
<t>This section considers the problem of enhanced VPN service <t>This section considers the problem of enhanced VPN service
differentiation and the representation of underlying network resources differentiation and the representation of underlying network resources
in the network layer. More specifically, it describes the possible in the network layer. More specifically, it describes the possible
data plane mechanisms to determine the network resources and the data plane mechanisms to determine the network resources and the
logical network topology or paths associated with an NRP.</t> logical network topology or paths associated with an NRP.</t>
<section anchor="deterministic-networking" numbered="true" toc="default"
<section anchor="deterministic-networking" >
title="Deterministic Networking"> <name>Deterministic Networking (DetNet)</name>
<t>Deterministic Networking (DetNet) <xref target="RFC8655"/> is a <t>DetNet <xref target="RFC8655" format="default"/> is a
technique being developed in the IETF to enhance the ability of technique being developed in the IETF to enhance the ability of
layer-3 networks to deliver packets more reliably and with greater L3 networks to deliver packets more reliably and with greater
control over the delay. The design cannot use re-transmission control over the delay. The design cannot use retransmission
techniques such as TCP since that can exceed the delay tolerated by techniques such as TCP because that can exceed the delay tolerated by
the applications. DetNet preemptively sends copies of the packet the applications. DetNet preemptively sends copies of the packet
over various paths to minimize the chance of all copies of a packet over various paths to minimize the chance of all copies of a packet
being lost. It also seeks to set an upper bound on latency, but the being lost. It also seeks to set an upper bound on latency, but the
goal is not to minimize latency. DetNet can be realized over IP data goal is not to minimize latency. DetNet can be realized over the IP da
plane <xref target="RFC8939"/> or MPLS data plane <xref ta
target="RFC8964"/>, and may be used to provide deterministic paths plane <xref target="RFC8939" format="default"/> or the MPLS data plane
<xref target="RFC8964" format="default"/>, and it may be used to provide determ
inistic paths
for enhanced VPN services.</t> for enhanced VPN services.</t>
</section> </section>
<section anchor="mpls-traffic-engineering-mpls-te" numbered="true" toc="
<section anchor="mpls-traffic-engineering-mpls-te" default">
title="MPLS Traffic Engineering (MPLS-TE)"> <name>MPLS Traffic Engineering (MPLS-TE)</name>
<t>MPLS-TE <xref target="RFC2702"/><xref target="RFC3209"/> <t>MPLS-TE (see <xref target="RFC2702" format="default"/> and <xref ta
rget="RFC3209" format="default"/>)
introduces the concept of reserving end-to-end bandwidth for a introduces the concept of reserving end-to-end bandwidth for a
TE-LSP, which can be used to provide a set of point-to-point TE-LSP, which can be used to provide a set of point-to-point
resource reserved paths across the underlay network to support VPN resource-reserved paths across the underlay network to support VPN
services. VPN traffic can be carried over dedicated TE-LSPs to services. VPN traffic can be carried over dedicated TE-LSPs to
provide guaranteed bandwidth for each specific connection in a VPN, provide guaranteed bandwidth for each specific connection in a VPN,
and VPNs with similar behavior requirements may be multiplexed onto and VPNs with similar behavior requirements may be multiplexed onto
the same TE-LSPs. Some network operators have concerns about the the same TE-LSPs. Some network operators have concerns about the
scalability and management overhead of MPLS-TE system, especially scalability and management overhead of MPLS-TE system, especially
with regard to those systems that use an active control plane, and with regard to those systems that use an active control plane, and
this has lead them to consider other solutions for traffic this has lead them to consider other solutions for TE in their network
engineering in their networks.</t> s.</t>
</section> </section>
<section anchor="SR" numbered="true" toc="default">
<section anchor="SR" title="Segment Routing"> <name>Segment Routing</name>
<t>Segment Routing (SR) <xref target="RFC8402"/> is a method that <t>SR <xref target="RFC8402" format="default"/> is
prepends instructions to packets at the head-end of a path. These a method that prepends instructions to packets at the headend of a
instructions are used to specify the nodes and links to be path. These instructions are used to specify the nodes and links to
traversed, and allow the packets to be routed on paths other than be traversed, and they allow the packets to be routed on paths other
the shortest path. By encoding the state in the packet, per-path than the shortest path. By encoding the state in the packet,
state is transitioned out of the network. SR can be instantiated per-path state is transitioned out of the network. SR can be
using MPLS data plane (SR-MPLS) <xref target="RFC8660"/> or IPv6 instantiated using the MPLS data plane (SR-MPLS) (see <xref
data plane (SRv6) <xref target="RFC8986"/>.</t> target="RFC8660" format="default"/>) or the IPv6 data plane (SRv6)
(see <xref target="RFC8986" format="default"/>).</t>
<t>An SR traffic engineered path operates with a granularity of a <t>An SR traffic-engineered path operates with the granularity of a
link. Hints about priority are provided using the Traffic Class (TC) link. Hints about priority are provided using the Traffic Class (TC)
field in the packet header. However, to achieve the performance and field in the packet header. However, to achieve the performance and
isolation characteristics that are sought by enhanced VPN customers, isolation characteristics that are sought by enhanced VPN customers,
it will be necessary to steer packets through specific virtual links it will be necessary to steer packets through specific virtual links
and/or queues on the same link and direct them to use specific and/or queues on the same link and direct them to use specific
resources. With SR, it is possible to introduce such fine-grained resources. With SR, it is possible to introduce such fine-grained
packet steering by specifying the queues and the associated packet steering by specifying the queues and the associated
resources through an SR instruction list. One approach to do this is resources through an SR instruction list. One approach to do this is
described in <xref described in <xref target="I-D.ietf-spring-resource-aware-segments" fo
target="I-D.ietf-spring-resource-aware-segments"/>.</t> rmat="default"/>.</t>
<t>Note that the concept of a queue is a useful abstraction for <t>Note that the concept of a queue is a useful abstraction for
different types of underlay mechanism that may be used to provide different types of underlay mechanisms that may be used to provide
enhanced isolation and performance support. How the queue satisfies enhanced isolation and performance support. How the queue satisfies
the requirement is implementation specific and is transparent to the the requirement is implementation specific and is transparent to the
layer-3 data plane and control plane mechanisms used.</t> L3 data plane and control plane mechanisms used.</t>
<t>With SR, the SR instruction list could be used to
<t>With Segment Routing, the SR instruction list could be used to
build a P2P SR path. In addition, a group of SR Segment Identifiers build a P2P SR path. In addition, a group of SR Segment Identifiers
(SIDs) could also be used to represent an MP2MP network. Thus, the (SIDs) could also be used to represent an MP2MP network. Thus, the
SR based mechanism could be used to provide both resource reserved SR-based mechanism could be used to provide both resource-reserved
paths and NRPs for enhanced VPN services.</t> paths and NRPs for enhanced VPN services.</t>
</section> </section>
<section numbered="true" toc="default">
<section title="New Encapsulation Extensions"> <name>New Encapsulation Extensions</name>
<t>In contrast to reusing existing data plane for enhanced VPN, <t>In contrast to reusing an existing data plane for enhanced VPN,
another possible approach is to introduce new encapsulations or another possible approach is to introduce new encapsulations or
extensions to existing data plane to allow dedicated identifiers for extensions to an existing data plane to allow dedicated identifiers fo
the underlay network resources of an enhanced VPN, and the logical r
the underlay network resources of an enhanced VPN and the logical
network topology or paths associated with an enhanced VPN. This may network topology or paths associated with an enhanced VPN. This may
require more protocol work, while the potential benefit is it can require more protocol work; however, the potential benefits are that i t can
reduce the impact to existing network operation and improve the reduce the impact to existing network operation and improve the
scalability of enhanced VPN. More details about the encapsulation scalability of enhanced VPN. More details about the encapsulation
extensions and the scalability concerns are described in <xref extensions and the scalability concerns are described in <xref target=
target="I-D.ietf-teas-nrp-scalability"/>.</t> "I-D.ietf-teas-nrp-scalability" format="default"/>.</t>
</section> </section>
</section> </section>
<section anchor="Non-Packet-DP" numbered="true" toc="default">
<section anchor="Non-Packet-DP" title="Non-Packet Data Plane"> <name>Non-Packet Data Plane</name>
<t>Non-packet underlay data plane technologies, such as optical based <t>Non-packet underlay data plane technologies, such as optical-based
data planes often have TE properties and behaviors, and meet many of data planes, often have TE properties and behaviors. They meet many of
the key requirements in particular for bandwidth guarantees, traffic the key requirements, particularly those for:</t>
<ul>
<li>bandwidth guarantees,</li>
<li>traffic
isolation (with physical isolation often being an integral part of the isolation (with physical isolation often being an integral part of the
technology), highly predictable latency and jitter characteristics, technology),</li>
measurable loss characteristics, and ease of identification of flows. <li>highly predictable latency and jitter characteristics,</li>
The cost is that the resources are allocated on a long-term and <li>measurable loss characteristics, and</li>
<li>ease of identification of flows.</li>
</ul>
<t>The cost is that the resources are allocated on a long-term and
end-to-end basis. Such an arrangement means that the full cost of the end-to-end basis. Such an arrangement means that the full cost of the
resources has to be borne by the client to which the resources are resources has to be borne by the client to which the resources are
allocated. When an NRP built with this data plane is used to support allocated. When an NRP built with this data plane is used to support
multiple enhanced VPN services, the cost could be distributed among multiple enhanced VPN services, the cost could be distributed among
such a group of services.</t> such a group of services.</t>
</section> </section>
<section anchor="control-plane" numbered="true" toc="default">
<name>Control Plane</name>
<section anchor="control-plane" title="Control Plane">
<t>The control plane of NRP-based enhanced VPNs is likely be based on <t>The control plane of NRP-based enhanced VPNs is likely be based on
a hybrid control mechanism that takes advantage of a logically a hybrid control mechanism that takes advantage of a logically
centralized controller for on-demand provisioning and global centralized controller for on-demand provisioning and global
optimization, whilst still relying on a distributed control plane to optimization while still relying on a distributed control plane to
provide scalability, high reliability, fast reaction, automatic provide scalability, high reliability, fast reaction, automatic
failure recovery, etc. Extension to and optimization of the failure recovery, etc. Extension to and optimization of the
centralized and distributed control plane is needed to support the centralized and distributed control plane is needed to support the
enhanced properties of an NRP-based enhanced VPN.</t> enhanced properties of an NRP-based enhanced VPN.</t>
<t>As described in <xref target="architecture-and-components-of-vpn"/>,
<t>As described in Section 4, the enhanced VPN control plane needs to the enhanced VPN control plane needs to
provide the following functions: <list style="symbols"> provide the following functions: </t>
<t>Collect information about the underlying network topology and <ul spacing="normal">
network resources, and exports this to network nodes and/or a <li>
<t>Collection of information about the underlying network topology a
nd
network resources and exportation of this to network nodes and/or a
centralized controller as required.</t> centralized controller as required.</t>
</li>
<t>Create NRPs with the network resource and topology properties <li>
<t>Creation of NRPs with the network resource and topology propertie
s
needed by NRP-based enhanced VPN services.</t> needed by NRP-based enhanced VPN services.</t>
</li>
<t>Distribute the attributes of NRPs to network nodes which <li>
<t>Distribution of the attributes of NRPs to network nodes that
participate in the NRPs and/or the centralized controller.</t> participate in the NRPs and/or the centralized controller.</t>
</li>
<t>Map enhanced VPN services to an appropriate NRP.</t> <li>
<t>Mapping of enhanced VPN services to an appropriate NRP.</t>
<t>Compute and set up service paths in each NRP to meet enhanced </li>
<li>
<t>Computation and set up of service paths in each NRP to meet enhan
ced
VPN service requirements.</t> VPN service requirements.</t>
</list></t> </li>
</ul>
<t>The collection of underlying network topology and resource <t>Underlying network topology and resource
information can be done using the existing IGP and Border Gateway information can be collected using mechanisms based on the existing IGP
Protocol - Link State (BGP-LS) <xref target="RFC9552"/> based and Border Gateway
mechanisms. The creation of NRPs and the distribution of NRP Protocol - Link State (BGP-LS) <xref target="RFC9552" format="default"/>
. The creation of NRPs and the distribution of NRP
attributes may need further control protocol extensions. The attributes may need further control protocol extensions. The
computation of service paths based on the attributes and constraints computation of service paths based on the attributes and constraints
of the NRP can be performed either by the headend node of the path or of the NRP can be performed either by the headend node of the path or
a centralized Path Computation Element (PCE) <xref by a centralized Path Computation Element (PCE) <xref target="RFC4655" f
target="RFC4655"/>.</t> ormat="default"/>.</t>
<t>Two candidate control plane mechanisms for path setup in the NRP <t>Two candidate control plane mechanisms for path setup in the NRP
are: RSVP-TE and Segment Routing (SR).</t> are RSVP-TE and SR.</t>
<ul spacing="normal">
<t><list style="symbols"> <li>
<t>RSVP-TE <xref target="RFC3209"/> provides the signaling <t>RSVP-TE, as described in <xref target="RFC3209" format="default"/
>, provides the signaling
mechanism for establishing a TE-LSP in an MPLS network with mechanism for establishing a TE-LSP in an MPLS network with
end-to-end resource reservation. This can be seen as an approach end-to-end resource reservation. This can be seen as an approach
of providing resource-reserved paths which could be used to bind of providing resource-reserved paths that could be used to bind
the VPN to specific set of network resources allocated within the the VPN to a specific set of network resources allocated within the
underlay, but there remain scalability concerns as mentioned in underlay; however, there remain scalability concerns, as mentioned i
<xref target="mpls-traffic-engineering-mpls-te"/>.</t> n
<xref target="mpls-traffic-engineering-mpls-te" format="default"/>.<
<t>The SR control plane <xref target="RFC8665"/> <xref /t>
target="RFC8667"/> <xref target="RFC9085"/> does not have the </li>
<li>
<t>The SR control plane, as described in <xref target="RFC8665" form
at="default"/>, <xref target="RFC8667" format="default"/>, and <xref target="RFC
9085" format="default"/>, does not have the
capability of signaling resource reservations along the path. On capability of signaling resource reservations along the path. On
the other hand, the SR approach provides a potential way of the other hand, the SR approach provides a potential way of
binding the underlay network resource and the NRPs without binding the underlay network resource and the NRPs without
requiring per-path state to be maintained in the network. A requiring per-path state to be maintained in the network. A
centralized controller can perform resource planning and centralized controller can perform resource planning and
reservation for NRPs, and it needs to instruct the network nodes reservation for NRPs, and it needs to instruct the network nodes
to ensure that resources are correctly allocated for the NRP. The to ensure that resources are correctly allocated for the NRP. The
controller could provision the SR paths based on the mechanism in controller could provision the SR paths based on the mechanism in
<xref target="RFC9256"/> to the headend nodes of the paths.</t> <xref target="RFC9256" format="default"/> to the headend nodes of th
</list></t> e paths.</t>
</li>
<t>According to the service requirements for connectivity, performance </ul>
<t>According to the service requirements for connectivity, performance,
and isolation, one enhanced VPN service may be mapped to a dedicated and isolation, one enhanced VPN service may be mapped to a dedicated
NRP, or a group of enhanced VPN services may be mapped to the same NRP or a group of enhanced VPN services may be mapped to the same
NRP. The mapping of enhanced VPN services to NRP can be achieved using NRP. The mapping of enhanced VPN services to an NRP can be achieved usin
existing control mechanisms with possible extensions, and it can be g
existing control mechanisms with possible extensions; it can be
based on either the characteristics of the data packet or the based on either the characteristics of the data packet or the
attributes of the VPN service routes.</t> attributes of the VPN service routes.</t>
</section> </section>
<section anchor="management-plane" numbered="true" toc="default">
<section anchor="management-plane" title="Management Plane"> <name>Management Plane</name>
<t>The management plane provides the interface between the enhanced <t>The management plane provides the interface between the enhanced
VPN service provider and the customers for life-cycle management of VPN service provider and the customers for life-cycle management of
the enhanced VPN service (i.e., creation, modification, the enhanced VPN service (i.e., creation, modification,
assurance/monitoring, and decommissioning). It relies on a set of assurance/monitoring, and decommissioning). It relies on a set of
service data models for the description of the information and service data models for the description of the information and
operations needed on the interface.</t> operations needed on the interface.</t>
<t>As an example, in the context of 5G end-to-end network slicing <t>As an example, in the context of 5G end-to-end network slicing
<xref target="TS28530"/>, the management of the transport network <xref target="TS28530" format="default"/>, the management of the
segment of the 5G end-to-end network slice can be realized with the transport network segment of the 5G end-to-end network slice can be
management plane of enhanced VPN. The 3GPP management system may realized with the management plane of the enhanced VPN. The 3GPP
provide the connectivity and performance-related parameters as management system may provide the connectivity and performance-related
requirements to the management plane of the transport network. It may parameters as requirements to the management plane of the transport
also require the transport network to expose the capabilities and network. It may also require the transport network to expose the
status of the network slice. Thus, an interface between the enhanced capabilities and status of the network slice. Thus, the coordination
VPN management plane and the 5G network slice management system, and of 5G end-to-end network slice management requires an interface
relevant service data models are needed for the coordination of 5G between the enhanced VPN management plane and the 5G network slice
end-to-end network slice management.</t> management system, and relevant service data models.</t>
<t>The management plane interface and data models for enhanced VPN <t>The management plane interface and data models for enhanced VPN
services can be based on the service models described in <xref services can be based on the service models described in <xref target="s
target="sdm-app"/>.</t> dm-app" format="default"/>.</t>
<t>It is important that the life-cycle management support in-place
<t>It is important that the management life-cycle supports in-place
modification of enhanced VPN services. That is, it should be possible modification of enhanced VPN services. That is, it should be possible
to add and remove end points, as well as to change the requested to add and remove endpoints, as well as to change the requested
characteristics of the service that is delivered. The management characteristics of the service that is delivered. The management
system needs to be able to assess the revised enhanced VPN requests system needs to be able to assess the revised enhanced VPN requests
and determine whether they can be provided by the existing NRPs or and determine whether they can be provided by the existing NRPs or
whether changes must be made, and it will additionally need to whether changes must be made. It will also need to
determine whether those changes to the NRP are possible. If not, then determine whether those changes to the NRP are possible. If not, then
the customer's modification request may be rejected.</t> the customer's modification request may be rejected.</t>
<t>When the modification of an enhanced VPN service is possible, the <t>When the modification of an enhanced VPN service is possible, the
management system must make every effort to make the changes in a management system must make every effort to make the changes in a
non-disruptive way. That is, the modification of the enhanced VPN non-disruptive way. That is, the modification of the enhanced VPN
service or the underlying NRP must not perturb traffic on the enhanced service or the underlying NRP must not perturb traffic on the enhanced
VPN service in a way that causes the service level to drop below the VPN service in a way that causes the service level to drop below the
agreed levels. Furthermore, changes to one enhanced VPN service should agreed levels. Furthermore, changes to one enhanced VPN service should
not cause disruption to other enhanced VPN services.</t> not cause disruption to other enhanced VPN services.</t>
<t>The network operator for the underlay network (i.e., the provider <t>The network operator for the underlay network (i.e., the provider
of the enhanced VPN service) may delegate some operational aspects of of the enhanced VPN service) may delegate some operational aspects of
the overlay VPN and the underlying NRP to the customer. In this way, the overlay VPN and the underlying NRP to the customer. In this way,
the enhanced VPN is presented to the customer as a virtual network, the enhanced VPN is presented to the customer as a virtual network,
and the customer can choose how to use that network. Some mechanisms and the customer can choose how to use that network. Some mechanisms
in the operator's network are needed, so that a customer cannot exceed in the operator's network are needed so that:</t>
the capabilities of the virtual links and nodes, but can decide how to <ul>
<li>a customer cannot exceed
the capabilities of the virtual links and nodes, but</li>
<li>it can decide how to
load traffic onto the network, for example, by assigning different load traffic onto the network, for example, by assigning different
metrics to the virtual links so that the customer can control how metrics to the virtual links so that the customer can control how
traffic is routed through the virtual network. This approach requires traffic is routed through the virtual network.</li>
a management system for the virtual network, but does not necessarily </ul>
<t>This approach requires
a management system for the virtual network but does not necessarily
require any coordination between the management systems of the virtual require any coordination between the management systems of the virtual
network and the physical network, except that the virtual network network and the physical network, except that the virtual network
management system might notice when the NRP is close to capacity or management system might notice when the NRP is close to capacity or
considerably under-used and automatically request changes in the considerably under-used and automatically request changes in the
service provided by the underlay network.</t> service provided by the underlay network.</t>
</section> </section>
<section anchor="sdm-app" numbered="true" toc="default">
<section anchor="sdm-app" <name>Applicability of Service Data Models to Enhanced VPNs</name>
title="Applicability of Service Data Models to Enhanced VPNs">
<t>This section describes the applicability of the existing and <t>This section describes the applicability of the existing and
in-progress service data models to enhanced VPNs. <xref in-progress service data models to enhanced VPNs. <xref target="RFC8309"
target="RFC8309"/> describes the scope and purpose of service models format="default"/> describes the scope and purpose of service models
and shows where a service model might fit into an SDN-based network and shows where a service model might fit into an SDN-based network
management architecture. New service models may also be introduced for management architecture. New service models may also be introduced for
some of the required management functions.</t> some of the required management functions.</t>
<t>Service data models are used to represent, monitor, and manage the <t>Service data models are used to represent, monitor, and manage the
virtual networks and services enabled by enhanced VPNs. The VPN virtual networks and services enabled by enhanced VPNs. The VPN
customer service models (e.g., the Layer 3 VPN Service Model (L3SM) customer service models (e.g., the L3VPN Service Model (L3SM)
<xref target="RFC8299"/>, the Layer 2 VPN Service Model (L2SM) <xref in <xref target="RFC8299" format="default"/>, the L2VPN Service Model (L
target="RFC8466"/>), or the ACTN Virtual Network (VN) model <xref 2SM) in <xref target="RFC8466" format="default"/>), or the ACTN VN model in <xre
target="I-D.ietf-teas-actn-vn-yang"/>) are service models which can f target="RFC9731" format="default"/>) are service models that can
provide the customer's view of the enhanced VPN service. The Layer-3 provide the customer's view of the enhanced VPN service. The L3VPN
VPN Network Model (L3NM) <xref target="RFC9182"/>, the Layer-2 VPN Network Model (L3NM) from <xref target="RFC9182" format="default"/> and
network model (L2NM) <xref target="RFC9291"/> provide the operator's the L2VPN
Network Model (L2NM) from <xref target="RFC9291" format="default"/> prov
ide the operator's
view of the managed infrastructure as a set of virtual networks and view of the managed infrastructure as a set of virtual networks and
the associated resources. The Service Attachment Points (SAPs) model the associated resources. The Service Attachment Points (SAPs) model
<xref target="RFC9408"/> provides an abstract view of the service in <xref target="RFC9408" format="default"/> provides an abstract view o
attachment points (SAPs) to various network services in the provider f the SAPs to various network services in the provider
network, where enhanced VPN could be one of the service types. <xref network, where enhanced VPN could be one of the service types. <xref tar
target="RFC9375"/> provides the data model for performance monitoring get="RFC9375" format="default"/> provides the data model for performance monitor
ing
of network and VPN services. Augmentation to these service models may of network and VPN services. Augmentation to these service models may
be needed to provide the enhanced VPN services. The NRP model <xref be needed to provide the enhanced VPN services. The NRP model in <xref t
target="I-D.ietf-teas-nrp-yang"/> further provides the management of arget="I-D.ietf-teas-nrp-yang" format="default"/> further provides the managemen
t of
the NRP topology and resources both in the controller and in the the NRP topology and resources both in the controller and in the
network devices to instantiate the NRPs needed for the enhanced VPN network devices to instantiate the NRPs needed for the enhanced VPN
services.</t> services.</t>
</section> </section>
</section> </section>
<section anchor="app-ns-realization" numbered="true" toc="default">
<section anchor="app-ns-realization" <name>Applicability in Network Slice Realization</name>
title="Applicability in Network Slice Realization">
<t>This section describes the applicability of NRP-based enhanced VPN <t>This section describes the applicability of NRP-based enhanced VPN
for network slice realization.</t> for network slice realization.</t>
<t>In order to provide network slice services to customers, a <t>In order to provide network slice services to customers, a
technology-agnostic network slice service model <xref technology-agnostic network slice service model <xref target="I-D.ietf-tea
target="I-D.ietf-teas-ietf-network-slice-nbi-yang"/> is needed for the s-ietf-network-slice-nbi-yang" format="default"/> is needed for the
customers to communicate the requirements of network slices (SDPs, customers to communicate the requirements of network slices (SDPs,
connectivity, SLOs, and SLEs). These requirements may be realized using connectivity, SLOs, and SLEs). These requirements may be realized using
technology specified in this document to instruct the network to deliver technology specified in this document to instruct the network to deliver
an enhanced VPN service so as to meet the requirements of the network an enhanced VPN service so as to meet the requirements of the network
slice customers. According to the location of SDPs used for the network slice customers. According to the location of SDPs used for the network
slice service (see Section 5.2 of <xref target="RFC9543"/>), an SDP can slice service (see <xref target="RFC9543" sectionFormat="of" section="5.2"
be mapped to a CE, a PE, a port on a CE, or a customer-facing port on a />), an SDP can
PE, any of which can be correlated to the end point of enhanced VPN be mapped to a Customer Edge (CE), a PE, a port on a CE, or a customer-fac
service. The detailed approach for SDP mapping is described in <xref ing port on a
target="I-D.ietf-teas-ietf-network-slice-nbi-yang"/>.</t> PE, any of which can be correlated to the endpoint of the enhanced VPN
service. The detailed approach for SDP mapping is described in <xref targe
<section title="NRP Planning"> t="I-D.ietf-teas-ietf-network-slice-nbi-yang" format="default"/>.</t>
<section numbered="true" toc="default">
<name>NRP Planning</name>
<t>An NRP is used to support the SLOs and SLEs required by the network <t>An NRP is used to support the SLOs and SLEs required by the network
slice services. According to the network operators' network resource slice services. According to the network operators' network resource
planning policy, or based on the requirements of one or a group of planning policy, or based on the requirements of one or a group of
customers or services, an NRP may need to be created to meet the customers or services, an NRP may need to be created to meet the
requirements of network slice services. One of the basic requirements requirements of network slice services. One of the basic requirements
for the NRP is to provide a set of dedicated network resources to for the NRP is to provide a set of dedicated network resources to
avoid unexpected interference from other services in the same network. avoid unexpected interference from other services in the same network.
Other possible requirements may include the required topology and Other possible requirements may include the required topology and
connectivity, bandwidth, latency, reliability, etc.</t> connectivity, bandwidth, latency, reliability, etc.</t>
<t>A centralized network controller can be responsible for calculating <t>A centralized network controller can be responsible for calculating
a subset of the underlay network topology (which is called a logical a subset of the underlay network topology (which is called a logical
topology) to support the NRP requirement. On the network nodes and topology) to support the NRP requirement. On the network nodes and
links within the logical topology, the set of network resources to be links within the logical topology, the set of network resources to be
allocated to the NRP can also be determined by the controller. allocated to the NRP can also be determined by the controller.
Normally such calculation needs to take the underlay network Normally, such calculation needs to take the underlay network
connectivity information and the available network resource connectivity information and the available network resource
information of the underlay network into consideration. The network information of the underlay network into consideration. The network
controller may also take the status of the existing NRPs into controller may also take the status of the existing NRPs into
consideration in the planning and calculation of a new NRP.</t> consideration in the planning and calculation of a new NRP.</t>
</section> </section>
<section numbered="true" toc="default">
<section title="NRP Creation"> <name>NRP Creation</name>
<t>According to the result of the NRP planning, the network nodes and <t>According to the result of the NRP planning, the network nodes and
links involved in the logical topology of the NRP are instructed to links involved in the logical topology of the NRP are instructed to
allocate the required set of network resources for the NRP. One or allocate the required set of network resources for the NRP. One or
multiple mechanisms as specified in section 5.1 can be used to multiple mechanisms as specified in <xref target="L2-DP"/> can be used t
partition the forwarding plane network resources and allocate o
partition the forwarding-plane network resources and allocate
different subsets of resources to different NRPs. In addition, the different subsets of resources to different NRPs. In addition, the
data plane identifiers which are used to identify the set of network data plane identifiers that are used to identify the set of network
resources allocated to the NRP are also provisioned on the network resources allocated to the NRP are also provisioned on the network
nodes. Depending on the data plane technologies used, the set of nodes. Depending on the data plane technologies used, the set of
network resources of an NRP can be identified using e.g. either network resources of an NRP can be identified using, e.g.,
resource-aware SR segments as specified in <xref resource-aware SR segments as specified in <xref target="I-D.ietf-spring
target="I-D.ietf-spring-resource-aware-segments"/> <xref -resource-aware-segments" format="default"/> and <xref target="I-D.ietf-spring-s
target="I-D.ietf-spring-sr-for-enhanced-vpn"/>, or a dedicated r-for-enhanced-vpn" format="default"/> or a dedicated
Resource ID as specified in <xref Resource ID as specified in <xref target="I-D.ietf-6man-enhanced-vpn-vtn
target="I-D.ietf-6man-enhanced-vpn-vtn-id"/> can be introduced. The -id" format="default"/> can be introduced. The
network nodes involved in an NRP may distribute the logical topology network nodes involved in an NRP may distribute the logical topology
information, the NRP-specific network resource information and the information, the NRP-specific network resource information, and the
Resource Identifier of the NRP using the control plane. Such Resource ID of the NRP using the control plane. Such
information could be used by the controller and the network nodes to information could be used by the controller and the network nodes to
compute the TE or shortest paths within the NRP, and install the NRP compute the TE or shortest paths within the NRP and to install the NRP-s
specific forwarding entries to network nodes.</t> pecific forwarding entries to network nodes.</t>
</section> </section>
<section numbered="true" toc="default">
<section title="Network Slice Service Provisioning"> <name>Network Slice Service Provisioning</name>
<t>According to the connectivity requirements of an network slice <t>According to the connectivity requirements of a network slice
service, an overlay VPN can be created using the existing or future service, an overlay VPN can be created using the existing or future
multi-tenancy overlay technologies as described in <xref multi-tenancy overlay technologies as described in <xref target="applica
target="applicability"/>.</t> bility" format="default"/>.</t>
<t>Then, according to the SLO and SLE requirements of a network slice
<t>Then according to the SLO and SLE requirements of a network slice
service, the network slice service is mapped to an appropriate NRP as service, the network slice service is mapped to an appropriate NRP as
the virtual underlay. The integration of the overlay VPN and the the virtual underlay. The integration of the overlay VPN and the
underlay NRP together provide a network slice service.</t> underlay NRP provides a network slice service.</t>
</section> </section>
<section numbered="true" toc="default">
<section title="Network Slice Traffic Steering and Forwarding "> <name>Network Slice Traffic Steering and Forwarding</name>
<t>At the edge of the operator's network, traffic of network slices <t>At the edge of the operator's network, network slice traffic
can be classified based on the rules defined by the operator's policy, can be classified based on the rules defined by the operator's policy;
so that the traffic which matches the rules for specific network slice this is so that the traffic that matches the rules for specific network
services can be mapped to the corresponding NRP. This way, packets slice
belonging to specific network slice service will be processed and services can be mapped to the corresponding NRP. Thus, packets
forwarded by network nodes based either the traffic-engineered paths belonging to a specific network slice service will be processed and
or the shortest paths in the associated network topology, using the forwarded by network nodes based on either:</t>
<ul spacing="compact">
<li>the traffic-engineered paths or</li>
<li>the shortest paths in the associated network topology</li>
</ul>
<t>using the
set of network resources of the corresponding NRP.</t> set of network resources of the corresponding NRP.</t>
</section> </section>
</section> </section>
<section anchor="scalability-considerations" numbered="true" toc="default">
<section anchor="scalability-considerations" <name>Scalability Considerations</name>
title="Scalability Considerations">
<t>NRP-based enhanced VPNs provide performance guaranteed services in <t>NRP-based enhanced VPNs provide performance guaranteed services in
packet networks, but with the potential cost of introducing additional packet networks; however, this comes with the potential cost of introducin g additional
state into the network. There are at least three ways that this state into the network. There are at least three ways that this
additional state might be brought into the network:</t> additional state might be added:</t>
<ul spacing="normal">
<t><list style="symbols"> <li>
<t>Introduce the complete state into the packet, as is done in SR. <t>by introducing the complete state into the packet, as is done in SR
.
This allows the controller to specify the detailed series of This allows the controller to specify the detailed series of
forwarding and processing instructions for the packet as it transits forwarding and processing instructions for the packet as it transits
the network. The cost of this is an increase in the packet header the network. The cost of this is an increase in the packet header
size. The cost is also that systems will have to provide NRP size. A further cost is that systems will have to provide NRP-specific
specific segments in case they are called upon by a service. This is segments in case they are called upon by a service. This is
a type of latent state, and increases as the segments and resources a type of latent state, and it increases as the segments and resources
that need to be exclusively available to enhanced VPN service are that need to be exclusively available to enhanced VPN service are
specified more precisely.</t> specified more precisely.</t>
</li>
<t>Introduce the state to the network. This is normally done by <li>
<t>by introducing the state to the network. This is normally done by
creating a path using signaling such as RSVP-TE. This could be creating a path using signaling such as RSVP-TE. This could be
extended to include any element that needs to be specified along the extended to include any element that needs to be specified along the
path, for example explicitly specifying queuing policy. It is also path, for example, explicitly specifying queuing policy. It is also
possible to use other methods to introduce path state, such as via possible to use other methods to introduce path state, such as via
an SDN controller, or possibly by modifying a routing protocol. With an SDN controller or possibly by modifying a routing protocol. With
this approach there is state per path: per-path characteristic that this approach, there is state per path: a per-path characteristic that
needs to be maintained over the life of the path. This is more needs to be maintained over the life of the path. This is more
network state than is needed using SR, but the packets are usually network state than is needed using SR, but the packets are usually
shorter.</t> shorter.</t>
</li>
<t>Provide a hybrid approach. One example is based on using binding <li>
SIDs <xref target="RFC8402"/> to represent path fragments, and bind <t>by providing a hybrid approach. One example is based on using bindi
ng
SIDs (see <xref target="RFC8402" format="default"/>) to represent path
fragments and binding
them together with SR. Dynamic creation of a VPN service path using them together with SR. Dynamic creation of a VPN service path using
SR requires less state maintenance in the network core at the SR requires less state maintenance in the network core at the
expense of larger packet headers. The packet size can be lower if a expense of larger packet headers. The packet size can be lower if a
form of loose source routing is used (using a few nodal SIDs), and form of loose source routing is used (using a few nodal SIDs), and
it will be lower if no specific functions or resources on the it will be lower if no specific functions or resources on the
routers are specified. For SRv6, the packet size may also be reduced routers are specified. For Segment Routing over IPv6 (SRv6), the packe
by utilizing the compression techniques as specified in <xref t size may also be reduced
target="I-D.ietf-spring-srv6-srh-compression"/>.</t> by utilizing the compression techniques specified in <xref target="I-D
</list></t> .ietf-spring-srv6-srh-compression" format="default"/>.</t>
</li>
<t>Reducing the state in the network is important to enhanced VPNs, as </ul>
it requires the overlay to be more closely integrated with the underlay
than with conventional VPNs. This tighter coupling would normally mean
that more state needs to be created and maintained in the network, as
the state about fine granularity processing would need to be loaded and
maintained in the routers. Aggregation is a well-established approach to
reduce the amount of state and improve scaling, and NRP is considered as
the network construct to aggregate the states of enhanced VPN services.
In addition, an SR approach allows much of the state to be spread
amongst the network ingress nodes, and transiently carried in the
packets as SIDs.</t>
<t>Reducing the state in the network is important to the deployment of
enhanced VPNs, as they require the overlay to be more closely integrated
with the underlay than with conventional VPNs. This tighter coupling
would normally mean that more state needs to be created and maintained
in the network, as state about fine-granularity processing would need to
be loaded and maintained in the routers. Aggregation is a
well-established approach to reduce the amount of state and improve
scaling, and NRP is considered to be the network construct to aggregate
the states of enhanced VPN services. In addition, an SR approach allows
much of the state to be spread amongst the network ingress nodes and
transiently carried in the packets as SIDs.</t>
<t>The following subsections describe some of the scalability concerns <t>The following subsections describe some of the scalability concerns
that need to be considered. Further discussion of the scalability that need to be considered. Further discussion of the scalability
considerations of the underlaying network constructs of NRP-based considerations of the underlaying network constructs of NRP-based
enhanced VPNs can be found in <xref enhanced VPNs can be found in <xref target="I-D.ietf-teas-nrp-scalability"
target="I-D.ietf-teas-nrp-scalability"/>.</t> format="default"/>.</t>
<section anchor="maximum-stack-depth" numbered="true" toc="default">
<section anchor="maximum-stack-depth" title="Maximum Stack Depth of SR"> <name>Maximum Stack Depth of SR</name>
<t>One of the challenges with SR is the stack depth that nodes are <t>One of the challenges with SR is the stack depth that nodes are
able to impose on packets <xref target="RFC8491"/>. This leads to a able to impose on packets <xref target="RFC8491" format="default"/>. Thi
difficult balance between adding state to the network and minimizing s leads to a
stack depth, or minimizing state and increasing the stack depth.</t> difficult balance between:</t>
<ul spacing="compact">
<li>adding state to the network and minimizing
stack depth and</li>
<li>minimizing state and increasing the stack depth.</li>
</ul>
</section> </section>
<section anchor="rsvp-scalability" numbered="true" toc="default">
<section anchor="rsvp-scalability" title="RSVP-TE Scalability"> <name>RSVP-TE Scalability</name>
<t>The established method of creating a resource-reserved path through <t>The established method of creating a resource-reserved path through
an MPLS network is to use the RSVP-TE protocol. However, there have an MPLS network is to use the RSVP-TE protocol. However, there have
been concerns that this requires significant continuous state been concerns that this requires significant continuous state
maintenance in the network. Work to improve the scalability of RSVP-TE maintenance in the network. Work to improve the scalability of RSVP-TE
LSPs in the control plane can be found in <xref LSPs in the control plane can be found in <xref target="RFC8370" format=
target="RFC8370"/>.</t> "default"/>.</t>
<t>There is also concern at the scalability of the forwarder footprint <t>There is also concern at the scalability of the forwarder footprint
of RSVP-TE as the number of paths through a label switching router of RSVP-TE as the number of paths through a Label Switching Router
(LSR) grows. <xref target="RFC8577"/> addresses this by employing SR (LSR) grows. <xref target="RFC8577" format="default"/> addresses this by
employing SR
within a tunnel established by RSVP-TE.</t> within a tunnel established by RSVP-TE.</t>
</section> </section>
<section numbered="true" toc="default">
<name>SDN Scaling</name>
<section title="SDN Scaling"> <t>The centralized SDN-based approach requires control plane state to be
<t/>
<t>The centralized approach of SDN requires control plane state to be
stored in the network, but can reduce the overhead of control channels stored in the network, but can reduce the overhead of control channels
to be maintained. Each individual network node may need to maintain a to be maintained. Each individual network node may need to maintain a
control channel with an SDN controller, which is considered more control channel with an SDN controller, which is considered more
scalable comparing to the need of maintaining control channels with a scalable compared to the need of maintaining control channels with a
set of neighbor nodes.</t> set of neighbor nodes.</t>
<t>However, SDN may transfer some of the scalability concerns from the <t>However, SDN may transfer some of the scalability concerns from the
network to a centralized controller. In particular, there may be a network to a centralized controller. In particular, there may be a
heavy processing burden at the controller, and a heavy load in the heavy processing burden at the controller and a heavy load in the
network surrounding the controller. A centralized controller may also network surrounding the controller. A centralized controller may also
present a single point of failure within the network.</t> present a single point of failure within the network.</t>
</section> </section>
</section> </section>
<section anchor="enhanced-resiliency" numbered="true" toc="default">
<section anchor="enhanced-resiliency" title="Enhanced Resiliency"> <name>Enhanced Resiliency</name>
<t>Each enhanced VPN service has a life cycle, and may need modification <t>Each enhanced VPN service has a life cycle and may need modification
during deployment as the needs of its tenant change. This is discussed during deployment as the needs of its tenant change (see <xref target="man
in <xref target="management-plane"/>. Additionally, as the network agement-plane" format="default"/>). Additionally, as the network
evolves, there may need to perform garbage collection to consolidate evolves, garbage collection may need to be performed to consolidate
resources into usable quanta.</t> resources into usable quanta.</t>
<t>Systems in which the path is imposed, such as SR or some form of <t>Systems in which the path is imposed, such as SR or some form of
explicit routing, tend to do well in these applications, because it is explicit routing, tend to do well in these applications because it is
possible to perform an atomic transition from one path to another. That possible to perform an atomic transition from one path to another. That
is, a single action by the head-end that changes the path without the is, a single action by the headend that changes the path without the
need for coordinated action by the routers along the path. However, need for coordinated action by the routers along the path. However,
implementations and the monitoring protocols need to make sure that the implementations and the monitoring protocols need to make sure that the
new path is operational and meets the required SLA before traffic is new path is operational and meets the required SLA before traffic is
transitioned to it. It is possible for deadlocks to arise as a result of transitioned to it. It is possible for deadlocks to arise as a result of
the network becoming fragmented over time, such that it is impossible to the network becoming fragmented over time, such that it is impossible to
create a new path or to modify an existing path without impacting the create a new path or to modify an existing path without impacting the
SLA of other paths. The global concurrent optimization mechanisms as SLA of other paths. The global concurrent optimization mechanisms as
described in <xref target="RFC5557"/> and discussed in <xref described in <xref target="RFC5557" format="default"/> and discussed in
target="RFC7399"/> may be helpful, while complete resolution of this <xref target="RFC7399" format="default"/> may be helpful, while complete
situation is as much a commercial issue as it is a technical issue.</t> resolution of this situation is as much a commercial issue as it is a
technical issue.</t>
<t>There are, however, two manifestations of the latency problem that <t>However, there are two manifestations of the latency problem that
are for further study in any of these approaches:</t> are for further study in any of these approaches:</t>
<ul spacing="normal">
<t><list style="symbols"> <li>
<t>The problem of packets overtaking one another if a path latency <t>Packets overtaking one another if path latency
reduces during a transition.</t> reduces during a transition.</t>
</li>
<t>The problem of transient variation in latency in either direction <li>
<t>Transient variation in latency in either direction
as a path migrates.</t> as a path migrates.</t>
</list></t> </li>
</ul>
<t>There is also the matter of what happens during failure in the <t>There is also the matter of what happens during failure in the
underlay infrastructure. Fast reroute is one approach, but that still underlay infrastructure. Fast reroute is one approach, but that still
produces a transient loss with a normal goal of rectifying this within produces a transient loss with a normal goal of rectifying this within
50ms <xref target="RFC5654"/>. An alternative is some form of N+1 50 ms <xref target="RFC5654" format="default"/>. An alternative is some fo rm of N+1
delivery such as has been used for many years to support protection from delivery such as has been used for many years to support protection from
service disruption. This may be taken to a different level using the service disruption. This may be taken to a different level using the
techniques of DetNet with multiple in-network replication and the techniques of DetNet with multiple in-network replications and the
culling of later packets <xref target="RFC8655"/>.</t> culling of later packets <xref target="RFC8655" format="default"/>.</t>
<t>In addition to the approach used to protect high-priority packets,
<t>In addition to the approach used to protect high priority packets, consideration should be given to the impact of best-effort traffic on
consideration should be given to the impact of best effort traffic on the high-priority packets during a transition. Specifically, if a
the high priority packets during a transition. Specifically, if a conventional re-convergence process is used, there will inevitably be
conventional re-convergence process is used there will inevitably be micro-loops and, while some form of explicit routing will protect the
micro-loops and whilst some form of explicit routing will protect the high-priority traffic, lower-priority traffic on best-effort shortest
high priority traffic, lower priority traffic on best effort shortest paths will micro-loop without the use of a loop-prevention technology.
paths will micro-loop without the use of a loop prevention technology. To provide the highest quality of service to high-priority traffic,
To provide the highest quality of service to high priority traffic, either this traffic must be shielded from the micro-loops or
either this traffic must be shielded from the micro-loops, or
micro-loops must be prevented completely.</t> micro-loops must be prevented completely.</t>
</section> </section>
<section anchor="oam-and-instrumentation" numbered="true" toc="default">
<section anchor="oam-and-instrumentation" <name>Manageability Considerations</name>
title="Manageability Considerations"> <t>This section describes the considerations about the OAM and telemetry
<t>This section describes the considerations about the OAM and Telemetry mechanisms used to support the verification, monitoring, and optimization
mechanisms used to support the verification, monitoring and optimization
of the characteristics and SLA fulfillment of NRP-based enhanced VPN of the characteristics and SLA fulfillment of NRP-based enhanced VPN
services. It should be read along with <xref target="management-plane"/> services. It should be read along with <xref target="management-plane" for
that gives consideration of the management plane techniques that can be mat="default"/>, which gives consideration to the management plane techniques th
at can be
used to build NRPs.</t> used to build NRPs.</t>
<section numbered="true" toc="default">
<name>OAM Considerations</name>
<section title="OAM Considerations"> <t>The following requirements need to be considered in the design of
<t>The design of OAM for enhanced VPN services needs to consider the OAM for enhanced VPN services:</t>
following requirements:</t> <ul spacing="normal">
<li>
<t><list style="symbols">
<t>Instrumentation of the NRP (the virtual underlay) so that the <t>Instrumentation of the NRP (the virtual underlay) so that the
network operator can be sure that the resources committed to a network operator can be sure that the resources committed to a
customer are operating correctly and delivering the required customer are operating correctly and delivering the required
performance. It is important that the OAM packets follow the same performance. It is important that the OAM packets follow the same
path and the set of resources as the service packets mapped to the path and set of resources as the service packets mapped to the
NRP.</t> NRP.</t>
</li>
<li>
<t>Instrumentation of the overlay by the customer. This is likely <t>Instrumentation of the overlay by the customer. This is likely
to be transparent to the network operator and to use existing to be transparent to the network operator and to use existing
methods. Particular consideration needs to be given to the need to methods. Particular consideration needs to be given to the need to
verify the various committed performance characteristics.</t> verify the various committed performance characteristics.</t>
</li>
<li>
<t>Instrumentation of the overlay by the service provider to <t>Instrumentation of the overlay by the service provider to
proactively demonstrate that the committed performance is being proactively demonstrate that the committed performance is being
delivered. This needs to be done in a non-intrusive manner, delivered. This needs to be done in a non-intrusive manner,
particularly when the tenant is deploying a performance-sensitive particularly when the tenant is deploying a performance-sensitive
application.</t> application.</t>
</list>A study of OAM in SR networks is documented in <xref </li>
target="RFC8403"/>.</t> </ul>
<t>A study of OAM in SR networks is documented in <xref target="RFC8403"
format="default"/>.</t>
</section> </section>
<section numbered="true" toc="default">
<section title="Telemetry Considerations"> <name>Telemetry Considerations</name>
<t>Network visibility is essential for network operation. Network <t>Network visibility is essential for network operation. Network
telemetry has been considered as an ideal means to gain sufficient telemetry has been considered to be an ideal means to gain sufficient
network visibility with better flexibility, scalability, accuracy, network visibility with better flexibility, scalability, accuracy,
coverage, and performance than conventional OAM technologies.</t> coverage, and performance than conventional OAM technologies.</t>
<t>As defined in <xref target="RFC9232" format="default"/>, the
<t>As defined in <xref target="RFC9232"/>, the objective of Network objective of network telemetry is to acquire network data remotely for
Telemetry is to acquire network data remotely for network monitoring network monitoring and operation. It is a general term for a large set
and operation. It is a general term for a large set of network of network visibility techniques and protocols. Network telemetry
visibility techniques and protocols. Network telemetry addresses the addresses the current network operation issues and enables smooth
current network operation issues and enables smooth evolution toward evolution toward intent-driven autonomous networks. Telemetry can be
intent-driven autonomous networks. Telemetry can be applied on the applied on the forwarding plane, the control plane, and the management
forwarding plane, the control plane, and the management plane in a plane in a network. The following requirements need to be considered
network. Telemetry for enhanced VPN service needs to consider the for telemetry for enhanced VPN service:</t>
following requirements: <list style="symbols"> <ul spacing="normal">
<li>
<t>Collecting data of NRPs for overall performance evaluation and <t>Collecting data of NRPs for overall performance evaluation and
the planning of the enhanced VPN services.</t> the planning of the enhanced VPN services.</t>
</li>
<li>
<t>Collecting data of each enhanced VPN service for monitoring and <t>Collecting data of each enhanced VPN service for monitoring and
analytics of the service characteristics and SLA fulfillment.</t> analytics of the service characteristics and SLA fulfillment.</t>
</list></t> </li>
</ul>
<t>How the telemetry mechanisms could be used or extended for enhanced <t>How the telemetry mechanisms could be used or extended for enhanced
VPN services is out of the scope of this document.</t> VPN services is out of the scope of this document.</t>
</section> </section>
</section> </section>
<section numbered="true" toc="default">
<section title="Operational Considerations"> <name>Operational Considerations</name>
<t>It is expected that NRP-based enhanced VPN services will be <t>It is expected that NRP-based enhanced VPN services will be
introduced in networks which already have conventional VPN services introduced in networks that already have conventional VPN services
deployed. Depending on service requirements, the tenants or the operator deployed. Depending on service requirements, the tenants or the operator
may choose to use a VPN or an enhanced VPN to fulfill a service may choose to use a VPN or an enhanced VPN to fulfill a service
requirement. The information and parameters to assist such a decision requirement. The information and parameters to assist such a decision
needs to be supplied on the management interface between the tenant and needs to be supplied on the management interface between the tenant and
the operator. The management interface and data models as described in the operator. The management interface and data models (as described in
<xref target="sdm-app"/> can be used for the life-cycle management of <xref target="sdm-app" format="default"/>) can be used for the life-cycle
management of
enhanced VPN services, such as service creation, modification, enhanced VPN services, such as service creation, modification,
performance monitoring and decommissioning.</t> performance monitoring, and decommissioning.</t>
<t/>
</section> </section>
<section anchor="security-considerations" numbered="true" toc="default">
<section anchor="security-considerations" title="Security Considerations"> <name>Security Considerations</name>
<t>All types of virtual network require special consideration to be <t>All types of virtual networks require special consideration to be
given to the isolation of traffic belonging to different tenants. That given to the isolation of traffic belonging to different tenants. That
is, traffic belonging to one VPN must not be delivered to end points is, traffic belonging to one VPN must not be delivered to endpoints
outside that VPN. In this regard the enhanced VPN neither introduces, outside that VPN. In this regard, the enhanced VPN neither introduces
nor experiences greater security risks than other VPNs.</t> nor experiences greater security risks than other VPNs.</t>
<t>However, in an enhanced VPN service, the additional service
<t>However, in an enhanced VPN service the additional service
requirements need to be considered. For example, if a service requires a requirements need to be considered. For example, if a service requires a
specific upper bound to latency then it can be damaged by simply specific upper bound to latency, then it can be damaged by simply
delaying the packets through the activities of another tenant, i.e., by delaying the packets through the activities of another tenant, i.e., by
introducing bursts of traffic for other services. In some respects this introducing bursts of traffic for other services. In some respects, this
makes the enhanced VPN more susceptible to attacks since the SLA may be makes the enhanced VPN more susceptible to attacks since the SLA may be
broken. But another view is that the operator must, in any case, preform broken. Another view is that the operator must, in any case, preform
monitoring of the enhanced VPN to ensure that the SLA is met, and this monitoring of the enhanced VPN to ensure that the SLA is met; thus, the op
means that the operator may be more likely to spot the early onset of a erator may be more likely to spot the early onset of a
security attack and be able to take preemptive protective action.</t> security attack and be able to take preemptive protective action.</t>
<t>The measures to address these dynamic security risks must be <t>The measures to address these dynamic security risks must be
specified as part of the specific solution to the isolation requirements specified as part of the specific solution to the isolation requirements
of an enhanced VPN service.</t> of an enhanced VPN service.</t>
<t>While an enhanced VPN service may be sold as offering encryption and <t>While an enhanced VPN service may be sold as offering encryption and
other security features as part of the service, customers would be well other security features as part of the service, customers would be well
advised to take responsibility for their own security requirements advised to take responsibility for their security requirements
themselves possibly by encrypting traffic before handing it off to the themselves, possibly by encrypting traffic before handing it off to the
service provider.</t> service provider.</t>
<t>The privacy of enhanced VPN service customers must be preserved. It <t>The privacy of enhanced VPN service customers must be preserved. It
should not be possible for one customer to discover the existence of should not be possible for one customer to discover the existence of
another customer, nor should the sites that are members of an enhanced another customer nor should the sites that are members of an enhanced
VPN be externally visible.</t> VPN be externally visible.</t>
<t>An enhanced VPN service (even one with traffic isolation requirements <t>An enhanced VPN service (even one with traffic isolation requirements
or with limited interaction with other enhanced VPNs) does not provide or with limited interaction with other enhanced VPNs) does not provide
any additional guarantees of privacy for customer traffic compared to any additional guarantees of privacy for customer traffic compared to
regular VPNs: the traffic within the network may be intercepted and regular VPNs: the traffic within the network may be intercepted and
errors may lead to mis-delivery. Users who wish to ensure the privacy of errors may lead to mis-delivery. Users who wish to ensure the privacy of
their traffic must take their own precautions including end-to-end their traffic must take their own precautions including end-to-end
encryption.</t> encryption.</t>
</section> </section>
<section anchor="iana-considerations" numbered="true" toc="default">
<name>IANA Considerations</name>
<section anchor="iana-considerations" title="IANA Considerations"> <t>This document has no IANA actions.</t>
<t>There are no requested IANA actions.</t>
</section>
<section anchor="contributors" title="Contributors">
<t><figure>
<artwork><![CDATA[
Daniel King
Email: daniel@olddog.co.uk
Adrian Farrel
Email: adrian@olddog.co.uk
Jeff Tantsura
Email: jefftant.ietf@gmail.com
Zhenbin Li
Email: lizhenbin@huawei.com
Qin Wu
Email: bill.wu@huawei.com
Bo Wu
Email: lana.wubo@huawei.com
Daniele Ceccarelli
Email: daniele.ietf@gmail.com
Mohamed Boucadair
Email: mohamed.boucadair@orange.com
Sergio Belotti
Email: sergio.belotti@nokia.com
Haomian Zheng
Email: zhenghaomian@huawei.com ]]></artwork>
</figure></t>
</section> </section>
<section title="Acknowledgements">
<t>The authors would like to thank Charlie Perkins, James N Guichard,
John E Drake, Shunsuke Homma, Luis M. Contreras, and Joel Halpern for
their review and valuable comments.</t>
<t>This work was supported in part by the European Commission funded
H2020-ICT-2016-2 METRO-HAUL project (G.A. 761727).</t>
</section>
</middle> </middle>
<back> <back>
<references title="Normative References">
<?rfc include='reference.RFC.9543'?>
<?rfc ?>
</references>
<references title="Informative References">
<reference anchor="TS23501"
target="https://portal.3gpp.org/desktopmodules/Specifications/S
pecificationDetails.aspx?specificationId=3144">
<front>
<title>3GPP TS23.501</title>
<author>
<organization/>
</author>
<date year=""/>
</front>
</reference>
<reference anchor="TS28530"
target="https://portal.3gpp.org/desktopmodules/Specifications/S
pecificationDetails.aspx?specificationId=3273">
<front>
<title>3GPP TS28.530</title>
<author>
<organization/>
</author>
<date year=""/>
</front>
</reference>
<reference anchor="NGMN-NS-Concept"
target="https://www.ngmn.org/fileadmin/user_upload/161010_NGMN_
Network_Slicing_framework_v1.0.8.pdf">
<front>
<title>NGMN NS Concept</title>
<author>
<organization>hao ,</organization>
</author>
<date year=""/>
</front>
</reference>
<reference anchor="FLEXE"
target="https://www.oiforum.com/wp-content/uploads/2019/01/OIF-
FLEXE-01.0.pdf">
<front>
<title>Flex Ethernet Implementation Agreement</title>
<author>
<organization/>
</author>
<date month="March" year="2016"/> <displayreference target="I-D.ietf-teas-ietf-network-slice-nbi-yang" to="NET
</front> WORK-SLICE-YANG"/>
</reference> <displayreference target="I-D.ietf-teas-nrp-scalability" to="NRP-SCALABILITY
"/>
<reference anchor="TSN" target="https://1.ieee802.org/tsn/"> <displayreference target="I-D.ietf-spring-resource-aware-segments" to="RESOU
<front> RCE-AWARE-SEGMENTS"/>
<title>"Time-Sensitive Networking", IEEE 802.1 Time-Sensitive <displayreference target="I-D.ietf-spring-sr-for-enhanced-vpn" to="SR-ENHANC
Networking (TSN) Task Group</title> ED-VPN"/>
<displayreference target="I-D.ietf-6man-enhanced-vpn-vtn-id" to="IPv6-NRP-OP
<author> TION"/>
<organization/> <displayreference target="I-D.ietf-teas-nrp-yang" to="NRP-YANG"/>
</author> <displayreference target="I-D.ietf-spring-srv6-srh-compression" to="SRv6-SRH
-COMPRESSION"/>
<date month="" year=""/> <references>
</front> <name>References</name>
</reference> <references>
<name>Normative References</name>
<?rfc include='reference.I-D.ietf-teas-actn-vn-yang'?> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9
543.xml"/>
<?rfc include='reference.I-D.ietf-teas-ietf-network-slice-nbi-yang'?> </references>
<references>
<?rfc include='reference.I-D.ietf-teas-nrp-scalability'?> <name>Informative References</name>
<?rfc include='reference.I-D.ietf-spring-resource-aware-segments'?>
<?rfc include='reference.I-D.ietf-spring-sr-for-enhanced-vpn'?>
<?rfc include='reference.I-D.ietf-6man-enhanced-vpn-vtn-id'?>
<?rfc include='reference.I-D.ietf-teas-nrp-yang'?>
<?rfc include='reference.I-D.ietf-spring-srv6-srh-compression'?>
<?rfc include='reference.RFC.2211'?>
<?rfc include='reference.RFC.2764'?>
<?rfc include='reference.RFC.3985'?>
<?rfc include='reference.RFC.4664'?>
<?rfc include='reference.RFC.2475'?>
<?rfc include='reference.RFC.2702'?>
<?rfc include='reference.RFC.3209'?>
<?rfc include='reference.RFC.3931'?>
<?rfc include='reference.RFC.4026'?>
<?rfc include='reference.RFC.4176'?>
<?rfc include='reference.RFC.4364'?>
<?rfc include='reference.RFC.4448'?>
<?rfc include='reference.RFC.4594'?>
<?rfc include='reference.RFC.4655'?>
<?rfc include='reference.RFC.4719'?>
<?rfc include='reference.RFC.5557'?>
<?rfc include='reference.RFC.5654'?>
<?rfc include='reference.RFC.7209'?>
<?rfc include='reference.RFC.7297'?>
<?rfc include='reference.RFC.7399'?>
<?rfc include='reference.RFC.7432'?>
<?rfc include='reference.RFC.7665'?>
<?rfc include='reference.RFC.7926'?> <reference anchor="TS23501" target="https://portal.3gpp.org/desktopmodul
es/Specifications/SpecificationDetails.aspx?specificationId=3144">
<front>
<title>System architecture for the 5G system (5GS)</title>
<author>
<organization>3GPP</organization>
</author>
<date year=""/>
</front>
<seriesInfo name="3GPP TS" value="23.501"/>
</reference>
<?rfc include='reference.RFC.8299'?> <reference anchor="TS28530" target="https://portal.3gpp.org/desktopmodul
es/Specifications/SpecificationDetails.aspx?specificationId=3273">
<front>
<title>Management and orchestration; Concepts, use cases and require
ments</title>
<author>
<organization>3GPP</organization>
</author>
<date year=""/>
</front>
<seriesInfo name="3GPP TS" value="28.530"/>
</reference>
<?rfc include='reference.RFC.8309'?> <reference anchor="NGMN-NS-Concept" target="https://www.ngmn.org/wp-cont
ent/uploads/Publications/2016/161010_NGMN_Network_Slicing_framework_v1.0.8.pdf">
<front>
<title>NGMN 5G P1 Requirements and Architecture Work Stream End-to-E
nd Architecture: Description of Network Slicing Concept</title>
<author>
<organization>NGMN Alliance</organization>
</author>
<date day="14" month="September" year="2016"/>
</front>
<refcontent>Version 1.0.8 (draft)</refcontent>
</reference>
<?rfc include='reference.RFC.8370'?> <reference anchor="FLEXE" target="https://www.oiforum.com/wp-content/upl
oads/2019/01/OIF-FLEXE-01.0.pdf">
<front>
<title>Flex Ethernet Implementation Agreement</title>
<author>
<organization>Optical Internetworking Forum</organization>
</author>
<date month="March" year="2016"/>
</front>
<refcontent>IA # OIF-FLEXE-01.0</refcontent>
</reference>
<?rfc include='reference.RFC.8402'?> <reference anchor="TSN" target="https://1.ieee802.org/tsn/">
<front>
<title>Time-Sensitive Networking (TSN) Task Group</title>
<author>
<organization>IEEE 802.1 Working Group</organization>
</author>
<date month="" year=""/>
</front>
</reference>
<?rfc include='reference.RFC.8403'?> <reference anchor="RFC9731" target="https://www.rfc-editor.org/info/rfc9731">
<front>
<title>
A YANG Data Model for Virtual Network (VN) Operations
</title>
<author initials="Y." surname="Lee" fullname="Young Lee" role="editor">
<organization>Samsung Electronics</organization>
</author>
<author initials="D." surname="Dhody" fullname="Dhruv Dhody" role="editor">
<organization>Huawei</organization>
</author>
<author initials="D." surname="Ceccarelli" fullname="Daniele Ceccarelli">
<organization>Cisco</organization>
</author>
<author initials="I." surname="Bryskin" fullname="Igor Bryskin">
<organization>Individual</organization>
</author>
<author initials="B. Y." surname="Yoon" fullname="Bin Yeong Yoon">
<organization>ETRI</organization>
</author>
<date month="March" year="2025"/>
</front>
<seriesInfo name="RFC" value="9731"/>
<seriesInfo name="DOI" value="10.17487/RFC9731"/>
</reference>
<?rfc include='reference.RFC.8453'?> <xi:include href="https://datatracker.ietf.org/doc/bibxml3/draft-ietf-te as-ietf-network-slice-nbi-yang.xml"/>
<?rfc include='reference.RFC.8466'?> <xi:include href="https://datatracker.ietf.org/doc/bibxml3/draft-ietf-te as-nrp-scalability.xml"/>
<?rfc include='reference.RFC.8491'?> <xi:include href="https://datatracker.ietf.org/doc/bibxml3/draft-ietf-sp ring-resource-aware-segments.xml"/>
<?rfc include='reference.RFC.8577'?> <xi:include href="https://datatracker.ietf.org/doc/bibxml3/draft-ietf-sp ring-sr-for-enhanced-vpn.xml"/>
<?rfc include='reference.RFC.8578'?> <xi:include href="https://datatracker.ietf.org/doc/bibxml3/draft-ietf-6m an-enhanced-vpn-vtn-id.xml"/>
<?rfc include='reference.RFC.8655'?> <xi:include href="https://datatracker.ietf.org/doc/bibxml3/draft-ietf-te as-nrp-yang.xml"/>
<?rfc include='reference.RFC.8660'?> <reference anchor="I-D.ietf-spring-srv6-srh-compression" target="https://datatra
cker.ietf.org/doc/html/draft-ietf-spring-srv6-srh-compression-18">
<front>
<title>Compressed SRv6 Segment List Encoding (CSID)</title>
<author initials="W." surname="Cheng" fullname="Weiqiang Cheng" role="editor">
<organization>China Mobile</organization>
</author>
<author initials="C." surname="Filsfils" fullname="Clarence Filsfils">
<organization>Cisco Systems, Inc.</organization>
</author>
<author initials="Z." surname="Li" fullname="Zhenbin Li">
<organization>Huawei Technologies</organization>
</author>
<author initials="B." surname="Decraene" fullname="Bruno Decraene">
<organization>Orange</organization>
</author>
<author initials="F." surname="Clad" fullname="Francois Clad" role="editor">
<organization>Cisco Systems, Inc.</organization>
</author>
<date month="February" day="6" year="2025"/>
</front>
<seriesInfo name="Internet-Draft" value="draft-ietf-spring-srv6-srh-compression-
23"/>
</reference>
<?rfc include='reference.RFC.8665'?> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2
211.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2
764.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.3
985.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4
664.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2
475.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2
702.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.3
209.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.3
931.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4
026.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4
176.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4
364.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4
448.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4
594.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4
655.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4
719.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.5
557.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.5
654.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7
209.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7
297.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7
399.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7
432.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7
665.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7
926.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
299.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
309.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
370.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
402.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
403.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
453.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
466.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
491.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
577.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
578.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
655.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
660.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
665.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
667.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
939.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
964.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8
986.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9
085.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9
182.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9
256.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9
291.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9
232.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9
375.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9
408.xml"/>
<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9
552.xml"/>
</references>
</references>
<?rfc include='reference.RFC.8667'?> <section numbered="false" toc="default">
<name>Acknowledgements</name>
<t>The authors would like to thank <contact fullname="Charlie Perkins"/>,
<contact fullname="James N. Guichard"/>,
<contact fullname="John E. Drake"/>, <contact fullname="Shunsuke Homma"/>,
<contact fullname="Luis M. Contreras"/>, and <contact fullname="Joel Halpern"/>
for
their review and valuable comments.</t>
<t>This work was supported in part by the European Commission funded
H2020-ICT-2016-2 METRO-HAUL project (G.A. 761727).</t>
</section>
<?rfc include='reference.RFC.8939'?> <section anchor="contributors" numbered="false" toc="default">
<name>Contributors</name>
<?rfc include='reference.RFC.8964'?> <contact fullname="Daniel King">
<address><email>daniel@olddog.co.uk</email></address>
</contact>
<?rfc include='reference.RFC.8986'?> <contact fullname="Adrian Farrel">
<address><email>adrian@olddog.co.uk</email></address>
</contact>
<?rfc include='reference.RFC.9085'?> <contact fullname="Jeff Tantsura">
<address><email>jefftant.ietf@gmail.com</email></address>
</contact>
<?rfc include='reference.RFC.9182'?> <contact fullname="Zhenbin Li">
<address><email>lizhenbin@huawei.com</email></address>
</contact>
<?rfc include='reference.RFC.9256'?> <contact fullname="Qin Wu">
<address><email>bill.wu@huawei.com</email></address>
</contact>
<?rfc include='reference.RFC.9291'?> <contact fullname="Bo Wu">
<address><email>lana.wubo@huawei.com</email></address>
</contact>
<?rfc include='reference.RFC.9232'?> <contact fullname="Daniele Ceccarelli">
<address><email>daniele.ietf@gmail.com</email></address>
</contact>
<?rfc include='reference.RFC.9375'?> <contact fullname="Mohamed Boucadair">
<address><email>mohamed.boucadair@orange.com</email></address>
</contact>
<?rfc include='reference.RFC.9408'?> <contact fullname="Sergio Belotti">
<address><email>sergio.belotti@nokia.com</email></address>
</contact>
<?rfc include='reference.RFC.9552'?> <contact fullname="Haomian Zheng">
</references> <address><email>zhenghaomian@huawei.com</email></address>
</contact>
</section>
</back> </back>
<!---->
</rfc> </rfc>
 End of changes. 362 change blocks. 
1033 lines changed or deleted 1177 lines changed or added

This html diff was produced by rfcdiff 1.48.