<?xml version='1.0' encoding='utf-8'?> version="1.0" encoding="UTF-8"?>

<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
<!ENTITY RFC4761 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4761.xml">
<!ENTITY RFC4762 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4762.xml">
<!ENTITY RFC6074 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6074.xml">
<!ENTITY RFC7041 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7041.xml">
<!ENTITY RFC7432 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7432.xml">
<!ENTITY RFC2119 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml">
<!ENTITY RFC8174 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml">
<!ENTITY I-D.ietf-idr-tunnel-encaps SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml3/reference.I-D.ietf-idr-tunnel-encaps.xml">
<!ENTITY RFC7623 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7623.xml">
<!ENTITY I-D.ietf-bess-evpn-overlay SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml3/reference.I-D.ietf-bess-evpn-overlay.xml">
<!ENTITY RFC7543 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7543.xml">
<!ENTITY RFC4684 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4684.xml">
<!ENTITY RFC7348 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7348.xml">
<!ENTITY RFC7637 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7637.xml">
<!ENTITY RFC4023 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4023.xml">
<!ENTITY RFC6870 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6870.xml">
<!ENTITY RFC3031 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3031.xml">
<!ENTITY I-D.sajassi-bess-evpn-virtual-eth-segment SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml3/reference.I-D.sajassi-bess-evpn-virtual-eth-segment.xml">
]> "rfc2629-xhtml.ent">

<rfc xmlns:xi="http://www.w3.org/2001/XInclude" submissionType="IETF" docName="draft-ietf-bess-dci-evpn-overlay-10"
     category="std" ipr="trust200902"> consensus="true"
     docName="draft-ietf-bess-dci-evpn-overlay-10" number="9014"
     ipr="trust200902" obsoletes="" updates="" xml:lang="en" symRefs="true"
     sortRefs="true" tocInclude="true" version="3">

  <!-- xml2rfc v2v3 conversion 2.45.2 -->
  <!-- Generated by id2xml 1.5.0 on 2020-05-27T19:59:24Z -->
	<?rfc strict="yes"?>
	<?rfc compact="yes"?>
	<?rfc subcompact="no"?>
	<?rfc symrefs="yes"?>
	<?rfc sortrefs="no"?>
	<?rfc text-list-symbols="o++*-"?>
	<!--
   draft-ietf-bess-dci-evpn-overlay-10-manual.txt(13): Warning: Expected an expires
   indication top left, found none
   --><?rfc toc="yes"?>

  <front>

    <title abbrev="Interconnect Solution for EVPN-Overlays">Interconnect Solution for EVPN
    Ethernet VPN (EVPN) Overlay networks</title> Networks</title>
    <seriesInfo name="RFC" value="9014"/>
    <author initials="J." surname="Rabadan" fullname="Jorge Rabadan" role="editor">
      <organization>Nokia</organization>
      <address>
        <postal>
          <street>777 E. Middlefield Road</street>
          <city>Mountain View</city>
          <region>CA</region>
          <code>94043</code>
          <country>USA</country>
        </postal>
        <email>jorge.rabadan@nokia.com</email>
      </address>
    </author>
    <author initials="S." surname="Sathappan" fullname="Senthil Sathappan">
      <organization>Nokia</organization>
      <address>
        <email>senthil.sathappan@nokia.com</email>
      </address>
    </author>
    <author initials="W." surname="Henderickx" fullname="Wim Henderickx">
      <organization>Nokia</organization>
      <address>
        <email>wim.henderickx@nokia.com</email>
      </address>
    </author>
    <author initials="A." surname="Sajassi" fullname="Ali Sajassi">
      <organization>Cisco</organization>
      <address>
        <email>sajassi@cisco.com</email>
      </address>
    </author>
    <author initials="J." surname="Drake" fullname="John Drake">
      <organization>Juniper</organization>
      <address>
        <email>jdrake@juniper.net</email>
      </address>
    </author>
    <date year="2020" year="2021" month="May"/>
    <workgroup>BESS Workgroup</workgroup>
	<abstract><t>

<abstract>
      <t>
   This document describes how Network Virtualization Overlays (NVO) (NVOs) can
   be connected to a Wide Area Network (WAN) in order to extend the
   layer-2
   Layer 2 connectivity required for some tenants. The solution analyzes
   the interaction between NVO networks running Ethernet Virtual Private
   Networks (EVPN) (EVPNs) and other L2VPN Layer 2 VPN (L2VPN) technologies used in the WAN, such as
   Virtual Private LAN Services (VPLS), (VPLSs), VPLS extensions for Provider
   Backbone Bridging (PBB-VPLS), EVPN EVPN, or PBB-EVPN. It also describes how
   the existing technical specifications apply to the Interconnection interconnection
   and extends the EVPN procedures needed in some cases. In particular,
   this document describes how EVPN routes are processed on Gateways
   (GWs) that interconnect EVPN-Overlay and EVPN-MPLS networks, as well
   as the Interconnect Ethernet Segment (I-ES) (I-ES), to provide multi-homing,
   and multihoming. This
   document also describes the use of the Unknown MAC route Route (UMR) to avoid MAC scale issues of a Media Access Control (MAC) scale on Data Center Network Virtualization Edge (NVE) devices.</t>
    </abstract>
  </front>
  <middle>
    <section title="Conventions and Terminology" anchor="sect-1"><t>
   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 <xref target="RFC2119"/> <xref target="RFC8174"/> when, and only when, they appear in all
   capitals, as shown here.</t> anchor="sect-2" numbered="true" toc="default">
      <name>Introduction</name>
      <t>
   AC: Attachment Circuit.</t>

	<t>
   ARP: Address Resolution Protocol.</t>

	<t>
   BUM: refers to Broadcast, Unknown unicast and Multicast traffic.</t>

	<t>
   CE: Customer Equipment.</t>

	<t>
   CFM: Connectivity Fault Management.</t>

	<t>
   DC and DCI: Data Center and Data Center Interconnect.</t>

	<t>
   DC RR(s) and WAN RR(s): refers to the Data Center and Wide Area
   Network Route Reflectors, respectively.</t>

	<t>
   DF and NDF: Designated Forwarder and Non-Designated Forwarder.</t>

	<t>
   EVPN: Ethernet Virtual Private Network, as in
   <xref target="RFC7432"/>.</t>

	<t>
   EVI: EVPN Instance.</t>

	<t>
   EVPN Tunnel binding: refers to a tunnel to a remote PE/NVE for a
   given EVI. Ethernet packets in these bindings are encapsulated with
   the Overlay or MPLS encapsulation and the EVPN label at the bottom of
   the stack.</t>

	<t>
   ES and vES: Ethernet Segment and virtual Ethernet Segment.</t>

	<t>
   ESI: Ethernet Segment Identifier.</t>

	<t>
   GW: Gateway or Data Center Gateway.</t>

	<t>
   I-ES and I-ESI: Interconnect Ethernet Segment and Interconnect Ethernet
   Segment Identifier. An I-ES is defined on the GWs for multi-homing to/from
   the WAN.</t>

	<t>
   MAC-VRF: refers to an EVI instance in a particular node.</t>

	<t>
   MP2P and LSM tunnels: refer to Multi-Point to Point and Label
   Switched Multicast tunnels.</t>

	<t>
   ND: Neighbor Discovery protocol.</t>

	<t>
   NVE: Network Virtualization Edge.</t>

	<t>
   NVGRE: Network Virtualization using Generic Routing Encapsulation.</t>

	<t>
   NVO: refers to Network Virtualization Overlays.</t>

	<t>
   OAM: Operations and Maintenance.</t>

	<t>
   PBB: Provider Backbone Bridging.</t>

	<t>
   PE: Provider Edge.</t>

	<t>
   PW: Pseudowire.</t>

	<t>
   RD: Route-Distinguisher.</t>

	<t>
   RT: Route-Target.</t>

	<t>
   S/C-TAG: refers to a combination of Service Tag and Customer Tag in a
   802.1Q frame.</t>

	<t>
   TOR: Top-Of-Rack switch.</t>

	<t>
   UMR: Unknown MAC Route.</t>

	<t>
   VNI/VSID: refers to VXLAN/NVGRE virtual identifiers.</t>

	<t>
   VPLS: Virtual Private LAN Service.</t>

	<t>
   VSI: Virtual Switch Instance or VPLS instance in a particular PE.</t>

	<t>
   VXLAN: Virtual eXtensible LAN.</t>

	</section>

	<section title="Introduction" anchor="sect-2"><t>
   <xref target="I-D.ietf-bess-evpn-overlay"/> target="RFC8365" format="default"/> discusses the use of Ethernet Virtual Private
   Networks (EVPN) (EVPNs) <xref target="RFC7432"/> target="RFC7432" format="default"/> as the control plane for Network
   Virtualization Overlays (NVO), (NVOs), where VXLAN <xref target="RFC7348"/>, target="RFC7348"
   format="default"/>, NVGRE <xref target="RFC7637"/> target="RFC7637" format="default"/>, or
   MPLS over GRE <xref target="RFC4023"/> target="RFC4023" format="default"/> can be used as
   possible data plane encapsulation options.</t>
      <t>
   While this model provides a scalable and efficient multi-tenant multitenant solution
   within the Data Center, it might not be easily extended to the Wide Area
   Network (WAN) in some cases cases, due to the requirements and existing deployed
   technologies. For instance, a Service Provider might have an already
   deployed Virtual Private LAN Service (VPLS) <xref target="RFC4761"/><xref target="RFC4762"/>, target="RFC4761" format="default"/> <xref target="RFC4762" format="default"/>, VPLS
   extensions for Provider Backbone Bridging (PBB-VPLS) <xref target="RFC7041"/>, target="RFC7041"
   format="default"/>, EVPN
   <xref target="RFC7432"/> target="RFC7432" format="default"/>, or PBB-EVPN <xref target="RFC7623"/>
   target="RFC7623" format="default"/> network that has to be used to
   interconnect
   Data Centers and WAN VPN users. A Gateway (GW) function is required in
   these cases. In fact, <xref target="I-D.ietf-bess-evpn-overlay"/> target="RFC8365" format="default"/> discusses two main Data Center
   Interconnect (DCI) solution groups: "DCI using GWs" and "DCI using ASBRs". This
   document specifies the solutions that correspond to the "DCI using GWs"
   group.</t>
      <t>
   It is assumed that the NVO Gateway (GW) GW and the WAN Edge functions
   can be decoupled in into two separate systems or integrated into the same
   system. The former option will be referred to as "Decoupled Interconnect "decoupled interconnect
   solution" throughout the document, whereas the latter one will be
   referred to as "Integrated Interconnect "integrated interconnect solution".</t>
      <t>
   The specified procedures are local to the redundant GWs connecting a
   DC to the WAN. The document does not preclude any combination across
   different DCs for the same tenant. For instance, a "Decoupled"
   solution can be used in GW1 and GW2 (for DC1) DC1), and an "Integrated"
   solution can be used in GW3 and GW4 (for DC2).</t>
      <t>
   While the Gateways and WAN PEs Provider Edges (PEs) use existing specifications in some
   cases, the document also defines extensions that are specific to DCI.
   In particular, those extensions are:</t>

	<t><list style="symbols"><t>The
      <ul spacing="normal">
        <li>The Interconnect Ethernet Segment (I-ES), an Ethernet Segment that
     can be associated to a set of PWs pseudowires (PWs) or other tunnels. The I-ES defined in
     this document is not associated with a set of Ethernet links, as
     per <xref target="RFC7432" format="default"/>, but rather with a set of virtual tunnels (e.g., a
     set of PWs). This set of virtual tunnels is referred to as vES
     <xref target="I-D.ietf-bess-evpn-virtual-eth-segment" format="default"/>.</li>
        <li>The use of the Unknown MAC Route (UMR) in a DCI scenario.</li>
        <li>The processing of EVPN routes on Gateways with MAC-VRFs connecting
     EVPN-Overlay and EVPN-MPLS networks, or EVPN-Overlay and EVPN-Overlay
     networks.</li>
      </ul>
    </section>
    <section anchor="sect-1" numbered="true" toc="default">
      <name>Conventions and Terminology</name>
        <t>
    The key words "<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>", "<bcp14>REQUIRED</bcp14>", "<bcp14>SHALL</bcp14>", "<bcp14>SHALL
    NOT</bcp14>", "<bcp14>SHOULD</bcp14>", "<bcp14>SHOULD NOT</bcp14>", "<bcp14>RECOMMENDED</bcp14>", "<bcp14>NOT RECOMMENDED</bcp14>",
    "<bcp14>MAY</bcp14>", and "<bcp14>OPTIONAL</bcp14>" in this document are to be interpreted as
    described in BCP&nbsp;14 <xref target="RFC2119"/> <xref target="RFC8174"/>
    when, and only when, they appear in all capitals, as shown here.
        </t>
<dl>
      <dt>AC:</dt><dd> Attachment Circuit</dd>
      <dt>ARP:</dt><dd>Address Resolution Protocol</dd>
      <dt>BUM:</dt><dd>Broadcast, Unknown Unicast and Multicast (traffic)</dd>
      <dt>CE:</dt><dd>Customer Equipment</dd>
      <dt>CFM:</dt><dd>Connectivity Fault Management</dd>
      <dt>DC:</dt><dd>Data Center</dd>
      <dt>DCI:</dt><dd>Data Center Interconnect</dd>
      <dt>DF:</dt><dd>Designated Forwarder</dd>
      <dt>EVI:</dt><dd>EVPN Instance</dd>
      <dt>EVPN:</dt><dd>Ethernet Virtual Private Network, as in <xref
      target="RFC7432" format="default"/></dd>
      <dt>EVPN Tunnel binding:</dt><dd>refers to a tunnel to a remote PE/NVE
      for a given EVI. Ethernet packets in these bindings are encapsulated with
   the Overlay or MPLS encapsulation and the EVPN label at the bottom of
   the stack.</dd>
      <dt>ES:</dt><dd>Ethernet Segment</dd>
      <dt>ESI:</dt><dd>Ethernet Segment Identifier</dd>
      <dt>GW:</dt><dd>Gateway or Data Center Gateway</dd>
      <dt>I-ES and I-ESI:</dt><dd>Interconnect Ethernet Segment and
      Interconnect Ethernet Segment Identifier. An I-ES is defined on the GWs
      for multihoming to/from the WAN.</dd>
      <dt>MAC</dt><dd>Media Access Control</dd>
      <dt>MAC-VRF:</dt><dd>refers to an EVI instance in
     this document is not associated with a set of Ethernet links, as
     per <xref target="RFC7432"/>, but rather with particular node</dd>
      <dt>MP2P and LSM tunnels:</dt><dd>refer to multipoint-to-point and label
      switched multicast tunnels</dd>
      <dt>ND:</dt><dd>Neighbor Discovery</dd>
      <dt>NDF:</dt><dd>Non-Designated Forwarder</dd>
      <dt>NVE:</dt><dd>Network Virtualization Edge</dd>
      <dt>NVGRE:</dt><dd>Network Virtualization using Generic Routing
      Encapsulation</dd>
      <dt>NVO:</dt><dd>Network Virtualization Overlay</dd>
      <dt>OAM:</dt><dd>Operations, Administration, and Maintenance</dd>
      <dt>PBB:</dt><dd>Provider Backbone Bridging</dd>
      <dt>PE:</dt><dd>Provider Edge</dd>
      <dt>PW:</dt><dd>Pseudowire</dd>
      <dt>RD:</dt><dd>Route Distinguisher</dd>
       <dt>RR:</dt><dd>Route Reflector</dd>
      <dt>RT:</dt><dd>Route Target</dd>
      <dt>S/C-TAG:</dt><dd>refers to a set combination of virtual tunnels (e.g., Service Tag and Customer
      Tag in a
     set of PWs). This set of virtual tunnels is referred to as vES
     <xref target="I-D.sajassi-bess-evpn-virtual-eth-segment"/>.</t>

	<t>The use of the Unknown 802.1Q frame</dd>
      <dt>TOR:</dt><dd>Top-Of-Rack</dd>
      <dt>UMR:</dt><dd>Unknown MAC route Route</dd>
      <dt>vES:</dt><dd>virtual Ethernet Segment</dd>
      <dt>VNI/VSID:</dt><dd>refers to VXLAN/NVGRE virtual identifiers</dd>
      <dt>VPLS:</dt><dd>Virtual Private LAN Service</dd>
      <dt>VSI:</dt><dd>Virtual Switch Instance or VPLS instance in a DCI scenario.</t>

	<t>The processing of EVPN routes on Gateways with MAC-VRFs connecting
     EVPN-Overlay and EVPN-MPLS networks, or EVPN-Overlay and EVPN-Overlay
     networks.</t>

	</list>
	</t>
      particular PE</dd>
      <dt>VXLAN:</dt><dd>Virtual eXtensible LAN</dd>
</dl>
    </section>

    <section title="Decoupled anchor="sect-3" numbered="true" toc="default">
      <name>Decoupled Interconnect solution Solution for EVPN overlay networks" anchor="sect-3"><t> EVPN-Overlay Networks</name>
      <t>
   This section describes the interconnect solution when the GW and WAN
   Edge functions are implemented in different systems. Figure 1 <xref target="fig-1"/> depicts
   the reference model described in this section. Note that, although
   not shown in Figure 1, <xref target="fig-1"/>, GWs may have local ACs (Attachment Circuits).</t> Attachment Circuits
      (ACs).</t>
      <figure title="Decoupled anchor="fig-1">
        <name>Decoupled Interconnect model" anchor="ure-decoupled-interconnect-model"><artwork><![CDATA[ Model</name>
        <artwork name="" type="" align="left" alt=""><![CDATA[
                                +--+
                                |CE|
                                +--+
                                  |
                               +----+
                          +----| PE |----+
        +---------+       |    +----+    |       +---------+
+----+  |        +---+  +----+        +----+  +---+        |  +----+
|NVE1|--|        |   |  |WAN |        |WAN |  |   |        |--|NVE3|
+----+  |        |GW1|--|Edge|        |Edge|--|GW3|        |  +----+
        |        +---+  +----+        +----+  +---+        |
        |  NVO-1   |      |     WAN      |      |   NVO-2  |
        |        +---+  +----+        +----+  +---+        |
        |        |   |  |WAN |        |WAN |  |   |        |
+----+  |        |GW2|--|Edge|        |Edge|--|GW4|        |  +----+
|NVE2|--|        +---+  +----+        +----+  +---+        |--|NVE4|
+----+  +---------+       |              |       +---------+  +----+
                          +--------------+

|<-EVPN-Overlay-->|<-VLAN->|<-WAN L2VPN->|<--PW-->|<--EVPN-Overlay->|
                   hand-off               hand-off
                   handoff               handoff
]]></artwork>
      </figure>
      <t>
   The following section describes the interconnect requirements for
   this model.</t>
      <section title="Interconnect requirements" anchor="sect-3.1"><t> anchor="sect-3.1" numbered="true" toc="default">
        <name>Interconnect Requirements</name>
        <t>
   The Decoupled Interconnect decoupled interconnect architecture is intended to be deployed in
   networks where the EVPN-Overlay and WAN providers are different
   entities and a clear demarcation is needed. This solution solves the
   following requirements:</t>

	<t><list style="symbols"><t>A
        <ul spacing="normal">
          <li>A simple connectivity hand-off handoff between the EVPN-Overlay network
     provider and the WAN provider so that QoS and security enforcement
     is
     are easily accomplished.</t>

	<t>Independence accomplished.</li>
          <li>Independence of the Layer Two VPN (L2VPN) L2VPN technology deployed in
     the WAN.</t>

	<t>Multi-homing WAN.</li>
          <li>Multihoming between GW and WAN Edge routers, including per-service
     load balancing. Per-flow load balancing is not a strong requirement requirement,
     since a deterministic path per service is usually required for an
     easy QoS and security enforcement.</t>

	<t>Support enforcement.</li>
          <li>Support of Ethernet OAM and Connectivity Fault Management (CFM)
     <xref target="IEEE.802.1AG"/><xref target="Y.1731"/> target="IEEE.802.1AG" format="default"/> <xref target="Y.1731" format="default"/> functions between the GW and the WAN Edge router
     to detect individual AC failures.</t> failures.</li>
          <li>
            <t>Support for the following optimizations at the GW:<list style="symbols"><t>Flooding GW:</t>
            <ul spacing="normal">
              <li>Flooding reduction of unknown unicast traffic sourced from the DC
        Network Virtualization Edge devices (NVEs).</t>

	<t>Control (NVE) devices.</li>
              <li>Control of the WAN MAC addresses advertised to the DC.</t>

	<t>Address DC.</li>
              <li>Address Resolution Protocol (ARP) and Neighbor Discovery (ND)
        flooding control for the requests coming from the WAN.</t>

	</list>
	</t>

	</list>
	</t> WAN.</li>
            </ul>
          </li>
        </ul>
      </section>
      <section title="VLAN-based hand-off" anchor="sect-3.2"><t> anchor="sect-3.2" numbered="true" toc="default">
        <name>VLAN-Based Handoff</name>
        <t>
   In this option, the hand-off handoff between the GWs and the WAN Edge routers
   is based on VLANs <xref target="IEEE.802.1Q-2014"/>. target="IEEE.802.1Q" format="default"/>. This
   is illustrated in Figure 1 <xref target="fig-1"/>
   (between the GWs in NVO-1 and the WAN Edge routers). Each MAC-VRF in
   the GW is connected to a different VSI/MAC-VRF instance in the WAN
   Edge router by using a different C-TAG VLAN ID or a different
   combination of S/C-TAG VLAN IDs that matches at both sides.</t>
        <t>
   This option provides the best possible demarcation between the DC and
   WAN providers providers, and it does not require control plane interaction
   between both providers. The disadvantage of this model is the
   provisioning overhead overhead, since the service has to be mapped to a C-TAG
   or S/C-TAG VLAN ID combination at both GW and WAN Edge routers.</t>
        <t>
   In this model, the GW acts as a regular Network Virtualization Edge
   (NVE) towards the DC. Its control plane, data plane procedures procedures, and
   interactions are described in <xref target="I-D.ietf-bess-evpn-overlay"/>.</t> target="RFC8365" format="default"/>.</t>
        <t>
   The WAN Edge router acts as a (PBB-)VPLS or (PBB-)EVPN PE with
   attachment circuits
   Attachment Circuits (ACs) to the GWs.

   Its functions are described in
   <xref target="RFC4761"/>, target="RFC4761" format="default"/>, <xref target="RFC4762"/>, target="RFC4762"
   format="default"/>, <xref target="RFC6074"/> or target="RFC6074" format="default"/>, <xref target="RFC7432"/>,
   target="RFC7432" format="default"/>, and <xref target="RFC7623"/>.</t> target="RFC7623"
   format="default"/>.</t>
      </section>
      <section title="PW-based (Pseudowire-based) hand-off" anchor="sect-3.3"><t> anchor="sect-3.3" numbered="true" toc="default">
        <name>PW-Based Handoff</name>
        <t>
   If MPLS between the GW and the WAN Edge router is an option, a PW-based
   Interconnect
   interconnect solution can be deployed. In this option option, the hand-off handoff between
   both routers is based on FEC128-based PWs <xref target="RFC4762"/> target="RFC4762" format="default"/> or FEC129-based PWs
   (for a greater level of network automation) <xref target="RFC6074"/>. target="RFC6074" format="default"/>. Note that this model
   still provides a clear demarcation boundary between DC and WAN (since there
   is a single PW between each MAC-VRF and peer VSI), and security/QoS
   policies may be applied on a per PW per-PW basis. This model provides better
   scalability than a C-TAG based hand-off C-TAG-based handoff and less provisioning overhead than
   a combined C/S-TAG hand-off. handoff. The PW-based hand-off handoff interconnect is
   illustrated in Figure 1 <xref target="fig-1"/> (between the NVO-2 GWs and the WAN Edge routers).</t>
        <t>
   In this model, besides the usual MPLS procedures between GW and WAN
   Edge router <xref target="RFC3031"/>, target="RFC3031" format="default"/>, the GW MUST <bcp14>MUST</bcp14> support an interworking function
   in each MAC-VRF that requires extension to the WAN:</t>

	<t><list style="symbols"><t>If
        <ul spacing="normal">
          <li>If a FEC128-based PW is used between the MAC-VRF (GW) and the VSI (WAN
     Edge), the corresponding VCID MUST Virtual Connection Identifier (VCID) <bcp14>MUST</bcp14> be provisioned on the MAC-VRF and
     match the VCID used in the peer VSI at the WAN Edge router.</t>

	<t>If router.</li>
          <li>If BGP Auto-discovery <xref target="RFC6074"/> target="RFC6074" format="default"/> and FEC129-based PWs are used
     between the GW MAC-VRF and the WAN Edge VSI, the provisioning of
     the VPLS-ID MUST <bcp14>MUST</bcp14> be supported on the MAC-VRF and MUST <bcp14>MUST</bcp14> match the
     VPLS-ID used in the WAN Edge VSI.</t>

	</list>
	</t> VSI.</li>
        </ul>
        <t>
   If a PW-based handoff is used, the GW's AC (or point of attachment to
   the EVPN Instance) instance) uses a combination of a PW label and VLAN IDs. PWs
   are treated as service interfaces interfaces, defined in <xref target="RFC7432"/>.</t> target="RFC7432" format="default"/>.</t>
      </section>
      <section title="Multi-homing solution anchor="sect-3.4" numbered="true" toc="default">
        <name>Multihoming Solution on the GWs" anchor="sect-3.4"><t> GWs</name>
        <t>
   EVPN single-active multi-homing, i.e. multihoming -- i.e., per-service load-balancing
   multi-homing
   multihoming -- is required in this type of interconnect.</t>
        <t>

   The GWs will be provisioned with a unique ES per for each WAN interconnect,
   and the hand-off handoff attachment circuits or PWs between the GW and the
   WAN Edge router will be assigned an ESI for each such ES. The ESI will be
   administratively configured on the GWs according to the procedures in
   <xref target="RFC7432"/>. target="RFC7432" format="default"/>. This Interconnect ES I-ES will be
   referred to as "I-ES" hereafter,
   and its identifier will be referred to as "I-ESI". <xref target="RFC7432"/> describes
   different Different ESI Types. types are described in <xref target="RFC7432"
   format="default"/>. The use of Type 0 for the I-ESI is RECOMMENDED <bcp14>RECOMMENDED</bcp14>
   in this document.</t>
        <t>
   The solution (on the GWs) MUST <bcp14>MUST</bcp14> follow the single-active multi-homing multihoming
   procedures as described in <xref target="I-D.ietf-bess-evpn-overlay"/> target="RFC8365" format="default"/> for
   the provisioned I-ESI,
   i.e. I-ESI -- i.e., Ethernet A-D routes per ES and per EVI will be advertised to the DC
   NVEs for the multi-homing multihoming functions, and ES routes will be advertised so that
   ES discovery and Designated Forwarder (DF) procedures can be followed. The
   MAC addresses learned (in the data plane) on the hand-off handoff links will be
   advertised with the I-ESI encoded in the ESI field.</t>
      </section>
      <section title="Gateway Optimizations" anchor="sect-3.5"><t> anchor="sect-3.5" numbered="true" toc="default">
        <name>Gateway Optimizations</name>
        <t>
   The following GW features are optional and optimize the control plane
   and data plane in the DC.</t>
        <section title="MAC anchor="sect-3.5.1" numbered="true" toc="default">
          <name>MAC Address Advertisement Control" anchor="sect-3.5.1"><t> Control</name>
          <t>
   The use of EVPN in NVO networks brings a significant number of
   benefits
   benefits, as described in <xref target="I-D.ietf-bess-evpn-overlay"/>. target="RFC8365" format="default"/>. However, if multiple DCs
   are interconnected into a single EVI, each DC will have to import all
   of the MAC addresses from each of the other DCs.</t>
          <t>
   Even if optimized BGP techniques like RT-constraint RT constraint <xref target="RFC4684"/> target="RFC4684" format="default"/> are
   used, the number of MAC addresses to advertise or withdraw (in case
   of failure) by the GWs of a given DC could overwhelm the NVEs within
   that DC, particularly when the NVEs reside in the hypervisors.</t>
          <t>
   The solution specified in this document uses the 'Unknown Unknown MAC Route' Route (UMR)
   which
   that is advertised into a given DC by each of the DC's GWs.  This route is
   defined in <xref target="RFC7543"/> target="RFC7543" format="default"/> and is a regular EVPN MAC/IP Advertisement route in
   which the MAC Address Length is set to 48, the MAC address is set to 0, and
   the ESI field is set to the DC GW's I-ESI.</t>
          <t>
   An NVE within that DC that understands and process processes the UMR will send
   unknown unicast frames to one of the DCs DC's GWs, which will then forward
   that packet to the correct egress PE. Note that, because the ESI is
   set to the DC GW's I-ESI, all-active multi-homing multihoming can be applied to
   unknown unicast MAC addresses. An NVE that does not understand the
   Unknown MAC route Route will handle unknown unicast as described in
   <xref target="RFC7432"/>.</t> target="RFC7432" format="default"/>.</t>
          <t>
   This document proposes that local policy determines determine whether MAC
   addresses and/or the UMR are advertised into a given DC. As an
   example, when all the DC MAC addresses are learned in the
   control/management plane, it may be appropriate to advertise only the
   UMR. Advertising all the DC MAC addresses in the control/management
   plane is usually the case when the NVEs reside in hypervisors. Refer
   to <xref target="I-D.ietf-bess-evpn-overlay"/> section 7.</t> target="RFC8365" sectionFormat="comma" section="7"/>.</t>
          <t>
   It is worth noting that the UMR usage in <xref target="RFC7543"/> target="RFC7543"
   format="default"/> and the UMR usage in
   this document are different. In the former, a Virtual Spoke (V-spoke) does
   not necessarily learn all the MAC addresses pertaining to hosts in other
   V-spokes of the same network. The communication between two V-spokes is
   done through the DMG, Default MAC Gateway (DMG) until the V-spokes learn each other's MAC
   addresses. In this document, two leaf switches in the same DC are
   recommended for V-spokes to learn each other's MAC addresses for the same EVI. The leaf
   to leaf
	  leaf-to-leaf communication is always direct and does not go through
	  the GW.</t>
        </section>
        <section title="ARP/ND flooding control" anchor="sect-3.5.2"><t> anchor="sect-3.5.2" numbered="true" toc="default">
          <name>ARP/ND Flooding Control</name>
          <t>
   Another optimization mechanism, naturally provided by EVPN in the
   GWs, is the Proxy ARP/ND function. The GWs should build a Proxy
   ARP/ND cache table table, as per <xref target="RFC7432"/>. target="RFC7432" format="default"/>. When
   the active GW receives an
   ARP/ND request/solicitation coming from the WAN, the GW does a Proxy
   ARP/ND table lookup and replies as long as the information is
   available in its table.</t>
          <t>
   This mechanism is especially recommended on the GWs, since it
   protects the DC network from external ARP/ND-flooding storms.</t>
        </section>
        <section title="Handling failures anchor="sect-3.5.3" numbered="true" toc="default">
          <name>Handling Failures between GW and WAN Edge routers" anchor="sect-3.5.3"><t> Routers</name>
          <t>
   Link/PE failures are handled on the GWs as specified in <xref target="RFC7432"/>. target="RFC7432" format="default"/>.
   The GW detecting the failure will withdraw the EVPN routes routes, as per
   <xref target="RFC7432"/>.</t> target="RFC7432" format="default"/>.</t>
          <t>
   Individual AC/PW failures may be detected by OAM mechanisms. For
   instance:</t>

	<t><list style="symbols"><t>If
          <ul spacing="normal">
            <li>If the Interconnect interconnect solution is based on a VLAN hand-off, handoff, Ethernet-CFM
     <xref target="IEEE.802.1AG"/><xref target="Y.1731"/> target="IEEE.802.1AG" format="default"/> <xref target="Y.1731" format="default"/> may be used to detect individual AC failures on both, both
     the GW and WAN Edge router. An individual AC failure will trigger the
     withdrawal of the corresponding A-D per EVI route as well as the MACs
     learned on that AC.</t>

	<t>If AC.</li>
            <li>If the Interconnect interconnect solution is based on a PW hand-off, handoff, the Label
     Distribution Protocol (LDP) PW Status bits TLV <xref target="RFC6870"/> target="RFC6870" format="default"/> may be
     used to detect individual PW failures on both, both the GW and WAN Edge
     router.</t>

	</list>
	</t>
     router.</li>
          </ul>
        </section>
      </section>
    </section>
    <section title="Integrated anchor="sect-4" numbered="true" toc="default">
      <name>Integrated Interconnect solution Solution for EVPN overlay networks" anchor="sect-4"><t> EVPN-Overlay Networks</name>
      <t>
   When the DC and the WAN are operated by the same administrative
   entity, the Service Provider can decide to integrate the GW and WAN
   Edge PE functions in the same router for obvious CAPEX reasons to save as relates to Capital Expenditure (CAPEX) and OPEX
   saving reasons. Operating Expenses (OPEX). This is illustrated in Figure 2. <xref target="fig-2"/>. Note that this
   model
   does not provide an explicit demarcation link between DC and WAN
   anymore. Although not shown in Figure 2, <xref target="fig-2"/>, note that the GWs may have
   local ACs.</t>
      <figure title="Integrated anchor="fig-2">
        <name>Integrated Interconnect model" anchor="ure-integrated-interconnect-model"><artwork><![CDATA[ Model</name>
        <artwork name="" type="" align="left" alt=""><![CDATA[
                          +--+
                          |CE|
                          +--+
                            |
                         +----+
                    +----| PE |----+
        +---------+ |    +----+    | +---------+
+----+  |        +---+            +---+        |  +----+
|NVE1|--|        |   |            |   |        |--|NVE3|
+----+  |        |GW1|            |GW3|        |  +----+
        |        +---+            +---+        |
        |  NVO-1   |       WAN      |   NVO-2  |
        |        +---+            +---+        |
        |        |   |            |   |        |
+----+  |        |GW2|            |GW4|        |  +----+
|NVE2|--|        +---+            +---+        |--|NVE4|
+----+  +---------+ |              | +---------+  +----+
                    +--------------+

|<--EVPN-Overlay--->|<-----VPLS--->|<---EVPN-Overlay-->|
                    |<--PBB-VPLS-->|
  Interconnect  ->  |<-EVPN-MPLS-->|
   options          |<--EVPN-Ovl-->|*
                    |<--PBB-EVPN-->|
]]></artwork>
	</figure>
	<t>&#42;

* EVPN-Ovl stands for EVPN-Overlay (and it's an Interconnect option).</t> interconnect option).
]]></artwork>
      </figure>
      <section title="Interconnect requirements" anchor="sect-4.1"><t> anchor="sect-4.1" numbered="true" toc="default">
        <name>Interconnect Requirements</name>
        <t>
   The Integrated Interconnect integrated interconnect solution meets the following
   requirements:</t>

	<t><list style="symbols"><t>Control
        <ul spacing="normal">
          <li>Control plane and data plane interworking between the EVPN-overlay EVPN-Overlay
     network and the L2VPN technology supported in the WAN, irrespective
     of the technology choice, i.e. choice -- i.e., (PBB-)VPLS or (PBB-)EVPN, as
     depicted in Figure 2.</t>

	<t>Multi-homing, <xref target="fig-2"/>.</li>
          <li>Multihoming, including single-active multi-homing multihoming with per-service load
     balancing or all-active multi-homing, i.e. multihoming -- i.e., per-flow load-balancing, load-balancing -- as
     long as the technology deployed in the WAN supports it.</t>

	<t>Support it.</li>
          <li>Support for end-to-end MAC Mobility, Static MAC protection and
     other procedures (e.g. (e.g., proxy-arp) described in <xref target="RFC7432"/> target="RFC7432" format="default"/> as long as
     EVPN-MPLS is the technology of choice in the WAN.</t>

	<t>Independent WAN.</li>
          <li>Independent inclusive multicast trees in the WAN and in the DC.
     That is, the inclusive multicast tree type defined in the WAN does
     not need to be the same as in the DC.</t>

	</list>
	</t> DC.</li>
        </ul>
      </section>
      <section title="VPLS anchor="sect-4.2" numbered="true" toc="default">
        <name>VPLS Interconnect for EVPN-Overlay networks" anchor="sect-4.2"><section title="Control/Data Networks</name>
        <section anchor="sect-4.2.1" numbered="true" toc="default">
          <name>Control/Data Plane setup procedures Setup Procedures on the GWs" anchor="sect-4.2.1"><t> GWs</name>
          <t>
   Regular MPLS tunnels and TLDP/BGP Targeted LDP (tLDP) / BGP sessions will be setup set up to the WAN
   PEs and RRs as per <xref target="RFC4761"/>, target="RFC4761" format="default"/>, <xref target="RFC4762"/>,
   target="RFC4762" format="default"/>, and <xref target="RFC6074"/> target="RFC6074" format="default"/>, and overlay
   tunnels and EVPN will be setup set up as per <xref target="I-D.ietf-bess-evpn-overlay"/>. target="RFC8365" format="default"/>. Note that
   different route-targets route targets for the DC and for the WAN are normally
   required (unless <xref target="RFC4762"/> target="RFC4762" format="default"/> is used in the WAN, in which case no WAN
   route-target
   route target is needed). A single type-1 RD per service may be used.</t>
          <t>
   In order to support multi-homing, multihoming, the GWs will be provisioned with an
   I-ESI (see section 3.4), that <xref target="sect-3.4"/>), which will be unique per for each
   interconnection. The
   I-ES in In this case case, the I-ES will represent the
   group of PWs to the WAN PEs and
   GWs. All the <xref target="RFC7432"/> target="RFC7432" format="default"/> procedures are still
   followed for the I-ES,
   e.g. I-ES -- e.g., any MAC address learned from the WAN will be advertised to the
   DC with the I-ESI in the ESI field.</t>
          <t>
   A MAC-VRF per EVI will be created in each GW. The MAC-VRF will have
   two different types of tunnel bindings instantiated in two different
   split-horizon-groups:</t>

	<t><list style="symbols">

	  <t>
   split-horizon groups:</t>
          <ul spacing="normal">
            <li> VPLS PWs will be instantiated in the "WAN split-horizon-group".</t>

	  <t> WAN split-horizon group.</li>
            <li> Overlay tunnel bindings (e.g. (e.g., VXLAN, NVGRE) will be instantiated
	  in the "DC split-horizon-group".</t>

	</list>
	</t> DC split-horizon group.</li>
          </ul>
          <t>
   Attachment circuits are also supported on the same MAC-VRF (although
   not shown in Figure 2), <xref target="fig-2"/>), but they will not be part of any of the above
   split-horizon-groups.</t>
   split-horizon groups.</t>
          <t>
   Traffic received in a given split-horizon-group split-horizon group will never be
   forwarded to a member of the same split-horizon-group.</t> split-horizon group.</t>
          <t>
   As far as BUM flooding is concerned, a flooding list will be composed
   of the sub-list sublist created by the inclusive multicast routes and the
   sub-list
   sublist created for VPLS in the WAN. BUM frames received from a
   local Attachment Circuit (AC) will be forwarded to the flooding list.
   BUM frames received from the DC or the WAN will be forwarded to the
   flooding list list, observing the split-horizon-group split-horizon group rule described above.</t>
          <t>
   Note that the GWs are not allowed to have an EVPN binding and a PW to
   the same far-end far end within the same MAC-VRF, so that loops and packet
   duplication are avoided. In case a GW can successfully establish
   both,
   both an EVPN binding and a PW to the same far-end PE, the EVPN
   binding will prevail prevail, and the PW will be brought operationally down.</t> down operationally.</t>
          <t>
   The optimizations optimization procedures described in section 3.5 <xref target="sect-3.5"/> can
   also be applied to this model.</t>
        </section>
        <section title="Multi-homing procedures anchor="sect-4.2.2" numbered="true" toc="default">
          <name>Multihoming Procedures on the GWs" anchor="sect-4.2.2"><t> GWs</name>
          <t>
   This model supports single-active multi-homing multihoming on the GWs. All-active
   multi-homing
   multihoming is not supported by VPLS, therefore VPLS; therefore, it cannot be used on
   the GWs.</t>
          <t>
   In this case, for a given EVI, all the PWs in the WAN split-horizon-group split-horizon group
   are assigned to I-ES. All the single-active multi-homing multihoming procedures as
   described by <xref target="I-D.ietf-bess-evpn-overlay"/> target="RFC8365" format="default"/> will be followed for the I-ES.</t>
          <t>
   The non-DF GW for the I-ES will block the transmission and reception
   of all the PWs in the "WAN split-horizon-group" WAN split-horizon group for BUM and unicast
   traffic.</t>
        </section>
      </section>
      <section title="PBB-VPLS anchor="sect-4.3" numbered="true" toc="default">
        <name>PBB-VPLS Interconnect for EVPN-Overlay networks" anchor="sect-4.3"><section title="Control/Data Networks</name>
        <section anchor="sect-4.3.1" numbered="true" toc="default">
          <name>Control/Data Plane setup procedures Setup Procedures on the GWs" anchor="sect-4.3.1"><t> GWs</name>
          <t>
   In this case, there is no impact on the procedures described in
   <xref target="RFC7041"/> target="RFC7041" format="default"/> for the B-component. However However, the
   I-component instances
   become EVI instances with EVPN-Overlay bindings and potentially local
   attachment circuits. A number of MAC-VRF instances can be multiplexed
   into the same B-component instance. This option provides significant
   savings in terms of PWs to be maintained in the WAN.</t>
          <t>
   The I-ESI concept described in section 4.2.1 <xref target="sect-4.2.1"/> will also be
   used for the PBB-VPLS-based Interconnect.</t> interconnect.</t>
          <t>
   B-component PWs and I-component EVPN-overlay EVPN-Overlay bindings established to
   the same far-end far end will be compared. The following rules will be
   observed:</t>

	<t><list style="symbols">

	<t>
          <ul spacing="normal">
            <li> Attempts to setup set up a PW between the two GWs within the B-component
	context will never be blocked.</t>

	<t> blocked.</li>
            <li> If a PW exists between two GWs for the B-component and an attempt
	is made to setup set up an EVPN binding on an I-component linked to that
	B-component, the EVPN binding will be kept operationally down. down operationally. Note
	that the BGP EVPN routes will still be valid but not used.</t>

	<t> used.</li>
            <li> The EVPN binding will only be up and used as long as there is no
	PW to the same far-end far end in the corresponding B-component. The EVPN
	bindings in the I-components will be brought down before the PW in the
	B-component is brought up.</t>

	</list>
	</t> up.</li>
          </ul>
          <t>
   The optimizations optimization procedures described in section 3.5 <xref target="sect-3.5"/> can also be
   applied to this Interconnect interconnect option.</t>
        </section>
        <section title="Multi-homing procedures anchor="sect-4.3.2" numbered="true" toc="default">
          <name>Multihoming Procedures on the GWs" anchor="sect-4.3.2"><t> GWs</name>
          <t>
   This model supports single-active multi-homing multihoming on the GWs. All-active
   multi-homing
   multihoming is not supported by this scenario.</t>
          <t>
   The single-active multi-homing multihoming procedures as described by <xref target="I-D.ietf-bess-evpn-overlay"/> target="RFC8365" format="default"/>
   will be followed for the I-ES for each EVI instance connected to the
   B-component. Note that in this case, for a given EVI, all the EVPN bindings
   in the I-component are assigned to the I-ES. The non-DF GW for the I-ES
   will block the transmission and reception of all the I-component EVPN
   bindings for BUM and unicast traffic. When learning MACs from the WAN, the
   non-DF MUST NOT <bcp14>MUST NOT</bcp14> advertise EVPN MAC/IP routes for those MACs.</t>
        </section>
      </section>
      <section title="EVPN-MPLS anchor="sect-4.4" numbered="true" toc="default">
        <name>EVPN-MPLS Interconnect for EVPN-Overlay networks" anchor="sect-4.4"><t> Networks</name>
        <t>
   If EVPN for MPLS tunnels, EVPN-MPLS hereafter, is tunnels (referred to as "EVPN-MPLS" hereafter) are supported in the
   WAN, an end-to-end EVPN solution can be deployed. The following
   sections describe the proposed solution as well as the its impact
   required
   on the procedures from <xref target="RFC7432"/> procedures.</t> target="RFC7432" format="default"/>.</t>
        <section title="Control Plane setup procedures anchor="sect-4.4.1" numbered="true" toc="default">
          <name>Control plane Setup Procedures on the GWs" anchor="sect-4.4.1"><t> GWs</name>
          <t>
   The GWs MUST <bcp14>MUST</bcp14> establish separate BGP sessions for sending/receiving
   EVPN routes to/from the DC and to/from the WAN. Normally Normally, each GW will
   setup
   set up one BGP EVPN session to the DC RR (or two BGP EVPN sessions if
   there are redundant DC RRs) and one session to the WAN RR (or two
   sessions if there are redundant WAN RRs).</t>
          <t>
   In order to facilitate separate BGP processes for DC and WAN, EVPN
   routes sent to the WAN SHOULD <bcp14>SHOULD</bcp14> carry a different route-distinguisher Route Distinguisher
   (RD) than the EVPN routes sent to the DC. In addition, although
   reusing the same value is possible, different route-targets route targets are
   expected to be handled for the same EVI in the WAN and the DC. Note
   that the EVPN service routes sent to the DC RRs will normally include
   a <xref target="I-D.ietf-idr-tunnel-encaps"/> target="RFC9012" format="default"/> BGP encapsulation extended community with a
   different tunnel type than the one sent to the WAN RRs.</t>
          <t>
   As in the other discussed options, an I-ES and its assigned I-ESI
   will be configured on the GWs for multi-homing. multihoming. This I-ES represents
   the WAN EVPN-MPLS PEs to the DC but also the DC EVPN-Overlay NVEs to
   the WAN. Optionally, different I-ESI values are configured for
   representing the WAN and the DC. If different EVPN-Overlay networks
   are connected to the same group of GWs, each EVPN-Overlay network
   MUST
   <bcp14>MUST</bcp14> get assigned a different I-ESI.</t>
          <t>
   Received EVPN routes will never be reflected on the GWs but instead will be
   consumed and re-advertised (if needed):</t>

	<t><list style="symbols">

	<t>Ethernet
          <ul spacing="normal">
            <li>Ethernet A-D routes, ES routes routes, and Inclusive Multicast routes are
	consumed by the GWs and processed locally for the corresponding <xref
	target="RFC7432"/> procedures.</t> target="RFC7432" format="default"/> procedures.</li>
            <li>
              <t>MAC/IP advertisement routes will be received, imported received and imported, and if they
        become active in the MAC-VRF, the information will be re-advertised as
        new routes with the following fields:

	<list style="symbols">

	  <t>The

              </t>
              <ul spacing="normal">
                <li>The RD will be the GW's RD for the MAC-VRF.</t>

	  <t>The MAC-VRF.</li>
                <li>The ESI will be set to the I-ESI.</t>

	  <t>The I-ESI.</li>
                <li>The Ethernet-tag value will be kept from the received NLRI the
	  received NLRI.</t>

	  <t>The NLRI.</li>
                <li>The MAC length, MAC address, IP Length Length, and IP address values will
	  be kept from the received NLRI.</t>

	  <t>The NLRI.</li>
                <li>The MPLS label will be a local 20-bit value (when sent to the
	  WAN) or a DC-global 24-bit value (when sent to the DC for
	  encapsulations using a VNI).</t>

	  <t>The VNI).</li>
                <li>The appropriate Route-Targets Route Targets (RTs) and <xref
	  target="I-D.ietf-idr-tunnel-encaps"/>
		target="RFC9012" format="default"/> BGP Encapsulation encapsulation extended
	  community will be used according to <xref
	  target="I-D.ietf-bess-evpn-overlay"/>.</t>

	</list>
	</t>

	</list>
	</t> target="RFC8365" format="default"/>.</li>
              </ul>
            </li>
          </ul>
          <t>
   The GWs will also generate the following local EVPN routes that will be
   sent to the DC and WAN, with their corresponding RTs and <xref target="I-D.ietf-idr-tunnel-encaps"/>
   target="RFC9012" format="default"/> BGP
   Encapsulation encapsulation extended community values:</t>

	<t><list style="symbols">

	<t>ES
          <ul spacing="normal">
            <li>ES route(s) for the I-ESI(s).</t>

	<t>Ethernet I-ESI(s).</li>
            <li>Ethernet A-D routes per ES and EVI for the I-ESI(s). The A-D
        per-EVI routes sent to the WAN and the DC will have consistent
        Ethernet-Tag values.</t>

	<t>Inclusive values.</li>
            <li>Inclusive Multicast routes with independent tunnel type tunnel-type value
        for the WAN and DC. E.g. For example, a P2MP LSP Label Switched Path (LSP) may be used in the WAN WAN,
        whereas ingress replication may be used in the DC. The routes
        sent to the WAN and the DC will have a consistent Ethernet-Tag.</t>

	<t>MAC/IP Ethernet-Tag.</li>

            <li>MAC/IP advertisement routes for MAC addresses learned in local
attachment circuits. Note that these routes will not include the
        I-ESI, but ESI=0
I-ESI value in the ESI field. These routes will include a zero ESI or different from 0 a non-zero ESI for local multi-homed multihomed
Ethernet Segments (ES). The routes sent to the WAN and the DC
        will have a consistent Ethernet-Tag.</t>

	</list>
	</t> Ethernet-Tag.</li>
          </ul>
          <t>
   Assuming GW1 and GW2 are peer GWs of the same DC, each GW will generate two
   sets of the above local service routes: Set-DC set-DC will be sent to the DC RRs
   and will include an A-D per EVI, Inclusive Multicast Multicast, and MAC/IP routes for the
   DC encapsulation and RT. Set-WAN will be sent to the WAN RRs and will
   include the same routes but using the WAN RT and encapsulation. GW1 and GW2
   will receive each other's set-DC and set-WAN. This is the expected behavior
   on GW1 and GW2 for locally generated routes:</t>

	<t><list style="symbols">

	<t>Inclusive
          <ul spacing="normal">
            <li>Inclusive multicast routes: when When setting up the flooding lists for a
        given MAC-VRF, each GW will include its DC peer GW only in the
        EVPN-MPLS flooding list (by default) and not the EVPN-Overlay flooding
        list. That is, GW2 will import two Inclusive Multicast routes from GW1
        (from set-DC and set-WAN) but will only consider one of the two,
        having
        giving the set-WAN route higher priority. An administrative option MAY <bcp14>MAY</bcp14>
        change this preference so that the set-DC route is selected first.</t>

	<t>MAC/IP first.</li>
            <li>MAC/IP advertisement routes for local attachment circuits: as As
        above, the GW will select only one, having giving the route from the
        set-WAN a higher priority. As with the Inclusive multicast
        routes, an administrative option MAY <bcp14>MAY</bcp14> change this priority.</t>

	</list>
	</t> priority.</li>
          </ul>
        </section>
        <section title="Data anchor="sect-4.4.2" numbered="true" toc="default">
          <name>Data Plane setup procedures Setup Procedures on the GWs" anchor="sect-4.4.2"><t> GWs</name>
          <t>
   The procedure explained at the end of the previous section will make sure
   there are no loops or packet duplication between the GWs of the same
   EVPN-Overlay network (for frames generated from local ACs) ACs), since only one
   EVPN binding per EVI (or per Ethernet Tag in the case of VLAN-aware bundle
   services) will be setup set up in the data plane between the two nodes. That
   binding will by default be added to the EVPN-MPLS flooding list.</t>
          <t>
   As for the rest of the EVPN tunnel bindings, they will be added to one of
   the two flooding lists that each GW sets up for the same MAC-VRF:</t>

	<t><list style="symbols">

	<t>EVPN-overlay
          <ul spacing="normal">
            <li>EVPN-Overlay flooding list (composed of bindings to the remote NVEs
	or multicast tunnel to the NVEs).</t>

	<t>EVPN-MPLS NVEs).</li>
            <li>EVPN-MPLS flooding list (composed of MP2P or LSM tunnel to the
	remote PEs)</t>

	</list>
	</t> PEs).</li>
          </ul>
          <t>
   Each flooding list will be part of a separate split-horizon-group: split-horizon group:
   the WAN split-horizon-group split-horizon group or the DC split-horizon-group. split-horizon group. Traffic
   generated from a local AC can be flooded to both
   split-horizon-groups.
   split-horizon groups. Traffic from a binding of a split-horizon-group split-horizon group
   can be flooded to the other split-horizon-group split-horizon group and local ACs, but
   never to a member of its own split-horizon-group.</t> split-horizon group.</t>
          <t>
   When either GW1 or GW2 receive receives a BUM frame on an MPLS tunnel tunnel, including an
   ESI label at the bottom of the stack, they will perform an ESI label lookup
   and split-horizon filtering as per <xref target="RFC7432"/> target="RFC7432"
   format="default"/>, in case the ESI label
   identifies a local ESI (I-ESI or any other non-zero nonzero ESI).</t>
        </section>
        <section title="Multi-homing procedure extensions anchor="sect-4.4.3" numbered="true" toc="default">
          <name>Multihoming Procedure Extensions on the GWs" anchor="sect-4.4.3"><t> GWs</name>
          <t>
   This model supports single-active as well as all-active multi-homing.</t> multihoming.</t>
          <t>
   All the <xref target="RFC7432"/> multi-homing target="RFC7432" format="default"/> multihoming procedures
   for the DF election on I-ES(s) I-ES(s), as
   well as the backup-path (single-active) and aliasing (all-active)
   procedures
   procedures, will be followed on the GWs. Remote PEs in the EVPN-MPLS network
   will follow regular <xref target="RFC7432"/> target="RFC7432" format="default"/> aliasing or backup-path procedures for
   MAC/IP routes received from the GWs for the same I-ESI. So will NVEs in the
   EVPN-Overlay network for MAC/IP routes received with the same I-ESI.</t>
          <t>
   As far as the forwarding plane is concerned, by default, the EVPN-Overlay
   network will have an analogous behavior to the access ACs in <xref target="RFC7432"/>
   multi-homed target="RFC7432" format="default"/>
   multihomed Ethernet Segments.</t>

	<t><list style="symbols">
         <t>The forwarding behavior on the GWs is described below:</t>
          <ul spacing="normal">
            <li>
              <t>Single-active multi-homing; multihoming; assuming a WAN split-horizon-group split-horizon group
        (comprised of EVPN-MPLS bindings), a DC split-horizon-group split-horizon group
        (comprised of EVPN-Overlay bindings) bindings), and local ACs on the GWs:

	<list style="symbols">

	  <t>Forwarding

              </t>
              <ul spacing="normal">
                <li>Forwarding behavior on the non-DF: the The non-DF MUST <bcp14>MUST</bcp14> block
           ingress and egress forwarding on the EVPN-Overlay bindings
           associated to the I-ES. The EVPN-MPLS network is considered to
           be the core network network, and the EVPN-MPLS bindings to the remote
           PEs and GWs will be active.</t>

	   <t>Forwarding active.</li>
                <li>Forwarding behavior on the DF: the The DF MUST NOT <bcp14>MUST NOT</bcp14> forward BUM or
	   unicast traffic received from a given split-horizon-group split-horizon group to a
	   member of his its own split-horizon group. Forwarding to other
	   split-horizon-groups
	   split-horizon groups and local ACs is allowed (as long as the ACs
	   are not part of an ES for which the node is non-DF). As per <xref
	   target="RFC7432"/> target="RFC7432" format="default"/> and for split-horizon purposes, when receiving
	   BUM traffic on the EVPN-Overlay bindings associated to an I-ES, the
	   DF GW SHOULD <bcp14>SHOULD</bcp14> add the I-ESI label when forwarding to the peer GW
	   over EVPN-MPLS.</t>

	   <t>When EVPN-MPLS.</li>
                <li>When receiving EVPN MAC/IP routes from the WAN, the non-DF MUST
	   NOT re-originate <bcp14>MUST
	   NOT</bcp14> reoriginate the EVPN routes and advertise them to the DC
	   peers. In the same way, EVPN MAC/IP routes received from the DC
	   MUST NOT
	   <bcp14>MUST NOT</bcp14> be advertised to the WAN peers. This is consistent with
	   <xref target="RFC7432"/> target="RFC7432" format="default"/> and allows the remote
	   PE/NVEs to know who the
	   primary GW is, based on the reception of the MAC/IP routes.</t>

	</list>
	</t>

	</list>
	</t>

	<t><list style="symbols"> routes.</li>
              </ul>
            </li>
          </ul>
          <ul spacing="normal">
            <li>
              <t>All-active multi-homing; multihoming; assuming a WAN split-horizon-group split-horizon group
        (comprised of EVPN-MPLS bindings), a DC split-horizon-group split-horizon group
        (comprised of EVPN-Overlay bindings) bindings), and local ACs on the GWs:

	<list style="symbols">

	  <t>Forwarding

              </t>
              <ul spacing="normal">
                <li>Forwarding behavior on the non-DF: the The non-DF follows the same
           behavior as the non-DF in the single-active case case, but only for BUM
           traffic. Unicast traffic received from a split-horizon-group MUST
           NOT split-horizon group <bcp14>MUST
           NOT</bcp14> be forwarded to a member of its own split-horizon-group split-horizon group but can
           be forwarded normally to the other split-horizon-groups split-horizon groups and local
           ACs. If a known unicast packet is identified as a "flooded" packet,
           the procedures for BUM traffic MUST <bcp14>MUST</bcp14> be followed.</t>

	   <t>Forwarding followed.</li>
                <li>Forwarding behavior on the DF: the The DF follows the same behavior
	   as the DF in the single-active case case, but only for BUM
	   traffic. Unicast traffic received from a split-horizon-group MUST
	   NOT split-horizon group <bcp14>MUST
	   NOT</bcp14> be forwarded to a member of its own split-horizon-group split-horizon group but can
	   be forwarded normally to the other split-horizon-group split-horizon group and local
	   ACs. If a known unicast packet is identified as a "flooded" packet,
	   the procedures for BUM traffic MUST <bcp14>MUST</bcp14> be followed. As per <xref
	   target="RFC7432"/> target="RFC7432" format="default"/> and for split-horizon purposes, when receiving
	   BUM traffic on the EVPN-Overlay bindings associated to an I-ES, the
	   DF GW MUST <bcp14>MUST</bcp14> add the I-ESI label when forwarding to the peer GW over
	   EVPN-MPLS.</t>

	   <t>Contrary
	   EVPN-MPLS.</li>
                <li>Contrary to the single-active multi-homing multihoming case, both DF and
           non-DF re-originate reoriginate and advertise MAC/IP routes received from
           the WAN/DC peers, adding the corresponding I-ESI so that the
           remote PE/NVEs can perform regular aliasing aliasing, as per <xref target="RFC7432"/>.</t>

	</list>
	</t>

	</list>
	</t>
	   target="RFC7432" format="default"/>.</li>
              </ul>
            </li>
          </ul>
          <t>
   The example in Figure 3 <xref target="fig-3"/> illustrates the forwarding of BUM traffic
   originated from an NVE on a pair of all-active multi-homing multihoming GWs.</t>
          <figure title="Multi-homing anchor="fig-3">
            <name>Multihoming BUM forwarding" anchor="ure-multi-homing-bum-forwarding"><artwork><![CDATA[ Forwarding</name>
            <artwork name="" type="" align="left" alt=""><![CDATA[
     |<--EVPN-Overlay--->|<--EVPN-MPLS-->|

             +---------+ +--------------+
      +----+ BUM       +---+             |
      |NVE1+----+----> |   +-+-----+     |
      +----+  | |   DF |GW1| |     |     |
              | |      +-+-+ |     |    ++--+
              | |        |   |     +--> |PE1|
              | +--->X +-+-+ |          ++--+
              |     NDF|   | |           |
      +----+  |        |GW2<-+           |
      |NVE2+--+        +-+-+             |
      +----+  +--------+ |  +------------+
                         v
                       +--+
                       |CE|
                       +--+
]]></artwork>
          </figure>
          <t>
   GW2 is the non-DF for the I-ES and blocks the BUM forwarding. GW1 is
   the DF and forwards the traffic to PE1 and GW2. Packets sent to GW2
   will include the ESI-label ESI label for the I-ES. Based on the ESI-label, ESI label, GW2
   identifies the packets as I-ES-generated packets and will only
   forward them to local ACs (CE in the example) and not back to the
   EVPN-Overlay network.</t>
        </section>
        <section title="Impact anchor="sect-4.4.4" numbered="true" toc="default">
          <name>Impact on MAC Mobility procedures" anchor="sect-4.4.4"><t> Procedures</name>
          <t>
   MAC Mobility procedures described in <xref target="RFC7432"/> target="RFC7432" format="default"/> are not modified by
   this document.</t>
          <t>
   Note that an intra-DC MAC move still leaves the MAC attached to the
   same I-ES, so under the rules of <xref target="RFC7432"/> target="RFC7432" format="default"/>,
   this is not considered a
   MAC mobility Mobility event. Only when the MAC moves from the WAN domain to
   the DC domain (or from one DC to another) will the MAC will be learned
   from a different ES ES, and the MAC Mobility procedures will kick in.</t>
          <t>
   The sticky bit sticky-bit indication in the MAC Mobility extended community MUST <bcp14>MUST</bcp14>
   be propagated between domains.</t>
        </section>
        <section title="Gateway optimizations" anchor="sect-4.4.5"><t> anchor="sect-4.4.5" numbered="true" toc="default">
          <name>Gateway Optimizations</name>
          <t>
   All the Gateway optimizations described in section 3.5 MAY <xref target="sect-3.5"/>
   <bcp14>MAY</bcp14> be applied
   to the GWs when the Interconnect interconnect is based on EVPN-MPLS.</t>
          <t>
   In particular, the use of the Unknown MAC Route, as described in
   section 3.5.1,
   <xref target="sect-3.5.1"/>, solves some transient packet duplication packet-duplication
   issues in
   cases of all-active multi-homing, multihoming, as explained below.</t>
          <t>
   Consider the diagram in Figure 2 <xref target="fig-2"/> for EVPN-MPLS Interconnect interconnect and all-active
   multi-homing,
   multihoming, and the following sequence:</t>

	<t><list style="format (%c)">

	  <t>MAC
          <ol spacing="normal" type="(%c)">
            <li>MAC Address M1 is advertised from NVE3 in EVI-1.</t>

	  <t>GW3 EVI-1.</li>
            <li>GW3 and GW4 learn M1 for EVI-1 and re-advertise M1 to the WAN
	  with I-ESI-2 in the ESI field.</t>

	  <t>GW1 field.</li>
            <li>GW1 and GW2 learn M1 and install GW3/GW4 as next-hops next hops following
	  the EVPN aliasing procedures.</t>

	  <t>Before procedures.</li>
            <li>Before NVE1 learns M1, a packet arrives at NVE1 with destination
	  M1. If the Unknown MAC Route had not been advertised into the DC,
	  NVE1 would have flooded the packet throughout the DC, in particular
	  to both GW1 and GW2. If the same VNI/VSID is used for both known
	  unicast and BUM traffic, as is typically the case, there is no
	  indication in the packet that it is a BUM packet packet, and both GW1 and
	  GW2 would have forwarded it, creating packet duplication. However,
	  because the Unknown MAC Route had been advertised into the DC, NVE1
	  will unicast the packet to either GW1 or GW2.</t>

	  <t>Since GW2.</li>
            <li>Since both GW1 and GW2 know M1, the GW receiving the packet will
	  forward it to either GW3 or GW4.</t>

	</list>
	</t> GW4.</li>
          </ol>
        </section>
        <section title="Benefits anchor="sect-4.4.6" numbered="true" toc="default">
          <name>Benefits of the EVPN-MPLS Interconnect solution" anchor="sect-4.4.6"><t> Solution</name>
          <t>
   The <xref target="I-D.ietf-bess-evpn-overlay"/> "DCI using ASBRs" solution described in <xref target="RFC8365" format="default"/> and the GW solution
   with EVPN-MPLS Interconnect interconnect may be seen similar as similar, since they both
   retain the EVPN attributes between Data Centers and throughout the
   WAN. However However, the EVPN-MPLS Interconnect interconnect solution on the GWs has
   significant benefits compared to the "DCI using ASBRs" solution:</t>

	<t><list style="symbols">

	  <t>As
          <ul spacing="normal">
            <li>As in any of the described GW models, this solution supports the
	  connectivity of local attachment circuits on the GWs. This is not
	  possible in a "DCI using ASBRs" solution.</t>

	  <t>Different solution.</li>
            <li>Different data plane encapsulations can be supported in the DC
	  and the WAN, while a uniform encapsulation is needed in the "DCI
	  using ASBRs" solution.</t>

	  <t>Optimized solution.</li>
            <li>Optimized multicast solution, with independent inclusive
	  multicast trees in DC and WAN.</t>

	  <t>MPLS Label WAN.</li>
            <li>MPLS label aggregation: for For the case where MPLS labels are
	  signaled from the NVEs for MAC/IP Advertisement advertisement routes, this
	  solution provides label aggregation. A remote PE MAY <bcp14>MAY</bcp14> receive a
	  single label per GW MAC-VRF MAC-VRF, as opposed to a label per NVE/MAC-VRF
	  connected to the GW MAC-VRF. For instance, in Figure 2, <xref target="fig-2"/>, PE would
	  receive only one label for all the routes advertised for a given
	  MAC-VRF from GW1, as opposed to a label per NVE/MAC-VRF.</t>

	  <t>The NVE/MAC-VRF.</li>
            <li>The GW will not propagate MAC mobility Mobility for the MACs moving within
	  a DC. Mobility intra-DC is solved by all the NVEs in the DC. The MAC
	  Mobility procedures on the GWs are only required in case of mobility
	  across DCs.</t>

	  <t>Proxy-ARP/ND DCs.</li>
            <li>Proxy-ARP/ND function on the DC GWs can be leveraged to reduce
	  ARP/ND flooding in the DC or/and in the WAN.</t>

	</list>
	</t> WAN.</li>
          </ul>
        </section>
      </section>
      <section title="PBB-EVPN anchor="sect-4.5" numbered="true" toc="default">
        <name>PBB-EVPN Interconnect for EVPN-Overlay networks" anchor="sect-4.5"><t> Networks</name>
        <t>
   PBB-EVPN <xref target="RFC7623"/> target="RFC7623" format="default"/> is yet another Interconnect interconnect option. It requires
   the use of GWs where I-components and associated B-components are
   part of EVI instances.</t>
        <section title="Control/Data anchor="sect-4.5.1" numbered="true" toc="default">
          <name>Control/Data Plane setup procedures Setup Procedures on the GWs" anchor="sect-4.5.1"><t> GWs</name>
          <t>
   EVPN will run independently in both components, the I-component MAC-VRF and
   B-component MAC-VRF. Compared to <xref target="RFC7623"/>, target="RFC7623" format="default"/>,
   the DC C-MACs customer MACs (C-MACs) are no longer
   learned in the data plane on the GW but in the control plane through EVPN
   running on the I-component. Remote C-MACs coming from remote PEs are still
   learned in the data plane. B-MACs in the B-component will be assigned and
   advertised following the procedures described in <xref target="RFC7623"/>.</t> target="RFC7623" format="default"/>.</t>
          <t>
   An I-ES will be configured on the GWs for multi-homing, multihoming, but its I-ESI will
   only be used in the EVPN control plane for the I-component EVI.  No
   non-reserved
   unreserved ESIs will be used in the control plane of the B-component EVI EVI,
   as per <xref target="RFC7623"/>, that target="RFC7623" format="default"/>. That is, the I-ES will be
   represented to the WAN PBB-EVPN
   PEs using shared or dedicated B-MACs.</t>
          <t>
   The rest of the control plane procedures will follow <xref target="RFC7432"/> target="RFC7432" format="default"/> for
   the I-component EVI and <xref target="RFC7623"/> target="RFC7623" format="default"/> for the B-component EVI.</t>
          <t>
   From the data plane perspective, the I-component and B-component EVPN
   bindings established to the same far-end far end will be compared compared, and the
   I-component EVPN-overlay EVPN-Overlay binding will be kept down following the rules
   described in section 4.3.1.</t> <xref target="sect-4.3.1"/>.</t>
        </section>
        <section title="Multi-homing procedures anchor="sect-4.5.2" numbered="true" toc="default">
          <name>Multihoming Procedures on the GWs" anchor="sect-4.5.2"><t> GWs</name>
          <t>
   This model supports single-active as well as all-active multi-homing.</t> multihoming.</t>
          <t>
   The forwarding behavior of the DF and non-DF will be changed based on
   the description outlined in section 4.4.3, only replacing <xref target="sect-4.4.3"/>, substituting the "WAN split-horizon-group"
   WAN split-horizon group for the B-component, and using <xref target="RFC7623"/>
   target="RFC7623" format="default"/>
   procedures for the traffic sent or received on the B-component.</t>
        </section>
        <section title="Impact anchor="sect-4.5.3" numbered="true" toc="default">
          <name>Impact on MAC Mobility procedures" anchor="sect-4.5.3"><t> Procedures</name>
          <t>
   C-MACs learned from the B-component will be advertised in EVPN within
   the I-component EVI scope. If the C-MAC was previously known in the
   I-component database, EVPN would advertise the C-MAC with a higher
   sequence number, as per <xref target="RFC7432"/>. target="RFC7432" format="default"/>. From a Mobility the
   perspective of Mobility and the related procedures described in <xref target="RFC7432"/>,
   target="RFC7432" format="default"/>, the C-MACs learned
   from the B-component are considered local.</t>
        </section>
        <section title="Gateway optimizations" anchor="sect-4.5.4"><t> anchor="sect-4.5.4" numbered="true" toc="default">
          <name>Gateway Optimizations</name>
          <t>
   All the considerations explained in section 4.4.5 <xref target="sect-4.4.5"/> are
   applicable to
   the PBB-EVPN Interconnect interconnect option.</t>
        </section>
      </section>
      <section title="EVPN-VXLAN anchor="sect-4.6" numbered="true" toc="default">
        <name>EVPN-VXLAN Interconnect for EVPN-Overlay networks" anchor="sect-4.6"><t> Networks</name>
        <t>
   If EVPN for Overlay tunnels is supported in the WAN WAN, and a GW function
   is required, an end-to-end EVPN solution can be deployed. While
   multiple Overlay tunnel combinations at the WAN and the DC are
   possible (MPLSoGRE, nvGRE, NVGRE, etc.), VXLAN is described here, given its
   popularity in the industry. This section focuses on the specific case
   of EVPN for VXLAN (EVPN-VXLAN hereafter) and the impact on the
   <xref target="RFC7432"/> target="RFC7432" format="default"/> procedures.</t>
        <t>
   The procedures described in section 4.4 <xref target="sect-4.4"/> apply to this section section, too, only
   replacing
   substituting EVPN-MPLS for EVPN-VXLAN control plane specifics and using
   <xref target="I-D.ietf-bess-evpn-overlay"/> target="RFC8365" format="default"/> "Local Bias" procedures instead
   of section 4.4.3. <xref target="sect-4.4.3"/>. Since
   there are no ESI-labels ESI labels in VXLAN, GWs need to rely on "Local Bias" to apply
   split-horizon
   split horizon on packets generated from the I-ES and sent to the peer GW.</t>
        <t>
   This use-case use case assumes that NVEs need to use the VNIs or VSIDs as a
   globally unique identifiers within a data center, Data Center, and a Gateway needs
   to be employed at the edge of the data center Data-Center network to translate
   the VNI or VSID when crossing the network boundaries. This GW
   function provides VNI and tunnel IP address tunnel-IP-address translation. The use-case use case
   in which local downstream assigned downstream-assigned VNIs or VSIDs can be used (like
   MPLS labels) is described by <xref target="I-D.ietf-bess-evpn-overlay"/>.</t> target="RFC8365" format="default"/>.</t>
        <t>
   While VNIs are globally significant within each DC, there are two
   possibilities in the Interconnect interconnect network:</t>

	<t><list style="format (%c)">

	  <t>Globally
        <ol spacing="normal">
          <li>Globally unique VNIs in the Interconnect network: interconnect network. In this case,
	  the GWs and PEs in the Interconnect interconnect network will agree on a common
	  VNI for a given EVI. The RT to be used in the Interconnect interconnect network
	  can be auto-derived autoderived from the agreed Interconnect agreed-upon interconnect VNI. The VNI used
	  inside each DC MAY <bcp14>MAY</bcp14> be the same as the Interconnect VNI.</t>

	  <t>Downstream assigned interconnect VNI.</li>
          <li>Downstream-assigned VNIs in the Interconnect interconnect network. In this
	  case, the GWs and PEs MUST <bcp14>MUST</bcp14> use the proper RTs to import/export the EVPN routes. Note that even if the VNI is downstream assigned in the
	  Interconnect
	  interconnect network, and unlike option (a), it only identifies the
	  &lt;Ethernet Tag, GW&gt; pair and not the &lt;Ethernet Tag, egress
	  PE&gt; pair. The VNI used inside each DC MAY <bcp14>MAY</bcp14> be the same as the
	  Interconnect
	  interconnect VNI. GWs SHOULD <bcp14>SHOULD</bcp14> support multiple VNI spaces per EVI
	  (one per Interconnect interconnect network they are connected to).
	</t>

	</list>
	</t>
	</li>
        </ol>
        <t>
   In both options, NVEs inside a DC only have to be aware of a single
   VNI space, and only GWs will handle the complexity of managing
   multiple VNI spaces. In addition to VNI translation above, the GWs
   will provide translation of the tunnel source IP for the packets
   generated from the NVEs, using their own IP address. GWs will use
   that IP address as the BGP next-hop next hop in all the EVPN updates to the
   Interconnect
   interconnect network.</t>
        <t>
   The following sections provide more details about these two options.</t>
        <section title="Globally unique anchor="sect-4.6.1" numbered="true" toc="default">
          <name>Globally Unique VNIs in the Interconnect network" anchor="sect-4.6.1"><t> Network</name>
          <t>
   Considering Figure 2, <xref target="fig-2"/>, if a host H1 in NVO-1 needs to communicate with a
   host H2 in NVO-2, and assuming that different VNIs are used in each DC for
   the same EVI, e.g. EVI (e.g., VNI-10 in NVO-1 and VNI-20 in NVO-2, NVO-2), then the VNIs MUST
   <bcp14>MUST</bcp14>
   be translated to a common Interconnect interconnect VNI (e.g. (e.g., VNI-100) on the GWs. Each
   GW is provisioned with a VNI translation mapping so that it can translate
   the VNI in the control plane when sending BGP EVPN route updates to the
   Interconnect
   interconnect network. In other words, GW1 and GW2 MUST <bcp14>MUST</bcp14> be configured to map
   VNI-10 to VNI-100 in the BGP update messages for H1's MAC route.

   This
   mapping is also used to translate the VNI in the data plane in both
   directions,
   directions: that is, VNI- 10 VNI-10 to VNI-100 when the packet is received from
   NVO-1 and the reverse mapping from VNI-100 to VNI-10 when the packet is
   received from the remote NVO-2 network and needs to be forwarded to NVO-1.</t>
          <t>
   The procedures described in section 4.4 <xref target="sect-4.4"/> will be followed,
   considering
   that the VNIs advertised/received by the GWs will be translated
   accordingly.</t>
        </section>
        <section title="Downstream assigned anchor="sect-4.6.2" numbered="true" toc="default">
          <name>Downstream-Assigned VNIs in the Interconnect network" anchor="sect-4.6.2"><t> Network</name>
          <t>
   In this case, if a host H1 in NVO-1 needs to communicate with a host
   H2 in NVO-2, and assuming that different VNIs are used in each DC for
   the same EVI, e.g. e.g., VNI-10 in NVO-1 and VNI-20 in NVO-2, then the VNIs
   MUST
   <bcp14>MUST</bcp14> be translated as in section 4.6.1. <xref
   target="sect-4.6.1"/>. However, in this case, there
   is no need to translate to a common Interconnect interconnect VNI on the GWs. Each
   GW can translate the VNI received in an EVPN update to a locally
   assigned VNI advertised to the Interconnect interconnect network. Each GW can use
   a different Interconnect VNI, hence interconnect VNI; hence, this VNI does not need to be
   agreed upon on all the GWs and PEs of the Interconnect interconnect network.</t>
          <t>
   The procedures described in section 4.4 <xref target="sect-4.4"/> will be followed,
   taking into account the considerations above for the VNI translation.</t>
        </section>
      </section>
    </section>
    <section title="Security Considerations" anchor="sect-5"><t> anchor="sect-5" numbered="true" toc="default">
      <name>Security Considerations</name>
      <t>
   This document applies existing specifications to a number of
   Interconnect
   interconnect models. The Security Considerations security considerations included in those
   documents, such as <xref target="RFC7432"/>, target="RFC7432" format="default"/>, <xref target="I-D.ietf-bess-evpn-overlay"/>,
   target="RFC8365" format="default"/>, <xref target="RFC7623"/>, target="RFC7623"
   format="default"/>, <xref target="RFC4761"/> target="RFC4761" format="default"/>,
   and <xref target="RFC4762"/> target="RFC4762" format="default"/> apply to this document
   whenever those technologies are
   used.</t>
      <t>
   As discussed, <xref target="I-D.ietf-bess-evpn-overlay"/> target="RFC8365" format="default"/> discusses two main DCI solution groups:
   "DCI using GWs" and "DCI using ASBRs". This document specifies the
   solutions that correspond to the "DCI using GWs" group. It is
   important to note that the use of GWs provide provides a superior level of
   security on a per tenant per-tenant basis, compared to the use of ASBRs. This is
   due to the fact that GWs need to perform a MAC lookup on the frames
   being received from the WAN, and they apply security procedures, such
   as filtering of undesired frames, filtering of frames with a source
   MAC that matches a protected MAC in the DC DC, or application of MAC
   duplication
   MAC-duplication procedures defined in <xref target="RFC7432"/>. target="RFC7432"
   format="default"/>. On ASBRs ASBRs, though, traffic
   is forwarded based on a label or VNI swap swap, and there is usually no
   visibility of the encapsulated frames, which can carry malicious
   traffic.</t>
      <t>
   In addition, the GW optimizations specified in this document, document provide
   additional protection of the DC Tenant Systems. tenant systems. For instance, the MAC
   address
   MAC-address advertisement control and Unknown MAC Route defined in
   section 3.5.1
   <xref target="sect-3.5.1"/> protect the DC NVEs from being overwhelmed with an
   excessive number of MAC/IP routes being learned on the GWs from the WAN.
   The ARP/ND flooding control described in 3.5.2 <xref target="sect-3.5.2"/> can reduce/suppress
   broadcast storms being injected from the WAN.</t>
      <t>
   Finally, the reader should be aware of the potential security
   implications of designing a DCI with the Decoupled Interconnect decoupled interconnect
   solution (section 3) (<xref target="sect-3"/>) or the Integrated Interconnect integrated interconnect solution (section
   4).
   (<xref target="sect-4"/>). In the Decoupled Interconnect solution decoupled interconnect solution, the DC is typically easier
   to protect from the WAN, since each GW has a single logical link to
   one WAN PE, whereas in the Integrated solution, the GW has logical
   links to all the WAN PEs that are attached to the tenant. In either
   model, proper control plane and data plane policies should be put in
   place in the GWs in order to protect the DC from potential attacks
   coming from the WAN.</t>
    </section>
    <section title="IANA Considerations" anchor="sect-6"><t> anchor="sect-6" numbered="true" toc="default">
      <name>IANA Considerations</name>
      <t>
   This document has no IANA actions.</t>
    </section>
  </middle>
  <back>
	<references title="Normative References">
	&RFC4761;
	&RFC4762;
	&RFC6074;
	&RFC7041;
	&RFC7432;
	&RFC2119;
	&RFC8174;
	&I-D.ietf-idr-tunnel-encaps;
	&RFC7623;
	&I-D.ietf-bess-evpn-overlay;
	&RFC7543;

<displayreference target="I-D.ietf-bess-evpn-virtual-eth-segment" to="VIRTUAL-ES"/>

    <references>
      <name>References</name>
      <references>
        <name>Normative References</name>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4761.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4762.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6074.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7041.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7432.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/>

	<!-- draft-ietf-idr-tunnel-encaps-15 is now RFC 9012-->
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.9012.xml"/>

        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7623.xml"/>

<!-- [I-D.ietf-bess-evpn-overlay] Published as RFC 8365 -->

        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8365.xml"/>

        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7543.xml"/>

      </references>
	<references title="Informative References">
	&RFC4684;
	&RFC7348;
	&RFC7637;
	&RFC4023;
      <references>
        <name>Informative References</name>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4684.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7348.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7637.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4023.xml"/>

        <reference anchor="Y.1731"><front> anchor="Y.1731">
          <front>
            <title>OAM functions and mechanisms for Ethernet based networks</title>
            <author>
	<organization>ITU-T Recommendation Y.1731</organization>
              <organization>ITU-T</organization>
            </author>
            <date month="July" year="2011"/> month="August" year="2019"/>
          </front>
	  <seriesInfo name="ITU-T Recommendation" value="Y.1731" />
        </reference>

        <reference anchor="IEEE.802.1AG"><front> anchor="IEEE.802.1AG">
          <front>
            <title>IEEE Standard for Local and Metropolitan Area Networks -
	    Virtual Bridged Local Area Networks Amendment 5: Connectivity
	    Fault Management</title>
            <author>
	<organization>IEEE 802.1AG_2007</organization>
              <organization>IEEE</organization>
            </author>
            <date month="January" year="2008"/>
          </front>
	  <seriesInfo name="IEEE standard" value="802.1ag-2007"/>
        </reference>

        <reference anchor="IEEE.802.1Q-2014"><front> anchor="IEEE.802.1Q">
          <front>
            <title>IEEE Standard for Local and metropolitan area
	    networks--Bridges and Bridged Networks</title>
            <author>
	<organization>IEEE 802.1Q-2014</organization>
              <organization>IEEE</organization>
            </author>
            <date month="December" year="2014"/>
          </front>
	  <seriesInfo name="IEEE standard" value="802.1Q-2014" />
	  <seriesInfo name="DOI" value="10.1109/IEEESTD.2014.6991462"/>
        </reference>
	&RFC6870;
	&RFC3031;
	&I-D.sajassi-bess-evpn-virtual-eth-segment;

        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6870.xml"/>
        <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3031.xml"/>

<!-- [I-D.sajassi-bess-evpn-virtual-eth-segment] Replaced by
draft-ietf-bess-evpn-virtual-eth-segment; IESG state I-D Exists -->

        <xi:include href="https://datatracker.ietf.org/doc/bibxml3/reference.I-D.ietf-bess-evpn-virtual-eth-segment.xml"/>

      </references>
    </references>
    <section title="Acknowledgments" anchor="sect-8"><t> anchor="sect-8" numbered="false" toc="default">
      <name>Acknowledgments</name>
      <t>
   The authors would like to thank Neil Hart, Vinod Prabhu and Kiran
   Nagaraj <contact fullname="Neil Hart"/>, <contact
   fullname="Vinod Prabhu"/>, and <contact fullname="Kiran Nagaraj"/> for
   their valuable comments and feedback. We would also like
   to thank Martin Vigoureux and Alvaro Retana <contact fullname="Martin Vigoureux"/> and <contact
   fullname="Alvaro Retana"/> for his their detailed review reviews and comments.</t>
    </section>
    <section title="Contributors" anchor="sect-9"><t> anchor="sect-9" numbered="false" toc="default">
      <name>Contributors</name>
      <t>
   In addition to the authors listed on the front page, the following
   co-authors
   coauthors have also contributed to this document:</t>

	<figure><artwork><![CDATA[
Ravi Shekhar
Anil Lohiya
Wen Lin
Juniper Networks

Florin Balus
Patrice Brissette
Cisco

Senad Palislamovic
Nokia

Dennis Cai
Alibaba
]]></artwork>
	</figure>

<contact fullname="Ravi Shekhar">
<organization>Juniper Networks</organization>
</contact>

<contact fullname="Anil Lohiya">
<organization>Juniper Networks</organization>
</contact>

<contact fullname="Wen Lin">
<organization>Juniper Networks</organization>
</contact>

<contact fullname="Florin Balus">
<organization>Cisco</organization>
</contact>

<contact fullname="Patrice Brissette">
<organization>Cisco</organization>
</contact>

<contact fullname="Senad Palislamovic">
<organization>Nokia</organization>
</contact>

<contact fullname="Dennis Cai">
<organization>Alibaba</organization>
</contact>
    </section>
  </back>

</rfc>