Internet Working Group                                  Ali Engineering Task Force (IETF)                   A. Sajassi, Ed.
Internet Draft                                               Samer
Request for Comments: 7623                                      S. Salam
Category: Standards Track                                          Cisco
                                                             Nabil
ISSN: 2070-1721                                                 N. Bitar
                                                                 Verizon
                                                            Aldrin
                                                                A. Isaac
                                                               Bloomberg
                                                          Wim
                                                                 Juniper
                                                           W. Henderickx
                                                          Alcatel-Lucent
Expires: November 14, 2015                                  May 14,
                                                          September 2015

    Provider Backbone Bridging Combined with Ethernet VPN (PBB-EVPN)
                      draft-ietf-l2vpn-pbb-evpn-10

Abstract

   This document discusses how Ethernet Provider Backbone Bridging (PBB)
   can be combined with Ethernet VPN (EVPN) in order to reduce the
   number of BGP MAC advertisement Advertisement routes by aggregating Customer/Client
   MAC (C-MAC) addresses via Provider Backbone MAC address (B-MAC), (B-MAC) address,
   provide client MAC address mobility using C-MAC aggregation, confine
   the scope of C-MAC learning to only active flows, offer per site
   policies per-site
   policies, and avoid C-MAC address flushing on topology changes.  The
   combined solution is referred to as PBB-EVPN.

Status of this This Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents an Internet Standards Track document.

   This document is a product of the Internet Engineering Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum
   (IETF).  It represents the consensus of six months the IETF community.  It has
   received public review and may be updated, replaced, or obsoleted has been approved for publication by other documents at any
   time.  It the
   Internet Engineering Steering Group (IESG).  Further information on
   Internet Standards is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work available in progress."

   The list Section 2 of RFC 5741.

   Information about the current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list status of Internet-Draft Shadow Directories can this document, any errata,
   and how to provide feedback on it may be accessed obtained at
   http://www.ietf.org/shadow.html
   http://www.rfc-editor.org/info/rfc7623.

Copyright and License Notice

   Copyright (c) 2015 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4 ....................................................3
   2.  Contributors . . . . . . . . . . . . . . . . . . . . . . . . .  4
   3. Terminology  . . . . . . . . . . . . . . . . . . . . . . . . .  4
   4. .....................................................4
   3. Requirements . . . . . . . . . . . . . . . . . . . . . . . . .  5
     4.1. ....................................................4
      3.1. MAC Advertisement Route Scalability  . . . . . . . . . . .  5
     4.2. ........................5
      3.2. C-MAC Mobility Independent of B-MAC Advertisements . . . .  5
     4.3. .........5
      3.3. C-MAC Address Learning and Confinement . . . . . . . . . .  6
     4.4.  Per Site .....................5
      3.4. Per-Site Policy Support  . . . . . . . . . . . . . . . . .  6
     4.5. ....................................6
      3.5. No C-MAC Address Flushing for All-Active Multi-Homing  . .  6
   5. Multihoming .......6
   4. Solution Overview  . . . . . . . . . . . . . . . . . . . . . .  6
   6. ...............................................6
   5. BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . .  7
     6.1. ....................................................7
      5.1. Ethernet Auto-Discovery Route  . . . . . . . . . . . . . .  7
     6.2. ..............................7
      5.2. MAC/IP Advertisement Route . . . . . . . . . . . . . . . .  8
     6.3. .................................7
      5.3. Inclusive Multicast Ethernet Tag Route . . . . . . . . . .  8
     6.4. .....................8
      5.4. Ethernet Segment Route . . . . . . . . . . . . . . . . . .  9
     6.5. .....................................8
      5.5. ESI Label Extended Community . . . . . . . . . . . . . . .  9
     6.6. ...............................8
      5.6. ES-Import Route Target . . . . . . . . . . . . . . . . . .  9
     6.7. .....................................9
      5.7. MAC Mobility Extended Community  . . . . . . . . . . . . .  9
     6.8. ............................9
      5.8. Default Gateway Extended Community . . . . . . . . . . . .  9
   7. .........................9
   6. Operation  . . . . . . . . . . . . . . . . . . . . . . . . . .  9
     7.1. .......................................................9
      6.1. MAC Address Distribution over Core . . . . . . . . . . . . 10
     7.2. .........................9
      6.2. Device Multi-homing  . . . . . . . . . . . . . . . . . . . 10
       7.2.1. Flow-based Load-balancing . . . . . . . . . . . . . . . 10
         7.2.1.1. Multihoming .........................................9
           6.2.1. Flow-Based Load-Balancing ...........................9
                  6.2.1.1. PE B-MAC Address Assignment  . . . . . . . . . . . 10
         7.2.1.2. ...............10
                  6.2.1.2. Automating B-MAC Address Assignment  . . . . . . . 12
         7.2.1.3 .......11
                  6.2.1.3. Split Horizon and Designated
                           Forwarder Election . . 12
       7.2.2. ........................12
           6.2.2. Load-Balancing based on I-SID Based Load-balancing  . . . . . . . . . . . . . . 13
         7.2.2.1. ......................12
                  6.2.2.1. PE B-MAC Address Assignment . . . . . . . . . . . . 13
         7.2.2.2. ...............12
                  6.2.2.2. Split Horizon and Designated
                           Forwarder Election . . 13
         7.2.2.3. ........................13
                  6.2.2.3. Handling Failure Scenarios  . . . . . . . . . . . . 13
     7.3. ................13
      6.3. Network Multi-homing . . . . . . . . . . . . . . . . . . . 14
     7.4. Multihoming .......................................14
      6.4. Frame Forwarding . . . . . . . . . . . . . . . . . . . . . 15
       7.4.1. ..........................................14
           6.4.1. Unicast  . . . . . . . . . . . . . . . . . . . . . . . 15
       7.4.2. ............................................15
           6.4.2. Multicast/Broadcast  . . . . . . . . . . . . . . . . . 15
     7.5. ................................15
      6.5. MPLS Encapsulation of PBB Frames . . . . . . . . . . . . . 16
   8. ..........................16
   7. Minimizing ARP/ND Broadcast  . . . . . . . . . . . . . . . . . 16
   9. ....................................16
   8. Seamless Interworking with IEEE 802.1aq/802.1Qbp  . . . . . . . 17
     9.1. 802.1aq / 802.1Qbp .............17
      8.1. B-MAC Address Assignment  . . . . . . . . . . . . . . . . . 17
     9.2. ..................................17
      8.2. IEEE 802.1aq / 802.1Qbp B-MAC Address Advertisement  . . . 17
     9.4. .......17
      8.3. Operation:  . . . . . . . . . . . . . . . . . . . . . . . . 17
   10. ................................................17
   9. Solution Advantages . . . . . . . . . . . . . . . . . . . . . 18
     10.1. ............................................18
      9.1. MAC Advertisement Route Scalability . . . . . . . . . . . 18
     10.2. .......................18
      9.2. C-MAC Mobility Independent of B-MAC Advertisements  . . . 18
     10.3. ........18
      9.3. C-MAC Address Learning and Confinement  . . . . . . . . . 19
     10.4. ....................19
      9.4. Seamless Interworking with 802.1aq Access Networks  . . . 19
     10.5.  Per Site ........19
      9.5. Per-Site Policy Support . . . . . . . . . . . . . . . . . 20
     10.6. ...................................20
      9.6. No C-MAC Address Flushing for All-Active Multi-Homing . . 20
   11.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . 20
   12. Multihoming ......20
   10. Security Considerations . . . . . . . . . . . . . . . . . . . 20
   13. .......................................20
   11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20
   14. ...........................................20
   12. References ....................................................21
      12.1. Normative References  . . . . . . . . . . . . . . . . . . . . 21
   15. .....................................21
      12.2. Informative References  . . . . . . . . . . . . . . . . . . . 21
   16. ...................................21
   Acknowledgements ..................................................22
   Contributors ......................................................22
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . 21 ................................................23

1.  Introduction

   [RFC7432] introduces a solution for multipoint L2VPN Layer 2 Virtual
   Private Network (L2VPN) services, with advanced multi-homing multihoming
   capabilities, using BGP for distributing customer/client MAC address reach-ability
   reachability information over the core MPLS/IP network.  [PBB]
   defines an architecture for Ethernet Provider Backbone Bridging
   (PBB), where MAC tunneling is employed to improve service instance
   and MAC address scalability in Ethernet as well as VPLS networks
   [RFC7080].

   In this document, we discuss how PBB can be combined with EVPN in
   order to: reduce the number of BGP MAC advertisement Advertisement routes by
   aggregating Customer/Client MAC (C-MAC) addresses via Provider
   Backbone MAC address (B-MAC), (B-MAC) address, provide client MAC address mobility
   using C-MAC aggregation, confine the scope of C-MAC learning to only
   active flows, offer per site policies per-site policies, and avoid C-MAC address
   flushing on topology changes.  The combined solution is referred to
   as PBB-EVPN.

3.

2.  Terminology

   ARP: Address Resolution Protocol
   BEB: Backbone Edge Bridge
   B-MAC: Backbone MAC Address
   B-VID: Backbone VLAN ID
   CE: Customer Edge
   C-MAC: Customer/Client MAC Address
   ES: Ethernet Segment
   ESI: Ethernet Segment Identifier
   EVI: EVPN Instance
   EVPN: Ethernet VPN
   I-SID: Service Instance Identifier (24 bits and global within a PBB
          network see [RFC7080])
   LSP: Label Switched Path
   MP2MP: Multipoint to Multipoint
   MP2P: Multipoint to Point
   ND: Neighbor Discovery
   NA: Neighbor Advertisement
   ND: Neighbor Discovery
   P2MP: Point to Multipoint
   P2P: Point to Point
   PBB: Provider Backbone Bridge
   PE: Provider Edge
   EVPN: Ethernet VPN
   EVI: EVPN Instance
   RT: Route Target
   VPLS: Virtual Private LAN Service

   Single-Active Redundancy Mode: When only a single PE, among a group
   of PEs attached to an Ethernet segment, is allowed to forward traffic
   to/from that Ethernet Segment, segment, then the Ethernet segment is defined
   to be operating in Single-Active redundancy mode.

   All-Active Redundancy Mode: When all PEs attached to an Ethernet
   segment are allowed to forward traffic to/from that Ethernet Segment, segment,
   then the Ethernet segment is defined to be operating in All-Active
   redundancy mode.

Conventions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119.

4. BCP 14 [RFC2119].

3.  Requirements

   The requirements for PBB-EVPN include all the requirements for EVPN
   that were described in [RFC7209], in addition to the following:

4.1.

3.1.  MAC Advertisement Route Scalability

   In typical operation, an EVPN PE sends a BGP MAC Advertisement Route route
   per customer/client MAC (C-MAC) C-MAC address.  In certain applications, this poses scalability
   challenges, as is the case in data center interconnect (DCI)
   scenarios where the number of virtual machines (VMs), and hence the
   number of C-MAC addresses, can be in the millions.  In such
   scenarios, it is required to reduce the number of BGP MAC
   Advertisement routes by relying on a 'MAC summarization' scheme, as
   is provided by PBB.

4.2.

3.2.  C-MAC Mobility Independent of B-MAC Advertisements

   Certain applications, such as virtual machine mobility, require
   support for fast C-MAC address mobility.  For these applications,
   when using EVPN, the virtual machine MAC address needs to be
   transmitted in BGP MAC Advertisement route.  Otherwise, traffic would
   be forwarded to the wrong segment when a virtual machine moves from
   one Ethernet
   segment ES to another.  This means MAC address prefixes cannot be used in
   data center applications.

   In order to support C-MAC address mobility, while retaining the
   scalability benefits of MAC summarization, PBB technology is used.
   It defines a Backbone MAC (B-MAC) B-MAC address space that is independent of the C-MAC
   address space, and aggregates C-MAC addresses via a single B-MAC
   address.

4.3.

3.3.  C-MAC Address Learning and Confinement

   In EVPN, all the PE nodes participating in the same EVPN instance are
   exposed to all the C-MAC addresses learnt learned by any one of these PE
   nodes because a C-MAC learned by one of the PE nodes is advertise advertised in
   BGP to other PE nodes in that EVPN instance.  This is the case even
   if some of the PE nodes for that EVPN instance are not involved in
   forwarding traffic to, or from, these C-MAC addresses.  Even if an
   implementation does not install hardware forwarding entries for C-MAC
   addresses that are not part of active traffic flows on that PE, the
   device memory is still consumed by keeping record of the C-MAC
   addresses in the routing table (RIB). information base (RIB) table.  In network
   applications with millions of C-MAC addresses, this introduces a non-trivial non-
   trivial waste of PE resources.  As such, it is required to confine
   the scope of visibility of C-MAC addresses only to only those PE nodes
   that are actively involved in forwarding traffic to, or from, these
   addresses.

4.4.  Per Site

3.4.  Per-Site Policy Support

   In many applications, it is required to be able to enforce
   connectivity policy rules at the granularity of a site (or segment).
   This includes the ability to control which PE nodes in the network
   can forward traffic to, or from, a given site.  Both EVPN and PBB-EVPN PBB-
   EVPN are capable of providing this granularity of policy control.  In
   the case where the policy needs to be at the granularity of per C-MAC
   address, then the C-MAC address learning in control-plane (in BGP) per
   [RFC7432] should be used.

4.5. learned in the control
   plane (in BGP) per [RFC7432].

3.5.  No C-MAC Address Flushing for All-Active Multi-Homing Multihoming

   Just as in [RFC7432], it is required to avoid C-MAC address flushing
   upon link, port port, or node failure for All-Active multi-homed multihomed segments.

5.

4.  Solution Overview

   The solution involves incorporating IEEE Backbone Edge Bridge (BEB)
   functionality on the EVPN PE nodes similar to PBB-VPLS, where BEB
   functionality is incorporated in the VPLS PE nodes.  The PE devices
   would then receive 802.1Q Ethernet frames from their attachment
   circuits, encapsulate them in the PBB header header, and forward the frames
   over the IP/MPLS core.  On the egress EVPN PE, the PBB header is
   removed following the MPLS disposition, and the original 802.1Q
   Ethernet frame is delivered to the customer equipment.

                   BEB   +--------------+  BEB
                   ||    |              |  ||
                   \/    |              |  \/
       +----+ AC1 +----+ |              | +----+   +----+
       | CE1|-----|    | |              | |    |---| CE2|
       +----+\    | PE1| |   IP/MPLS    | | PE3|   +----+
              \   +----+ |   Network    | +----+
               \         |              |
             AC2\ +----+ |              |
                 \|    | |              |
                  | PE2| |              |
                  +----+ |              |
                    /\   +--------------+
                    ||
                    BEB
         <-802.1Q-> <------PBB over MPLS------> <-802.1Q->

                        Figure 1: PBB-EVPN Network
   The PE nodes perform the following functions:- functions:

   -  Learn customer/client MAC addresses (C-MACs) over the attachment
      circuits in the data- data plane, per normal bridge operation.

   -  Learn remote C-MAC to B-MAC bindings in the data-plane data plane for traffic
      received from the core per [PBB] the bridging operation. operation described in
      [PBB].

   -  Advertise local B-MAC address reach-ability reachability information in BGP to
      all other PE nodes in the same set of service instances.  Note
      that every PE has a set of B-MAC addresses that uniquely identify
      identifies the device.  B-MAC address assignment is described in
      details in section
   7.2.2. Section 6.2.2.

   -  Build a forwarding table from remote BGP advertisements received
      associating remote B-MAC addresses with remote PE IP addresses and
      the associated MPLS label(s).

6.

5.  BGP Encoding

   PBB-EVPN leverages the same BGP Routes routes and Attributes attributes defined in
   [RFC7432], adapted as follows:

6.1. described below.

5.1.  Ethernet Auto-Discovery Route

   This route and all of its associated modes are not needed in PBB-EVPN
   because PBB encapsulation provides the required level of indirection
   for C-MAC addresses - -- i.e., an ES can be represented by a B-MAC
   address for the purpose of data-plane learning/forwarding.

   The receiving PE knows that it need not wait for the receipt of the
   Ethernet A-D (auto-discovery) route for route resolution by means of
   the reserved
   Ethernet Segment Identifier (ESI) ESI encoded in the MAC Advertisement route: the ESI
   values of 0 and MAX-ESI indicate that the receiving PE can resolve
   the path without an Ethernet A-D route.

6.2.

5.2.  MAC/IP Advertisement Route

   The EVPN MAC/IP Advertisement Route route is used to distribute B-MAC
   addresses of the PE nodes instead of the C-MAC addresses of end-
   stations/hosts.  This is because the C-MAC addresses are learnt learned in
   the
   data-plane data plane for traffic arriving from the core.  The MAC
   Advertisement
   Route route is encoded as follows:

   -  The MAC address field contains the B-MAC address.

   -  The Ethernet Tag field is set to 0.

   -  The Ethernet Segment Identifier field must be set either to either 0 (for
      single-homed segments or multi-homed multihomed segments with per-ISID per-I-SID load-
      balancing) or to MAX-ESI (for multi-homed multihomed segments with per-flow
      load-balancing).  All other values are not permitted.

   -  All other fields are set as defined in [RFC7432].

   This route is tagged with the Route Target (RT) RT corresponding to its EVI.  This EVI
   is analogous to a B-VID.

6.3.

5.3.  Inclusive Multicast Ethernet Tag Route

   This route is used for multicast pruning per I-SID.  It is used for
   auto-discovery of PEs participating in a given I-SID so that a
   multicast tunnel (MP2P, P2P, P2MP, or MP2MP LSP) can be setup set up for
   that I-SID . I-SID.  [RFC7080] uses multicast pruning per I-SID based on
   [MMRP]
   [MMRP], which is a soft-state protocol.  The advantages of multicast
   pruning using this BGP route over [MMRP] are that a) it scales very
   well for a large number of PEs and b) it works with any type of LSP
   (MP2P, P2P, P2MP, or MP2MP); whereas, [MMRP] only works over P2P
   pseudowires.  The Inclusive Multicast Ethernet Tag Route route is encoded
   as
   follow: follows:

   -  The Ethernet Tag field is set with the appropriate I-SID value.

   -  All other fields are set as defined in [RFC7432].

   This route is tagged with an RT.  This RT SHOULD be set to a value
   corresponding to its EVI (which is analogous to a B-VID).  The RT for
   this route MAY also be auto-derived from the corresponding Ethernet
   Tag (I-SID) based on the procedure specified in section Section 5.1.2.1 of
   [OVERLAY].

6.4.

5.4.  Ethernet Segment Route

   This route is used for auto-discovery of member PEs belonging to a given
   redundancy group (e.g., attached to a given Ethernet Segment) ES) per [RFC7432].

6.5.

5.5.  ESI Label Extended Community

   This extended community is not used in PBB-EVPN.  In [RFC7432], this
   extended community is used along with the Ethernet AD A-D route to
   advertise an MPLS label for the purpose of split-horizon filtering.
   Since in PBB-EVPN, the split-horizon filtering is performed natively
   using B-MAC SA, source address, there is no need for this extended
   community.

6.6.

5.6.  ES-Import Route Target

   This RT is used as defined in [RFC7432].

6.7.

5.7.  MAC Mobility Extended Community

   This extended community is defined in [RFC7432] and it is used with a
   MAC route (B-MAC route in case of PBB-EVPN).  The B-MAC route is
   tagged with the RT corresponding to its EVI (which is analogous to a
   B-VID).  When this extended community is used along with a B-MAC
   route in PBB-EVPN, it indicates that all C-MAC addresses associated
   with that B-MAC address across all corresponding I-SIDs must be
   flushed.

   When a PE first advertises a B-MAC, it MAY advertise it with this
   Extended Community
   extended community where the sticky/static flag is set to 1 and the
   sequence number is set to zero.  In such cases where the PE wants to
   signal the stickiness of a B-MAC, then when a flush indication is
   needed, the PE advertises the B-MAC along with the MAC Mobility
   extended community where the sticky/static flag remains set and the
   sequence number is incremented.

6.8.

5.8.  Default Gateway Extended Community

   This extended community is not used in PBB-EVPN.

7.

6.  Operation

   This section discusses the operation of PBB-EVPN, specifically in
   areas where it differs from [RFC7432].

7.1.

6.1.  MAC Address Distribution over Core

   In PBB-EVPN, host MAC addresses (i.e. (i.e., C-MAC addresses) need not be
   distributed in BGP.  Rather, every PE independently learns the C-MAC
   addresses in the data-plane data plane via normal bridging operation.  Every PE
   has a set of one or more unicast B-MAC addresses associated with it,
   and those are the addresses distributed over the core in MAC
   Advertisement routes.

7.2.

6.2.  Device Multi-homing

7.2.1. Flow-based Load-balancing Multihoming

6.2.1.  Flow-Based Load-Balancing

   This section describes the procedures for supporting device multi-
   homing
   multihoming in an All-Active redundancy mode (i.e., flow-based load-
   balancing).

7.2.1.1.

6.2.1.1.  PE B-MAC Address Assignment

   In [PBB] [PBB], every BEB is uniquely identified by one or more B-MAC
   addresses.  These addresses are usually locally administered by the
   Service Provider.
   service provider.  For PBB-EVPN, the choice of B-MAC address(es) for
   the PE nodes must be examined carefully as it has implications on the
   proper operation of multi-homing. multihoming.  In particular, for the scenario
   where a CE is multi-homed multihomed to a number of PE nodes with All-Active
   redundancy mode, a given C-MAC address would be reachable via
   multiple PE nodes concurrently.  Given that any given remote PE will
   bind the C-MAC address to a single B-MAC address, then the various PE
   nodes connected to the same CE must share the same B-MAC address.
   Otherwise, the MAC address table of the remote PE nodes will keep
   oscillating between the B-MAC addresses of the various PE devices.
   For example, consider the network of Figure 1, and assume that PE1
   has B-MAC address BM1 and PE2 has B-MAC address BM2.  Also, assume
   that both links from CE1 to the PE nodes are part of the same
   Ethernet link aggregation group.  If BM1 is not equal to BM2, the
   consequence is that the MAC address table on PE3 will keep
   oscillating such that the C-MAC address M1 of CE1 would flip-flop
   between BM1 or BM2, depending on the load-balancing decision on CE1
   for traffic destined to the core.

   Considering that there could be multiple sites (e.g. (e.g., CEs) that are
   multi-homed
   multihomed to the same set of PE nodes, then it is required for all
   the PE devices in a Redundancy Group redundancy group to have a unique B-MAC address
   per site.  This way, it is possible to achieve fast convergence in
   the case where a link or port failure impacts the attachment circuit
   connecting a single site to a given PE.

                               +---------+
                +-------+  PE1 | IP/MPLS |
               /               |         |
            CE1                | Network |     PEr
           M1  \               |         |
                +-------+  PE2 |         |
                /-------+      |         |
               /               |         |
            CE2                |         |
          M2   \               |         |
                \              |         |
                 +------+  PE3 +---------+

                    Figure 2: B-MAC Address Assignment

   In the example network shown in Figure 2 above, two sites
   corresponding to CE1 and CE2 are dual-homed to PE1/PE2 and PE2/PE3,
   respectively.  Assume that BM1 is the B-MAC used for the site
   corresponding to CE1.  Similarly, BM2 is the B-MAC used for the site
   corresponding to CE2.  On PE1, a single B-MAC address (BM1) is
   required for the site corresponding to CE1.  On PE2, two B-MAC
   addresses (BM1 and BM2) are required, one per site.  Whereas on PE3,
   a single B-MAC address (BM2) is required for the site corresponding
   to CE2.  All three PE nodes would advertise their respective B-MAC
   addresses in BGP using the MAC Advertisement routes defined in
   [RFC7432].  The remote PE, PEr, would learn via BGP that BM1 is
   reachable via PE1 and PE2, whereas BM2 is reachable via both PE2 and
   PE3.  Furthermore, PEr establishes, via the PBB bridge learning
   procedure, that C-MAC M1 is reachable via BM1, and C-MAC M2 is
   reachable via BM2.  As a result, PEr can load-balance traffic
   destined to M1 between PE1 and PE2, as well as traffic destined to M2
   between both PE2 and PE3. In the case of a failure that causes, for example,
   CE1 to be isolated from PE1, the latter can withdraw the route it has
   advertised for BM1. This way, PEr would update its path list for BM1,
   and will send all traffic destined to M1 over to PE2 only.

   For Single-Homed or Single-Active sites, it is possible to assign a
   unique B-MAC address per site, or have all the Single-Homed sites or
   Single-Active sites connected to a given PE share a single B-MAC
   address. The advantage of the first model over the second model is
   the ability to avoid C-MAC destination address lookup on the
   disposition PE (even though source C-MAC learning is still required
   in the data-plane). The disadvantage of the first model over the
   second model is additional B-MAC advertisements in BGP.

   In summary, every PE may use a unicast B-MAC address shared by all
   single-homed sites or a unicast B-MAC address per single-homed site
   and, in addition, a unicast B-MAC address per All-Active multi-homed
   site. PE3.  In the latter case, the B-MAC address MUST case of a failure that causes, for
   example, CE1 to be isolated from PE1, the same latter can withdraw the
   route it has advertised for BM1.  This way, PEr would update its path
   list for BM1 and will send all
   PE nodes in a Redundancy Group connected traffic destined to the same site.

7.2.1.2. M1 over to PE2
   only.

6.2.1.2.  Automating B-MAC Address Assignment

   The PE B-MAC address used for Single-Homed single-homed or Single-Active sites can
   be automatically derived from the hardware (using for e.g. example the
   backplane's address and/or PE's reserved MAC pool ).  However, the B-
   MAC
   B-MAC address used for All-Active sites must be coordinated among the
   RG
   redundancy group members.  To automate the assignment of this latter
   address, the PE can derive this B-MAC address from the MAC Address address
   portion of the CE's Link Aggregation Control Protocol (LACP) System
   Identifier by flipping the 'Locally Administered' bit of the CE's
   address.  This guarantees the uniqueness of the B-MAC address within
   the network, and ensures that all PE nodes connected to the same All-Active All-
   Active CE use the same value for the B-MAC address.

   Note that with this automatic provisioning of the B-MAC address
   associated with All-Active CEs, it is not possible to support the
   uncommon scenario where a CE has multiple link bundles within the
   same LACP session towards the PE nodes, and the service involves
   hair-pinning traffic from one bundle to another.  This is because the
   split-horizon filtering relies on B-MAC addresses rather than Site-ID
   Labels (as will be described in the next section).  The operator must
   explicitly configure the B-MAC address for this fairly uncommon
   service scenario.

   Whenever a B-MAC address is provisioned on the PE, either manually or
   automatically (as an outcome of CE auto-discovery), the PE MUST
   transmit an a MAC Advertisement Route route for the B-MAC address with a
   downstream assigned MPLS label that uniquely identifies that address
   on the advertising PE.  The route is tagged with the RTs of the
   associated EVIs as described above.

7.2.1.3

6.2.1.3.  Split Horizon and Designated Forwarder Election

   [RFC7432] relies on access split horizon, split-horizon filtering for a multi-homed ES,
   where the Ethernet Segment
   Label ES label is used for egress filtering on the attachment
   circuit in order to prevent forwarding loops.  In PBB-EVPN, the B-MAC
   source address can be used for the same purpose, as it uniquely
   identifies the originating site of a given frame.  As such, Ethernet Segment (ES)
   Labels ES labels
   are not used in PBB-EVPN, and the egress split-horizon filtering is
   done based on the B-MAC source address.  It is worth noting here that
   [PBB] defines this B-MAC address based address-based filtering function as part of
   the I-Component options, hence options; hence, no new functions are required to
   support split-horizon filtering beyond what is already defined in
   [PBB].

   The Designated Forwarder (DF) election procedures are defined in
   [RFC7432].

7.2.2.

6.2.2.  Load-Balancing based on I-SID Based Load-balancing

   This section describes the procedures for supporting device multi-
   homing
   multihoming in a Single-Active redundancy mode with per-ISID per-I-SID load-
   balancing.

7.2.2.1.

6.2.2.1.  PE B-MAC Address Assignment

   In the case where per-ISID per-I-SID load-balancing is desired among the PE
   nodes in a given redundancy group, multiple unicast B-MAC addresses
   are allocated per multi-homed Ethernet Segment: multihomed ES: Each PE connected to the multi-homed multihomed
   segment is assigned a unique B-MAC.  Every PE then advertises its
   B-MAC address using the BGP MAC advertisement Advertisement route.  In this mode of
   operation, two B-MAC address assignment address-assignment models are possible:

   -  The PE may use a shared B-MAC address for multiple Ethernet
   Segments (ES's). This includes the all its single-homed
      segments as well as
   the and/or all its multi-homed Single-Active segments (e.g.,
      segments operating with per-ISID in per-I-SID load-balancing mode. mode).

   -  The PE may use a dedicated B-MAC address for each ES operating
      with
   per-ISID per-I-SID load-balancing mode.

   A PE implementation MAY choose to support either the shared B-MAC
   address model or the dedicated B-MAC address model without causing
   any interoperability issues.  The advantage of the dedicated B-MAC
   over the shared B-MAC address for multi-homed Single-Active segments,
   is that when C-MAC flushing is needed, fewer C-MAC addresses are
   impacted.  Furthermore, it gives the disposition PE the ability to
   avoid C-MAC destination address lookup even though source C-MAC
   learning is still required in the data plane.  Its disadvantage is
   that there are additional B-MAC advertisements in BGP.

   A remote PE initially floods traffic to a destination C-MAC address,
   located in a given multi-homed Ethernet Segment, multihomed ES, to all the PE nodes configured with
   that I-SID.  Then, when reply traffic arrives at the remote PE, it
   learns (in the data-path) data path) the B-MAC address and associated next-hop
   PE to use for said C-MAC address.

7.2.2.2.

6.2.2.2.  Split Horizon and Designated Forwarder Election

   The procedures are similar to the flow-based load-balancing case,
   with the only difference being that the DF filtering must be applied
   to unicast as well as multicast traffic, and in both core-to-segment
   as well as segment-to-core directions.

7.2.2.3.

6.2.2.3.  Handling Failure Scenarios

   When a PE connected to a multi-homed Ethernet Segment multihomed ES loses connectivity to the
   segment, due to link or port failure, it needs to notify the remote
   PEs to trigger C-MAC address flushing.  This can be achieved in one
   of two ways, depending on the B-MAC assignment model:

   -  If the PE uses a shared B-MAC address for multiple ES's, Ethernet
      segments, then the C-MAC flushing is signaled by means of having
      the failed PE re-
   advertise re-advertise the MAC Advertisement route for the
      associated B-MAC, tagged with the MAC Mobility Extended Community extended community
      attribute.  The value of the Counter field in that attribute must
      be incremented prior to advertisement.  This causes the remote PE
      nodes to flush all C-MAC addresses associated with the B-MAC in
      question.  This is done across all I-SIDs that are mapped to the
      EVI of the withdrawn MAC route.

   -  If the PE uses a dedicated B-MAC address for each Ethernet Segment ES operating
      under per-ISID per-I-SID load-balancing mode, the failed PE simply
      withdraws the B-MAC route previously advertised for that segment.
      This causes the remote PE nodes to flush all C-MAC addresses
      associated with the B-MAC in question.  This is done across all
      I-SIDs that are mapped to the EVI of the withdrawn MAC route.

   When a PE connected to a multi-homed Ethernet Segment multihomed ES fails (i.e. (i.e., node failure) or
   when the PE becomes completely isolated from the EVPN network, the
   remote PEs will start purging the MAC Advertisement routes that were
   advertised by the failed PE.  This is done either as an outcome of
   the remote PEs detecting that the BGP session to the failed PE has
   gone down, or by having a Route Reflector withdrawing all the routes
   that were advertised by the failed PE.  The remote PEs, in this case,
   will perform C-MAC address flushing as an outcome of the MAC
   Advertisement route withdrawals.

   For all failure scenarios (link/port failure, node failure failure, and PE
   node isolation), when the fault condition clears, the recovered PE
   re-advertises the associated Ethernet Segment ES route to other members of its Redundancy Group.
   redundancy group.  This triggers the backup PE(s) in the
   Redundancy Group redundancy
   group to block the I-SIDs for which the recovered PE is a DF.  When a
   backup PE blocks the I-SIDs, it triggers a C-MAC address flush
   notification to the remote PEs by re-advertising the MAC
   Advertisement route for the associated B-MAC, with the MAC Mobility
   Extended Community
   extended community attribute.  The value of the Counter field in that
   attribute must be incremented prior to advertisement.  This causes
   the remote PE nodes to flush all C-MAC addresses associated with the B-
   MAC
   B-MAC in question.  This is done across all I-SIDs that are mapped to
   the EVI of the withdrawn/readvertised withdrawn/re-advertised MAC route.

7.3.

6.3.  Network Multi-homing Multihoming

   When an Ethernet network is multi-homed multihomed to a set of PE nodes running
   PBB-EVPN, Single-Active redundancy model can be supported with per per-
   service instance (i.e. (i.e., I-SID) load-balancing.  In this model, DF
   election is performed to ensure that a single PE node in the
   redundancy group is responsible for forwarding traffic associated
   with a given I-SID.  This guarantees that no forwarding loops are
   created.  Filtering based on DF state applies to both unicast and
   multicast traffic, and in both access-to-core as well as core-to-
   access directions just like a Single-Active multi-homed multihomed device
   scenario (but unlike an All-Active multi-homed multihomed device scenario where
   DF filtering is limited to multi-destination frames in the core-to-access core-to-
   access direction).  Similar to a Single-Active multi-homed multihomed device
   scenario, with I-SID load-balancing based load-balancing, on I-SID, a unique B-MAC address
   is assigned to each of the PE nodes connected to the multi-homed multihomed
   network
   (Segment).

7.4. (segment).

6.4.  Frame Forwarding

   The frame forwarding frame-forwarding functions are divided in between the Bridge
   Module, which hosts the [PBB] Backbone Edge Bridge (BEB) BEB functionality, and the MPLS
   Forwarder which handles the MPLS imposition/disposition.  The details
   of frame forwarding for unicast and multi-destination frames are
   discussed next.

7.4.1.

6.4.1.  Unicast

   Known unicast traffic received from the AC Attachment Circuit (AC) will
   be PBB-encapsulated by the PE using the B-MAC source address
   corresponding to the originating site.  The unicast B-MAC destination
   address is determined based on a lookup of the C-MAC destination
   address (the binding of the two is done via transparent learning of
   reverse traffic).  The resulting frame is then encapsulated with an
   LSP tunnel label and the
   MPLS an EVPN label which uniquely identifies associated with the B-MAC
   destination address on
   the egress PE. address.  If per flow load-balancing over ECMPs in the
   MPLS core is required, then a flow label is added below the label
   associated with the BMAC B-MAC address in the label stack.

   For unknown unicast traffic, the PE forwards these frames over the
   MPLS core.  When these frames are to be forwarded, then the same set
   of options used for forwarding multicast/broadcast frames (as
   described in next section) are used.

7.4.2.

6.4.2.  Multicast/Broadcast

   Multi-destination frames received from the AC will be PBB-
   encapsulated by the PE using the B-MAC source address corresponding
   to the originating site.  The multicast B-MAC destination address is
   selected based on the value of the I-SID as defined in [PBB].  The
   resulting frame is then forwarded over the MPLS core using one out of the
   following two options:

   Option 1: the MPLS Forwarder can perform ingress replication over a
      set of MP2P or P2P tunnel LSPs.  The frame is encapsulated with a
      tunnel LSP label and the EVPN ingress replication label advertised
      in the Inclusive Multicast Route. Ethernet Tag [RFC7432].

   Option 2: the MPLS Forwarder can use P2MP tunnel LSP per the
      procedures defined in [RFC7432].  This includes either the use of
      Inclusive or Aggregate Inclusive trees.  Furthermore, the MPLS
      Forwarder can use MP2MP tunnel LSP if Inclusive trees are used.

   Note that the same procedures for advertising and handling the
   Inclusive Multicast Route Ethernet Tag defined in [RFC7432] apply here.

7.5.

6.5.  MPLS Encapsulation of PBB Frames

   The encapsulation for the transport of PBB frames over MPLS is
   similar to that of classical Ethernet, albeit with the additional PBB
   header, as shown in the figure below:

                           +------------------+
                           | IP/MPLS Header   |
                           +------------------+
                           | PBB Header       |
                           +------------------+
                           | Ethernet Header  |
                           +------------------+
                           | Ethernet Payload |
                           +------------------+
                           | Ethernet FCS     |
                           +------------------+

                   Figure 8: 3: PBB over MPLS Encapsulation

8.

7.  Minimizing ARP/ND Broadcast

   The PE nodes MAY implement an ARP/ND-proxy function in order to
   minimize the volume of ARP/ND traffic that is broadcasted over the
   MPLS network.  In case of ARP proxy, this is achieved by having each
   PE node snoop on ARP request and response messages received over the
   access interfaces or the MPLS core.  The PE builds a cache of IP / MAC IP/MAC
   address bindings from these snooped messages.  The PE then uses this
   cache to respond to ARP requests ingress on access ports and
   targeting target
   hosts that are in remote sites.  If the PE finds a match for the IP
   address in its ARP cache, it responds back to the requesting host and
   drops the request.  Otherwise, if it does not find a match, then the
   request is flooded over the MPLS network using either ingress
   replication or P2MP LSPs.  In case of ND proxy, this is achieved
   similar to the above but with ND/NA messages per [RFC4389].

9.

8.  Seamless Interworking with IEEE 802.1aq/802.1Qbp 802.1aq / 802.1Qbp

                           +--------------+
                           |              |
           +---------+     |     MPLS     |    +---------+
   +----+  |         |   +----+        +----+  |         |  +----+
   |SW1 |--|         |   | PE1|        | PE2|  |         |--| SW3|
   +----+  | 802.1aq |---|    |        |    |--| 802.1aq |  +----+
   +----+  |  .1Qbp  |   +----+        +----+  |  .1Qbp  |  +----+
   |SW2 |--|         |     |   Backbone   |    |         |--| SW4|
   +----+  +---------+     +--------------+    +---------+  +----+

   |<------ IS-IS -------->|<-----BGP----->|<------ IS-IS ------>|  CP

   |<-------------------------   PBB  -------------------------->|  DP
                           |<----MPLS----->|

   Legend: CP = Control Plane Control-Plane View
           DP = Data Plane Data-Plane View

    Figure 7: 4: Interconnecting 802.1aq/802.1Qbp 802.1aq / 802.1Qbp Networks with PBB-EVPN

9.1.

8.1.  B-MAC Address Assignment

   The B-MAC addresses need to be globally unique across all networks
   including PBB-EVPN and IEEE 802.1aq / 802.1Qbp networks.  The B-MAC
   addresses used for Single-Home single-homed and Single-Active Ethernet Segments segments
   should be unique because they are typically auto-derived from the
   PE's pools of reserved MAC addresses that are unique.  The B-MAC
   addresses used for All-Active Ethernet Segments segments should also be unique
   given that each network operator typically has its own assigned
   Organizationally Unique Identifier (OUI) and thus can assign its own
   unique MAC addresses.

9.2.

8.2.  IEEE 802.1aq / 802.1Qbp B-MAC Address Advertisement

   B-MAC addresses associated with 802.1aq / 802.1Qbp switches are
   advertised using the EVPN MAC/IP route advertisement already defined
   in [RFC7432].

9.4.

8.3.  Operation:

   When a PE receives a PBB-encapsulated Ethernet frame from the access
   side, it performs a lookup on the B-MAC destination address to
   identify the next hop.  If the lookup yields that the next hop is a
   remote PE, the local PE would then encapsulate the PBB frame in MPLS.
   The label stack comprises of the VPN label (advertised by the remote
   PE), followed by an LSP/IGP label.  From that point onwards, regular
   MPLS forwarding is applied.

   On the disposition PE, assuming penultimate-hop-popping is employed,
   the PE receives the MPLS-encapsulated PBB frame with a single label:
   the VPN label.  The value of the label indicates to the disposition
   PE that this is a PBB frame, so the label is popped, the TTL field
   (in the 802.1Qbp F-Tag) is reinitialized reinitialized, and normal PBB processing
   is employed from this point onwards.

10.

9.  Solution Advantages

   In this section, we discuss the advantages of the PBB-EVPN solution
   in the context of the requirements set forth in section 3 above.

10.1. Section 3.

9.1.  MAC Advertisement Route Scalability

   In PBB-EVPN PBB-EVPN, the number of MAC Advertisement Routes routes is a function of
   the number of Ethernet Segments segments (e.g., sites), sites) rather than the number
   of hosts/servers.  This is because the B-MAC addresses of the PEs,
   rather than C-MAC addresses (of hosts/servers) hosts/servers), are being advertised
   in BGP.  As discussed above, there's a one-to-one mapping between All-
   Active multi-homed
   All-Active multihomed segments and their associated B-MAC addresses, and addresses;
   there can be either a one-to-one or many-to-one mapping between
   Single-Active multi-homed multihomed segments and their associated B-MAC
   addresses,
   addresses; and finally there is a many-to-one mapping between single-
   home sites and their associated B-MAC addresses on a given PE.  This
   means a single B-MAC is associated with one or more segments where
   each segment can be associated with many C-MAC addresses.  As a
   result, the volume of MAC Advertisement Routes routes in PBB-EVPN may be
   multiple orders of magnitude less than EVPN.

10.2.

9.2.  C-MAC Mobility Independent of B-MAC Advertisements

   As described above, in PBB-EVPN, a single B-MAC address can aggregate
   many C-MAC addresses.  Given that B-MAC addresses are associated with
   segments attached to a PE or to the PE itself, their locations are
   fixed and thus not impacted what so ever by C-MAC mobility.
   Therefore, C-MAC mobility does not affect B-MAC addresses (e.g., any
   re-advertisements of them).  This is because the association of C-MAC
   address to B-
   MAC B-MAC address association is learnt learned in the data-plane and C-MAC
   addresses are not advertised in BGP.  Aggregation via B-MAC addresses
   in PBB-EVPN performs much better than EVPN.

   To illustrate how this compares to EVPN, consider the following
   example:

      If a PE running EVPN advertises reachability for N MAC addresses
      via a particular segment, and then 50% of the MAC addresses in
      that segment move to other segments (e.g. (e.g., due to virtual machine
      mobility), then N/2 additional MAC Advertisement routes need to be
      sent for the MAC addresses that have moved.  With PBB-EVPN, on the
      other hand, the B-MAC addresses which that are statically associated
      with PE nodes, nodes are not subject to mobility.  As C-MAC addresses
      move from one segment to another, the binding of C-MAC to B-MAC
      addresses is updated via data-plane learning in PBB-EVPN.

10.3.

9.3.  C-MAC Address Learning and Confinement

   In PBB-EVPN, C-MAC address reachability information is built via
   data-plane learning.  As such, PE nodes not participating in active
   conversations involving a particular C-MAC address will purge that
   address from their forwarding tables.  Furthermore, since C-MAC
   addresses are not distributed in BGP, PE nodes will not maintain any
   record of them in the control-plane routing table.

10.4.

9.4.  Seamless Interworking with 802.1aq Access Networks

   Consider the scenario where two access networks, one running MPLS and
   the other running 802.1aq, are interconnected via an MPLS backbone
   network.  The figure below shows such an example network.

                               +--------------+
                               |              |
               +---------+     |     MPLS     |    +---------+
       +----+  |         |   +----+        +----+  |         |  +----+
       | CE |--|         |   | PE1|        | PE2|  |         |--| CE |
       +----+  | 802.1aq |---|    |        |    |--|  MPLS   |  +----+
       +----+  |         |   +----+        +----+  |         |  +----+
       | CE |--|         |     |   Backbone   |    |         |--| CE |
       +----+  +---------+     +--------------+    +---------+  +----+

                  Figure 9: 5: Interoperability with 802.1aq

   If the MPLS backbone network employs EVPN, then the 802.1aq data-
   plane encapsulation must be terminated on PE1 or the edge device
   connecting to PE1.  Either way, all the PE nodes that are part of the
   associated service instances will be exposed to all the C-MAC
   addresses of all hosts/servers connected to the access networks.
   However, if the MPLS backbone network employs PBB-EVPN, then the
   802.1aq encapsulation can be extended over the MPLS backbone, thereby
   maintaining C-MAC address transparency on PE1.  If PBB-EVPN is also
   extended over the MPLS access network on the right, then C-MAC
   addresses would be transparent to PE2 as well.

10.5.  Per Site

9.5.  Per-Site Policy Support

   In PBB-EVPN, the per site per-site policy can be supported via B-MAC addresses
   via assigning a unique B-MAC address for every site/segment
   (typically multi-homed multihomed but can also be single-homed).  Given that the
   B-MAC addresses are sent in BGP MAC/IP route advertisement, it is
   possible to define per site (i.e. per-site (i.e., B-MAC) forwarding policies
   including policies for E-TREE service.

10.6.

9.6.  No C-MAC Address Flushing for All-Active Multi-Homing Multihoming

   Just as in [RFC7432], with PBB-EVPN, it is possible to avoid C-MAC
   address flushing upon topology change affecting an All-Active multi-
   homed
   multihomed segment.  To illustrate this, consider the example network
   of Figure 1.  Both PE1 and PE2 advertise the same B-MAC address (BM1)
   to PE3.  PE3 then learns the C-MAC addresses of the servers/hosts
   behind CE1 via data-plane learning.  If AC1 fails, then PE3 does not
   need to flush any of the C-MAC addresses learnt learned and associated with
   BM1.  This is because PE1 will withdraw the MAC Advertisement routes
   associated with BM1, thereby leading PE3 to have a single adjacency
   (to PE2) for this B-MAC address.  Therefore, the topology change is
   communicated to PE3 and no C-MAC address flushing is required.

12.

10.  Security Considerations

   All the security considerations in [RFC7432] apply directly to this
   document because this document leverages [RFC7432] the control plane described
   in [RFC7432] and their associated procedures - -- although not the
   complete set but rather a subset.

   This draft document does not introduce any new security considerations
   beyond that of [RFC7432] and [RFC4761] because advertisements and
   processing of B-MAC addresses follow that of [RFC7432], [RFC7432] and processing
   of C-MAC addresses follow that of [RFC4761] - -- i.e, B-MAC addresses
   are learned in the control plane and C-MAC addresses are learned in
   data plane.

13.

11.  IANA Considerations

   There is are no additional IANA considerations for PBB-EVPN beyond what
   is already described in [RFC7432].

14.

12.  References

12.1.  Normative References

   [PBB]      IEEE, "IEEE Standard for Local and metropolitan area
              networks - Media Access Control (MAC) Bridges and Virtual
              Bridged Local Area Networks", Clauses 25 and 26, IEEE Std
              802.1Q, DOI 10.1109/IEEESTD.2011.6009146.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <http://www.rfc-editor.org/info/rfc2119>.

   [RFC7432]  A.  Sajassi, et al., A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
              Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS Based MPLS-Based
              Ethernet VPN", RFC
              7432 , 7432, DOI 10.17487/RFC7432, February 2015.

   [PBB]      Clauses 25 and 26 of
              2015, <http://www.rfc-editor.org/info/rfc7432>.

12.2.  Informative References

   [MMRP]     IEEE, "IEEE Standard for Local and metropolitan area
              networks - Media Access Control (MAC) Bridges and Virtual
              Bridged Local Area Networks", Clause 10, IEEE Std 802.1Q, 2013.

15.  Informative References

   [RFC7080]  A.
              DOI 10.1109/IEEESTD.2011.6009146.

   [OVERLAY]  Sajassi, et al., "Virtual Private LAN Service (VPLS)
              Interoperability with Provider Backbone Bridges", RFC
              7080, December 2013.

   [RFC7209]  D. Thaler, et al., "Requirements for Ethernet VPN (EVPN)",
              RFC 7209, May 2014. A., Ed., Drake, J., Ed., Bitar, N., Isaac, A.,
              Uttaro, J., Henderickx, W., Shekhar, R., Salam, S., Patel,
              K., Rao, D., and S. Thoria, "A Network Virtualization
              Overlay Solution using EVPN",
              draft-ietf-bess-evpn-overlay-01, February 2015.

   [RFC4389]  A. Sajassi, et al.,  Thaler, D., Talwar, M., and C. Patel, "Neighbor Discovery
              Proxies (ND Proxy)", RFC 4389, DOI 10.17487/RFC4389, April 2006.
              2006, <http://www.rfc-editor.org/info/rfc4389>.

   [RFC4761]  K.  Kompella, et al., K., Ed., and Y. Rekhter, Ed., "Virtual Private
              LAN Service (VPLS) Using BGP for Auto-Discovery and
              Signaling", RFC 4761,
              Jauary 2007.

   [OVERLAY]  A. DOI 10.17487/RFC4761, January 2007,
              <http://www.rfc-editor.org/info/rfc4761>.

   [RFC7080]  Sajassi, et al., "A Network Virtualization Overlay
              Solution using EVPN", draft-ietf-bess-evpn-overlay-01,
              work in progress, February 2015.

   [MMRP]     Clause 10 of "IEEE Standard for Local A., Salam, S., Bitar, N., and metropolitan
              area networks - Media Access Control (MAC) Bridges F. Balus, "Virtual
              Private LAN Service (VPLS) Interoperability with Provider
              Backbone Bridges", RFC 7080, DOI 10.17487/RFC7080,
              December 2013, <http://www.rfc-editor.org/info/rfc7080>.

   [RFC7209]  Sajassi, A., Aggarwal, R., Uttaro, J., Bitar, N.,
              Henderickx, W., and
              Virtual Bridged Local Area Networks", IEEE Std 802.1Q,
              2013.

11. A. Isaac, "Requirements for Ethernet
              VPN (EVPN)", RFC 7209, DOI 10.17487/RFC7209, May 2014,
              <http://www.rfc-editor.org/info/rfc7209>.

Acknowledgements

   The authors would like to thank Jose Liste and Patrice Brissette for
   their reviews and comments of this document.  We would also like to
   thank Giles Heron for several rounds of reviews and providing
   valuable inputs to get this draft document ready for IESG submission.

2.

Contributors

   In addition to the authors listed above, listed, the following individuals also
   contributed to this document.

   Lizhong Jin, ZTE
   Sami Boutros, Cisco
   Dennis Cai, Cisco
   Keyur Patel, Cisco
   Sam Aldrin, Huawei
   Himanshu Shah, Ciena
   Jorge Rabadan, ALU

16.

Authors' Addresses

   Ali Sajassi Sajassi, editor
   Cisco
   170 West Tasman Drive
   San Jose, CA  95134, US  95134
   United States
   Email: sajassi@cisco.com

   Samer Salam
   Cisco
   595 Burrard Street, Suite # 2123
   Vancouver, BC V7X 1J1, 1J1
   Canada
   Email: ssalam@cisco.com

   Nabil Bitar
   Verizon Communications
   Email : nabil.n.bitar@verizon.com

   Aldrin Isaac
   Bloomberg
   Juniper
   Email: aisaac71@bloomberg.net aisaac@juniper.net

   Wim Henderickx
   Alcatel-Lucent
   Email: wim.henderickx@alcatel-lucent.be wim.henderickx@alcatel-lucent.com