<?xml version="1.0" encoding="US-ASCII"?>
<!-- edited with XMLSPY v5 rel. 3 U (http://www.xmlspy.com)
     by Daniel M Kohn (private) -->
<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
<!ENTITY rfc2119 PUBLIC "" "http://xml.resource.org/public/rfc/bibxml/reference.RFC.2119.xml">
]>
<?rfc toc="yes"?>
<?rfc tocompact="yes"?>
<?rfc tocdepth="3"?>
<?rfc tocindent="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc comments="yes"?>
<?rfc inline="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="no"?>
<rfc category="info" consensus="true"
     docName="draft-ietf-teas-native-ip-scenarios-12"
     ipr="trust200902"> ipr="trust200902"
     number="8735" obsoletes="" sortRefs="true" submissionType="IETF"
     symRefs="true" tocDepth="3" tocInclude="true" updates="" version="3"
     xml:lang="en">
  <front>
    <title abbrev="CCDR Scenario and Simulation Results">Scenarios and
    Simulation Results of PCE in a Native IP Network</title>

    <seriesInfo name="RFC" value="8735"/>

    <author fullname="Aijun Wang" initials="A" surname="Wang">
      <organization>China Telecom</organization>

      <address>
        <postal>
          <street>Beiqijia Town, Changping District</street>

          <city>Beijing</city>

          <region>Beijing</region>

          <code>102209</code>

          <country>China</country>
        </postal>

        <email>wangaj3@chinatelecom.cn</email>
      </address>
    </author>

    <author fullname="Xiaohong Huang" initials="X" surname="Huang">
      <organization abbrev="BUPT">Beijing University of Posts and
      Telecommunications</organization>

      <address>
        <postal>
          <street>No.10 Xitucheng Road, Haidian District</street>

          <street/>

          <city>Beijing</city>

          <region/>

          <code/>

          <country>China</country>
        </postal>

        <email>huangxh@bupt.edu.cn</email>
      </address>
    </author>

    <author fullname="Caixia Kou" initials="C" surname="Kou">
      <organization abbrev="BUPT">Beijing University of Posts and
      Telecommunications</organization>

      <address>
        <postal>
          <street>No.10 Xitucheng Road, Haidian District</street>

          <city>Beijing</city>

          <region/>

          <code/>

          <country>China</country>
        </postal>

        <phone/>

        <facsimile/>

        <email>koucx@lsec.cc.ac.cn</email>

        <uri/>
      </address>
    </author>

    <author fullname="Zhenqiang Li" initials="Z" surname="Li">
      <organization>China Mobile</organization>

      <address>
        <postal>
          <street>32 Xuanwumen West Ave, Xicheng District</street>

          <city>Beijing</city>

          <region/>

          <code>100053</code>

          <country>China</country>
        </postal>

        <phone/>

        <facsimile/>

        <email>li_zhenqiang@hotmail.com</email>

        <uri/>
      </address>
    </author>

    <author fullname="Penghui Mi" initials="P" surname="Mi">
      <organization>Huawei Technologies</organization>

      <address>
        <postal>
          <street>Tower C of Bldg.2, Cloud Park, No.2013 of Xuegang
          Road</street>

          <city>Shenzhen</city>

          <region>Bantian,Longgang District</region>

          <code>518129</code>

          <country>China</country>
        </postal>

        <phone/>

        <facsimile/>

        <email>mipenghui@huawei.com</email>

        <uri/>
      </address>
    </author>

    <date day="29" month="October" year="2019"/> month="February" year="2020"/>

    <area>RTG Area</area>

    <workgroup>TEAS Working Group</workgroup>

    <keyword>RFC</keyword>

    <keyword>CCDR, Traffic Engineering,</keyword>

    <abstract>
      <t>Requirements for providing the End to End(E2E) End-to-End (E2E) performance assurance
      are emerging within the service provider networks. While there are
      various technology solutions, there is no single solution that can
      fulfill these requirements for a native IP network. In particular, there
      is a need for a universal (E2E) E2E solution that can cover both intra- and
      inter-domain scenarios.</t>

      <t>One feasible E2E traffic engineering traffic-engineering solution is the addition of
      central control in a native IP network. This document describes various
      complex scenarios and simulation results when applying the Path
      Computation Element (PCE) in a native IP network. This solution,
      referred to as Centralized Control Dynamic Routing (CCDR), integrates
      the advantage of using distributed protocols and the power of a
      centralized control technology, providing traffic engineering for native
      IP networks in a manner that applies equally to intra- and inter-domain
      scenarios.</t>
    </abstract>
  </front>

  <middle>
    <section anchor="intro" title="Introduction"> numbered="true" toc="default">
      <name>Introduction</name>

      <t>A service provider network is composed of thousands of routers that
      run distributed protocols to exchange the reachability information. The path
      for the destination network is mainly calculated, and controlled, by the
      distributed protocols. These distributed protocols are robust enough to
      support most applications, applications; however, they have some difficulties
      supporting the complexities needed for traffic engineering traffic-engineering applications, e.g.
      e.g., E2E performance assurance, or maximizing the link utilization
      within an IP network.</t>

      <t>Multiprotocol Label Switching (MPLS) using Traffic Engineering Traffic-Engineering (TE)
      technology (MPLS-TE)<xref target="RFC3209"/>is (MPLS-TE) <xref format="default" target="RFC3209"/> is one
      solution for traffic
      engineering networks TE networks, but it introduces an MPLS network and along with
      related
      technology technology, which would be an overlay of the IP network. MPLS-TE
      technology is often used for Label Switched Path (LSP) protection and
      setting up complex path set-up paths within a domain. It has not been widely
      deployed for meeting E2E (especially in inter-domain) dynamic
      performance assurance requirements for an IP network.</t>

      <t>Segment Routing <xref format="default" target="RFC8402"/> is another
      solution that integrates some advantages of using a distributed protocol
      and a
      centrally central control technology, but it requires the underlying network,
      especially the provider edge router, to do a an in-depth label push and
      pop action
      in-depth, and adds while adding complexity when coexisting with the Non-Segment
      Routing non-segment
      routing network. Additionally, it can only maneuver the E2E paths for
      MPLS and IPv6 traffic via different mechanisms.</t>

      <t>Deterministic Networking (DetNet)<xref (DetNet) <xref format="default"
      target="RFC8578"/> is another possible solution. It is primarily focused
      on providing bounded latency for a flow and introduces additional
      requirements on the domain edge router. The current DetNet scope is
      within one domain. The use cases defined in this document do not require
      the additional complexity of deterministic properties and so differ from
      the DetNet use cases.</t>

      <t>This draft document describes several scenarios for a native IP network
      where a Centralized Control Dynamic Routing (CCDR) framework can produce
      qualitative improvement in efficiency without requiring a change of to the
      data-plane behavior on the router. Using knowledge of BGP(Border the Border Gateway
      Protocol)
      Protocol (BGP) session-specific prefixes advertised by a router, the
      network topology and the near real time link utilization near-real-time link-utilization information
      from network management systems, a central PCE is able to compute an
      optimal path and give the underlay underlying routers the destination address to
      use to reach the BGP nexthop, such that the distributed routing protocol
      will use the computed path via traditional recursive lookup procedure.
      Some results from simulations of path optimization are also presented, presented to
      concretely illustrate a variety of scenarios where CCDR shows
      significant improvement over traditional distributed routing
      protocols.</t>

      <t>This draft document is the base document of the following two drafts: documents:
      the universal solution draft, document, which is suitable for intra-domain and
      inter-domain TE scenario, is described in <xref format="default"
      target="I-D.ietf-teas-pce-native-ip"/>; and the related protocol
      extension contents is described in <xref
      target="I-D.ietf-pce-pcep-extension-native-ip"/></t> format="default"
      target="I-D.ietf-pce-pcep-extension-native-ip"/>.</t>
    </section>

    <section title="Terminology">
      <t>This document uses the following terms numbered="true" toc="default">
      <name>Terminology</name>

      <t>In this document, PCE is used as defined in <xref
      target="RFC5440"/>: PCE.</t>

      <t>The format="default"
      target="RFC5440"/>. The following terms are defined in this document:</t>

      <t><list style="symbols">
          <t>BRAS: Broadband used as described here:</t>

      <dl indent="8" spacing="normal">
        <dt>BRAS:</dt>

        <dd>Broadband Remote Access Server</t>

          <t>CD: Congestion Degree</t>

          <t>CR: Core Router</t>

          <t>CCDR: Centralized Server</dd>

        <dt>CD:</dt>

        <dd>Congestion Degree</dd>

        <dt>CR:</dt>

        <dd>Core Router</dd>

        <dt>CCDR:</dt>

        <dd>Centralized Control Dynamic Routing</t>

          <t>E2E: End to End</t>

          <t>IDC: Internet Routing</dd>

        <dt>E2E:</dt>

        <dd>End to End</dd>

        <dt>IDC:</dt>

        <dd>Internet Data Center</t>

          <t>MAN: Metro Center</dd>

        <dt>MAN:</dt>

        <dd>Metro Area Network</t>

          <t>QoS: Quality of Service</t>

          <t>SR: Service Router</t>

          <t>TE: Traffic Engineering</t>

          <t>UID: Utilization Network</dd>

        <dt>QoS:</dt>

        <dd>Quality of Service</dd>

        <dt>SR:</dt>

        <dd>Service Router</dd>

        <dt>TE:</dt>

        <dd>Traffic Engineering</dd>

        <dt>UID:</dt>

        <dd>Utilization Increment Degree</t>

          <t>WAN: Wide Degree</dd>

        <dt>WAN:</dt>

        <dd>Wide Area Network</t>
        </list></t> Network</dd>
      </dl>
    </section>

    <section title="CCDR Scenarios"> numbered="true" toc="default">
      <name>CCDR Scenarios</name>

      <t>The following sections describe various deployment scenarios where
      applying the CCDR framework is intuitively expected to produce
      improvements,
      improvements based on the macro-scale properties of the framework and
      the scenario.</t>

      <section title="QoS numbered="true" toc="default">
        <name>QoS Assurance for Hybrid Cloud-based Application"> Cloud-Based Application</name>

        <t>With the emergence of cloud computing technologies, enterprises are
        putting more and more services on a public oriented public-oriented cloud environment,
        but environment
        while keeping core business within their private cloud. The
        communication between the private and public cloud sites will span spans the
        Wide Area Network (WAN) network.
        WAN. The bandwidth requirements between them are variable variable, and the
        background traffic between these two sites varies over time.
        Enterprise applications require assurance of the E2E
        Quality of Service(QoS) QoS performance
        on demand for variable bandwidth services.</t>

        <t>CCDR, which integrates the merits of distributed protocols and the
        power of centralized control, is suitable for this scenario. The
        possible solution framework is illustrated below:</t>

        <figure>
          <artwork><![CDATA[

        <figure anchor="hybrid-cloud-comm"
                title="Hybrid Cloud Communication Scenario">
          <artwork align="left" alt="" name="" type=""><![CDATA[
                         +------------------------+
                         | Cloud Based Cloud-Based Application|
                         +------------------------+
                                     |
                               +-----------+
                               |    PCE    |
                               +-----------+
                                     |
                                     |
                            //--------------\\
                       /////                  \\\\\
  Private Cloud Site ||       Distributed          |Public Cloud Site
                      |       Control Network      |
                       \\\\\                  /////
                            \\--------------//

                 Figure 1: Hybrid Cloud Communication Scenario
]]></artwork>
        </figure>

        <t>As illustrated in Figure 1, <xref target="hybrid-cloud-comm"/>, the source
        and destination of the
        "Cloud Based "Cloud-Based Application" traffic are located
        at "Private Cloud Site" and "Public Cloud Site" Site", respectively.</t>

        <t hangText="Section 4">By

        <t>By default, the traffic path between the private and public cloud
        site is determined by the distributed control network. When an
        application requires the E2E QoS assurance, it can send these requirements
        to the PCE, PCE and let the PCE compute one E2E path path, which is based on the
        underlying network topology and the real traffic information, in order to
        accommodate the application's QoS requirements. <xref format="default"
        target="Section-4.4"/> of this document describes the simulation
        results for this use case.</t>
      </section>

      <section title="Link numbered="true" toc="default">
        <name>Link Utilization Maximization"> Maximization</name>

        <t>Network topology within a Metro Area Network (MAN) is generally in
        a star mode as illustrated in Figure 2, <xref target="star-mode-man"/>, with
        different devices connected to different customer types. The traffic
        from these customers is often in a tidal pattern, pattern with the links
        between the Core
        Router(CR)/Broadband Router (CR) / roadband Remote Access Server(BRAS) Server (BRAS)
        and CR/Service
        Router(SR) Router (SR) experiencing congestion in different periods, because the
        periods due to subscribers under BRAS often use using the network at night, night
        and the leased line users under SR often use using the network during the
        daytime. The link between BRAS/SR and CR must satisfy the maximum
        traffic volume between them, respectively, and this which causes these links often to
        often be
        under-utilized.</t>

        <figure>
          <artwork><![CDATA[ underutilized.</t>

        <figure anchor="star-mode-man"
                title="Star-Mode Network Topology within MAN">
          <artwork align="left" alt="" name="" type=""><![CDATA[
                         +--------+
                         |   CR   |
                         +----|---+
                              |
                    --------|--------|-------|
                  |-------|--------|-------|
                  |       |        |       |
               +--|-+   +-|-   +-|+    +--|-+   +-|+
               |BRAS|   |SR|    |BRAS|   |SR|
               +----+   +--+    +----+   +--+

           Figure 2: Star-mode Network Topology within MAN]]></artwork>
]]></artwork>
        </figure>

        <t>If we consider connecting the BRAS/SR with a local link loop (which
        is usually lower cost), cost) and control the overall MAN topology with the
        CCDR framework, we can exploit the tidal phenomena between the BRAS/CR
        and SR/CR links, maximizing the utilization of these central trunk
        links (which are usually higher cost than the local loops).</t>

        <t><figure>
            <artwork><![CDATA[

        <figure anchor="link-max-ccdr"
                title="Link Utilization Maximization via CCDR">
          <artwork align="left" alt="" name="" type=""><![CDATA[
                                  +-------+
                              -----  PCE  |
                              |   +-------+
                         +----|---+
                         |   CR   |
                         +----|---+
                              |
                    --------|--------|-------|
                  |-------|--------|-------|
                  |       |        |       |
               +--|-+   +-|-   +-|+    +--|-+   +-|+
               |BRAS-----SR|    |BRAS-----SR|
               +----+   +--+    +----+   +--+

        Figure 3: Link Utilization Maximization via CCDR]]></artwork>
          </figure></t>
]]></artwork>
        </figure>
      </section>

      <section title="Traffic numbered="true" toc="default">
        <name>Traffic Engineering for Multi-Domain"> Multi-domain</name>

        <t>Service provider networks are often comprised of different domains,
        interconnected with each other, forming a very complex topology as
        illustrated in Figure 4. <xref target="te-topology"/>. Due to the traffic
        pattern to/from the MAN and IDC, the utilization of the links between
        them are often asymmetric. It is almost impossible to balance the
        utilization of these links via a distributed protocol, but this
        unbalance can be overcome utilizing the CCDR framework.</t>

        <t><figure>
            <artwork><![CDATA[

        <figure anchor="te-topology"
                title="Traffic Engineering for Complex Multi-domain Topology">
          <artwork align="left" alt="" name="" type=""><![CDATA[
               +---+                +---+
                 |MAN|-----------------IDC|
                 +-|-|
               |MAN|----------------|IDC|
               +---+       |        +-|-+        +---+
                 |     ---------|     ----------     |
                   ------|BackBone|------
                 |-----|Backbone|-----|
                 |     ----|----|     ----|-----     |
                 |         |          |
                 +-|--
               +---+       |        ----+        +---+
               |IDC|----------------|MAN|
                 +---|                |---+

     Figure 4: Traffic Engineering for Complex Multi-Domain Topology]]></artwork>
          </figure></t>
               +---+                +---+
]]></artwork>
        </figure>

        <t>A solution for this scenario requires the gathering of NetFlow
        information, analysis of the source/destination AS, autonomous system
        (AS), and determining what is the main cause of the congested link(s). link(s) is.
        After this, the operator can use the external Border Gateway Protocol(eBGP) Protocol
        (eBGP) sessions to schedule the traffic among the different domains
        according to the solution described in the CCDR framework.</t>
      </section>

      <section title="Network numbered="true" toc="default">
        <name>Network Temporal Congestion Elimination"> Elimination</name>

        <t>In more general situations, there are is often temporal congestion
        within the service provider&rsquo;s provider's network, for example example, due to daily or
        weekly periodic bursts, bursts or large events that are scheduled well in
        advance. Such congestion phenomena often appear regularly, and if the
        service provider has methods to mitigate it, it will certainly improve
        their network operations operation capabilities and increase satisfaction for
        their
        customers. CCDR is also suitable for such scenarios, as the controller
        can schedule traffic out of the congested links, lowering
        the their
        utilization of them during these times. <xref format="default"
        target="Section-4.5"/> describes the simulation results of this
        scenario.</t>
      </section>
    </section>

    <section anchor="Section-4" title="CCDR Simulation"> numbered="true" toc="default">
      <name>CCDR Simulation</name>

      <t>The following sections describe a specific case study to illustrate
      the workings of the CCDR algorithm with concrete paths/metrics, as well
      as a procedure for generating topology and traffic matrices and the
      results from simulations applying CCDR for E2E QoS (assured path and
      congestion elimination) over the generated topologies and traffic
      matrices. In all cases examined, the CCDR algorithm produces
      qualitatively significant improvement over the reference (OSPF)
      algorithm, suggesting that CCDR will have broad applicability.</t>

      <t>The structure and scale of the simulated topology is similar to that
      of the real networks. Multiple different traffic matrices were generated
      to simulate different congestion conditions in the network. Only one of
      them is illustrated since the others produce similar results.</t>

      <section title="Case numbered="true" toc="default">
        <name>Case Study for CCDR Algorithm"> Algorithm</name>

        <t>In this section section, we consider a specific network topology for case
        study,
        study: examining the path selected by OSPF and CCDR and evaluating how
        and why the paths differ. Figure 5 <xref target="ccdr-algo"/> depicts the
        topology of the network in this case. There are 8 eight forwarding
        devices in the network. The original cost and utilization are marked
        on it, it as shown in the figure. For example, the original cost and
        utilization for the link
        (1,2) (1, 2) are 3 and 50% 50%, respectively. There are
        two flows: f1 and f2. Both of these two flows are from node 1 to node
        8. For simplicity, it is assumed that the bandwidth of the link in the
        network is 10Mb/s. 10 Mb/s. The flow rate of f1 is 1Mb/s, 1 Mb/s and the flow rate of
        f2 is 2Mb/s. 2 Mb/s. The threshold of the link in congestion is 90%.</t>

        <t>If the OSPF protocol (ISIS protocol, which adopts Dijkstra's algorithm (IS-IS is similar,
        similar because it also use the
        Dijstra's algorithm) uses Dijkstra's algorithm), is applied in the
        network, which adopts
        Dijkstra's algorithm, the two flows from node 1 to node 8 can only use the OSPF
        path (p1: 1-&gt;2-&gt;3-&gt;8). It This is because Dijkstra's algorithm
        mainly considers the original cost of the link. Since CCDR considers
        cost and utilization simultaneously, the same path as OSPF will not be
        selected due to the severe congestion of the link (2,3). (2, 3). In this
        case, f1 will select the path (p2: 1-&gt;5-&gt;6-&gt;7-&gt;8) since
        the new cost of this path is better than that of the OSPF path.
        Moreover, the path p2 is also better than the path (p3:
        1-&gt;2-&gt;4-&gt;7-&gt;8) for for flow f1. However, f2 will not select
        the same path since it will cause the new congestion in the link (6,7). (6, 7).
        As a result, f2 will select the path (p3:
        1-&gt;2-&gt;4-&gt;7-&gt;8).</t>

        <t><figure>
            <artwork><![CDATA[

        <figure anchor="ccdr-algo" title="Case Study for CCDR's Algorithm">
          <artwork align="left" alt="" name="" type=""><![CDATA[
      +----+      f1                +-------> +-----+ ----> +-----+
      |Edge|-----------+            |+--------|  3  |-------|  8  |
      |Node|---------+ |            ||+-----> +-----+ ----> +-----+
      +----+         | |       4/95%|||              6/50%     |
                     | |            |||                   5/60%|
                     | v            |||                        |
      +----+       +-----+ -----> +-----+      +-----+      +-----+
      |Edge|-------|  1  |--------|  2  |------|  4  |------|  7  |
      |Node|-----> +-----+ -----> +-----+7/60% +-----+5/45% +-----+
      +----+  f2      |     3/50%                              |
                      |                                        |
                      |   3/60%   +-----+ 5/55%+-----+   3/75% |
                      +-----------|  5  |------|  6  |---------+
                                  +-----+      +-----+
                (a) Dijkstra's Algorithm (OSPF/ISIS) (OSPF/IS-IS)

      +----+      f1                          +-----+ ----> +-----+
      |Edge|-----------+             +--------|  3  |-------|  8  |
      |Node|---------+ |             |        +-----+ ----> +-----+
      +----+         | |       4/95% |               6/50%    ^|^
                     | |             |                   5/60%|||
                     | v             |                        |||
      +----+       +-----+ -----> +-----+ ---> +-----+ ---> +-----+
      |Edge|-------|  1  |--------|  2  |------|  4  |------|  7  |
      |Node|-----> +-----+        +-----+7/60% +-----+5/45% +-----+
      +----+  f2     ||     3/50%                              |^
                     ||                                        ||
                     ||   3/60%   +-----+5/55% +-----+   3/75% ||
                     |+-----------|  5  |------|  6  |---------+|
                     +----------> +-----+ ---> +-----+ ---------+
                   (b) CCDR Algorithm

              Figure 5: Case Study for CCDR's Algorithm
]]></artwork>
          </figure></t>
        </figure>
      </section>

      <section title="Topology Simulation"> numbered="true" toc="default">
        <name>Topology Simulation</name>

        <t>Moving on from the specific case study, we now consider a class of
        networks more representative of real deployments, with a fully-linked fully linked
        core network that serves to connect edge nodes, which themselves
        connect to only a subset of the core. An example of such a topology is
        shown in Figure 6, <xref target="top-sim"/> for the case of 4 core nodes and 5
        edge nodes. The CCDR simulations presented in this work use topologies
        involving 100 core nodes and 400 edge nodes. While the resulting graph
        does not fit on this page, this scale of network is similar to what is
        deployed in production environments.</t>

        <t><figure>
            <artwork><![CDATA[

        <figure anchor="top-sim" title="Topology of Simulation">
          <artwork align="left" alt="" name="" type=""><![CDATA[
                                +----+
                               /|Edge|\
                              | +----+ |
                              |        |
                              |        |
                +----+    +----+     +----+
                |Edge|----|Core|-----|Core|---------+
                +----+    +----+     +----+         |
                        /  |    \   /   |           |
                  +----+   |     \ /    |           |
                  |Edge|   |      X     |           |
                  +----+   |     / \    |           |
                        \  |    /   \   |           |
                +----+    +----+     +----+         |
                |Edge|----|Core|-----|Core|         |
                +----+    +----+     +----+         |
                            |          |            |
                            |          +------\   +----+
                            |                  ---|Edge|
                            +-----------------/   +----+

                   Figure 6: Topology of Simulation
]]></artwork>
          </figure></t>
        </figure>

        <t>For the simulations, the number of links connecting one edge node
        to the set of core nodes is randomly chosen between 2 to 30, two and thirty,
        and the total number of links is more than 20000. 20,000. Each link has a
        congestion threshold, which can be arbitrarily set set, for example, to (e.g.)
        90% of the nominal link capacity without affecting the simulation
        results.</t>
      </section>

      <section title="Traffic numbered="true" toc="default">
        <name>Traffic Matrix Simulation"> Simulation</name>

        <t>For each topology, a traffic matrix is generated based on the link
        capacity of the topology. It can result in many kinds of situations, situations
        such as congestion, mild congestion congestion, and non-congestion.</t>

        <t>In the CCDR simulation, the dimension of the traffic matrix is
        500*500 (100 core nodes plus 400 edge nodes). About 20% of links are
        overloaded when the Open Shortest Path First (OSPF) protocol is used
        in the network.</t>
      </section>

      <section anchor="Section-4.4" title="CCDR numbered="true" toc="default">
        <name>CCDR End-to-End Path Optimization"> Optimization</name>

        <t>The CCDR E2E path optimization is to find entails finding the best path path, which
        is the lowest in metric value and value, as well as having utilization far below
        the congestion threshold for each link of the path, the
        utilizatioin is far below link&rsquo;s congestion threshold. path. Based on the
        current state of the network, the PCE within CCDR framework combines
        the shortest path algorithm with a penalty theory of classical
        optimization and graph theory.</t>

        <t>Given a background traffic matrix, which is unscheduled, when a set
        of new flows comes into the network, the E2E path optimization finds
        the optimal paths for them. The selected paths bring the least
        congestion degree to the network.</t>

        <t>The link Utilization Increment Degree(UID), Degree (UID), when the new flows are
        added into the network, is shown in Figure 7. <xref
        target="sim-congestion-elimination"/>. The first graph in
        Figure 7 <xref
        target="sim-congestion-elimination"/> is the UID with OSPF OSPF, and the
        second graph is the UID with CCDR E2E path optimization. The average
        UID of the first graph is more than 30%. After path optimization, the
        average UID is less than 5%. The results show that the CCDR E2E path
        optimization has an eye-catching decrease in UID relative to the path
        chosen based on OSPF.</t>

        <t>While real-world results invariably differ from simulations (for
        example, real-world topologies are likely to exhibit correlation in
        the attachment patterns for edge nodes to the core, which are not
        reflected in these results), the dramatic nature of the improvement in
        UID and the choice of simulated topology to resemble real-world
        conditions suggests suggest that real-world deployments will also experience
        significant improvement in UID results.</t>

        <t><figure>
            <artwork><![CDATA[

        <figure anchor="sim-congestion-elimination"
                title="Simulation Results with Congestion Elimination">
          <artwork align="left" alt="" name="" type=""><![CDATA[
       +-----------------------------------------------------------+
       |                *                               *    *    *|
     60|                *                             * * *  *    *|
       |*      *       **     * *         *   *   *  ** * *  * * **|
       |*   * ** *   * **   *** **  *   * **  * * *  ** * *  *** **|
       |* * * ** *  ** **   *** *** **  **** ** ***  **** ** *** **|
     40|* * * ***** ** ***  *** *** **  **** ** *** ***** ****** **|
 UID(%)|* * ******* ** ***  *** ******* **** ** *** ***** *********|
       |*** ******* ** **** *********** *********** ***************|
       |******************* *********** *********** ***************|
     20|******************* ***************************************|
       |******************* ***************************************|
       |***********************************************************|
       |***********************************************************|
      0+-----------------------------------------------------------+
      0    100   200   300   400   500   600   700   800   900  1000
       +-----------------------------------------------------------+
       |                                                           |
     60|                                                           |
       |                                                           |
       |                                                           |
       |                                                           |
     40|                                                           |
 UID(%)|                                                           |
       |                                                           |
       |                                                           |
     20|                                                           |
       |                                                          *|
       |                                     *                    *|
       |        *         *  *    *       *  **                 * *|
      0+-----------------------------------------------------------+
      0    100   200   300   400   500   600   700   800   900  1000
                            Flow Number

          Figure 7: Simulation Result with Congestion Elimination
]]></artwork>
          </figure></t>
        </figure>
      </section>

      <section anchor="Section-4.5"
               title="Network numbered="true" toc="default">
        <name>Network Temporal Congestion Elimination"> Elimination</name>

        <t>During the simulations, different degrees of network congestion
        were considered. To examine the effect of CCDR on link congestion, we
        consider the Congestion Degree (CD) of a link, defined as the link
        utilization beyond its threshold.</t>

        <t>The CCDR congestion elimination performance is shown in Figure 8. <xref
        target="sim-congestion-elimination-2"/>. The first graph is the CD
        distribution before the process of congestion elimination. The average
        CD of all congested links is about 20%. The second graph shown in Figure 8
        <xref target="sim-congestion-elimination-2"/> is the CD distribution
        after using the congestion elimination process. It shows that only 12
        twelve links among the total of 20000 links 20,000 exceed the threshold, and all the
        CD values are less than 3%. Thus, after scheduling of the traffic away
        from the congested paths, the degree of network congestion is greatly
        eliminated and the network utilization is in balance.</t>

        <t><figure>
            <artwork><![CDATA[

        <figure anchor="sim-congestion-elimination-2"
                title="Simulation Results with Congestion Elimination">
          <artwork align="left" alt="" name="" type=""><![CDATA[
	    Before congestion elimination
        +-----------------------------------------------------------+
        |                *                            ** *   ** ** *|
      20|                *                     *      **** * ** ** *|
        |*      *       **     * **       **  **** * ***** *********|
        |*   *  * *   * **** ****** *  ** *** **********************|
      15|* * * ** *  ** **** ********* *****************************|
        |* * ******  ******* ********* *****************************|
  CD(%) |* ********* ******* ***************************************|
      10|* ********* ***********************************************|
        |*********** ***********************************************|
        |***********************************************************|
       5|***********************************************************|
        |***********************************************************|
        |***********************************************************|
       0+-----------------------------------------------------------+
           0            0.5            1            1.5            2

                     After congestion elimination
       +-----------------------------------------------------------+
       |                                                           |
     20|                                                           |
       |                                                           |
       |                                                           |
     15|                                                           |
       |                                                           |
 CD(%) |                                                           |
     10|                                                           |
       |                                                           |
       |                                                           |
     5 |                                                           |
       |                                                           |
       |        *        **  * *  *  **   *  **                 *  |
     0 +-----------------------------------------------------------+
        0            0.5            1            1.5            2
                         Link Number(*10000)

         Figure 8: Simulation Result with Congestion Elimination
]]></artwork>
          </figure></t>
        </figure>

        <t>It is clear that by using an active path-computation mechanism that
        is able to take into account observed link traffic/congestion, the
        occurrence of congestion events can be greatly reduced. Only when a
        preponderance of links in the network are near their congestion
        threshold will the central controller be unable to find a clear path, path
        as opposed to when a static metric-based procedure is used, which will
        produce congested paths once a single bottleneck approaches its
        capacity. More detailed information about the algorithm can be found
        in<xref target="PTCS"/> .</t>
        in <xref format="default" target="PTCS"/>.</t>
      </section>
    </section>

    <section title="CCDR numbered="true" toc="default">
      <name>CCDR Deployment Consideration"> Consideration</name>

      <t>The above CCDR scenarios and simulation results demonstrate that a
      single general solution can be found that copes with multiple complex
      situations. The specific situations considered are not known to have any
      special properties, so it is expected that the benefits demonstrated
      will have general applicability. Accordingly, the integrated use of a
      centralized controller for the more complex optimal path computations in
      a native IP network should result in significant improvements without
      impacting the underlay underlying network infrastructure.</t>

      <t>For intra-domain or inter-domain native IP TE scenarios, the
      deployment of a CCDR solution is similar, similar with the centralized controller
      being able to compute paths and along with no changes being required to the
      underlying network infrastructure. This universal deployment
      characteristic can facilitate a generic traffic engineering solution, traffic-engineering solution
      where operators do not need to differentiate between intra-domain and
      inter-domain TE cases.</t>

      <t>To deploy the CCDR solution, the PCE should collect the underlay underlying
      network topology dynamically, for example example, via BGP-LS<xref Border Gateway Protocol -
      Link State (BGP-LS) <xref format="default" target="RFC7752"/>. It also
      needs to gather the network traffic information periodically from the
      network management platform. The simulation results show that the PCE
      can compute the E2E optimal path within seconds, thus seconds; thus, it can cope with the
      a change of underlay network on to the scale underlying network in a matter of minutes. More agile
      requirements would need to increase the sample rate of underlay the underlying
      network and decrease the detection and notification interval of the underlay
      underlying network. The methods to gather and
      decrease the latency of these gathering this information as well as
      decreasing its latency are out of the scope of this
      draft.</t> document.</t>
    </section>

    <section title="Security Considerations"> numbered="true" toc="default">
      <name>Security Considerations</name>

      <t>This document considers mainly the integration of distributed
      protocols and the central control capability of a PCE. While it
      certainly can ease
      certainly simplify the management of a network in various traffic
      engineering
      traffic-engineering scenarios as described in this document, the
      centralized control also bring brings a new point that may be easily attacked.
      Solutions for CCDR scenarios need to consider protection of the PCE and
      communication with the underlay underlying devices.</t>

      <t><xref format="default" target="RFC5440"/> and <xref format="default"
      target="RFC8253"/> provide additional information.</t>

      <t>The control priority and interaction process should also be carefully
      designed for the combination of the distributed protocol and central
      control. Generally, the central control instruction instructions should have higher
      priority than the forwarding actions determined by the distributed
      protocol. When the communication between PCE and the underlay underlying devices is
      not in function,
      disrupted, the distributed protocol should take over the control
      right of the underlay
      underlying network. <xref format="default"
      target="I-D.ietf-teas-pce-native-ip"/> provides more considerations
      corresponding to the solution.</t>
    </section>

    <section title="IANA Considerations"> numbered="true" toc="default">
      <name>IANA Considerations</name>

      <t>This document does not require any has no IANA actions.</t>
    </section>

    <section title="Contributors">
      <t>Lu Huang contributed to the content of this draft.</t>
    </section>

    <section title="Acknowledgement">
      <t>The author would like to thank Deborah Brungard, Adrian Farrel,
      Huaimo Chen, Vishnu Beeram and Lou Berger for their support and comments
      on this draft.</t>

      <t>Thanks Benjamin Kaduk for his careful review and valuable suggestions
      to this draft. Also thanks Roman Danyliw, Alvaro Retana and &Eacute;ric
      Vyncke for their views and comments.</t>
    </section>
  </middle>

  <back>
    <references title="Normative References">
      <?rfc include="reference.RFC.5440"?>

      <?rfc include="reference.RFC.7752"?>

      <?rfc include="reference.RFC.8253"?>
    <displayreference target="I-D.ietf-teas-pce-native-ip"
		      to="PCE-NATIVE-IP"/>

    <displayreference target="I-D.ietf-pce-pcep-extension-native-ip"
                      to="PCEP-NATIVE-IP-EXT"/>

    <references>
      <name>References</name>

      <references>
        <name>Normative References</name>

        <xi:include
	    href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5440.xml"
                    xmlns:xi="http://www.w3.org/2001/XInclude"/>

        <xi:include
	    href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7752.xml"
                    xmlns:xi="http://www.w3.org/2001/XInclude"/>

        <xi:include
	    href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8253.xml"
                    xmlns:xi="http://www.w3.org/2001/XInclude"/>
      </references>

    <references title="Informative References">
      <?rfc include="reference.RFC.3209"?>

      <?rfc include="reference.RFC.8402"?>

      <?rfc include="reference.RFC.8578"?>

      <?rfc include="reference.I-D.ietf-teas-pce-native-ip"?>

      <?rfc include="reference.I-D.ietf-pce-pcep-extension-native-ip"?>

      <references>
        <name>Informative References</name>

        <xi:include
	    href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3209.xml"
                    xmlns:xi="http://www.w3.org/2001/XInclude"/>

        <xi:include
	    href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8402.xml"
                    xmlns:xi="http://www.w3.org/2001/XInclude"/>

        <xi:include
	    href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8578.xml"
                    xmlns:xi="http://www.w3.org/2001/XInclude"/>

        <xi:include
	    href="https://xml2rfc.ietf.org/public/rfc/bibxml3/reference.I-D.ietf-teas-pce-native-ip.xml"
                    xmlns:xi="http://www.w3.org/2001/XInclude"/>

        <xi:include
	    href="https://xml2rfc.ietf.org/public/rfc/bibxml3/reference.I-D.ietf-pce-pcep-extension-native-ip.xml"
                    xmlns:xi="http://www.w3.org/2001/XInclude"/>

        <reference anchor="PTCS"
                 target="http://ieeexplore.ieee.org/document/8657733">
                   target="https://ieeexplore.ieee.org/document/8657733">
          <front>
            <title>A Practical Traffic Control Scheme With Load Balancing
            Based on PCE Architecture</title>

            <seriesInfo name="DOI" value="10.1109/ACCESS.2019.2902610"/>

            <seriesInfo name="IEEE Access" value="18526773"/>

            <author fullname="Pei Zhang" initials="P." surname="Zhang">
              <organization/>
            </author>

            <author fullname="Kun Xie" initials="K." surname="Xie">
              <organization/>
            </author>

            <author fullname="Caixia Kou" initials="C." surname="Kou">
              <organization/>
            </author>

            <author fullname="Xiaohong Huang" initials="X." surname="Huang">
              <organization/>
            </author>

            <author fullname="Aijun Wang" initials="A." surname="Wang">
              <organization/>
            </author>

            <author fullname="Qiong Sun" initials="Q." surname="Sun">
              <organization/>
            </author>

            <date day="4" month="March" year="2019"/>

          <abstract>
            <t>This document describes a framework by which Integrated
            Services may be supported over Diffserv networks. This memo
            provides information for the Internet community.</t>
          </abstract>
          </front>

        <seriesInfo name="IEEE Access" value="18526773"/>

        <seriesInfo name="DOI" value="10.1109/ACCESS.2019.2902610"/>
        </reference>

      <?rfc ?>
      </references>
    </references>

    <section numbered="false" toc="default">
      <name>Acknowledgements</name>

      <t>The authors would like to thank <contact
      fullname="Deborah       Brungard"/>, <contact
      fullname="Adrian Farrel"/>, <contact fullname="Huaimo Chen"/>, <contact
      fullname="Vishnu Beeram"/>, and <contact fullname="Lou Berger"/> for
      their support and comments on this document.</t>

      <t>Thanks to <contact fullname="Benjamin Kaduk"/> for his careful review
      and valuable suggestions on this document. Also, thanks to <contact
      fullname="Roman Danyliw"/>, <contact fullname="Alvaro Retana"/>, and
      <contact fullname="Eric Vyncke"/> for their reviews and comments.</t>
    </section>

    <section numbered="false" toc="default">
      <name>Contributors</name>

      <t><contact fullname="Lu Huang"/> contributed to the content of this
      document.</t>
    </section>
  </back>
</rfc>