Architecture of an IP/MPLS Network with
Hardened PipesHuawei Technologies Co., LtdQ13 Huawei CampusNo. 156 Beiqing RoadHai-dian DistrictBeijing100095Chinahaojiangtao@huawei.comBharti Airtel, Ltd.Plot No. 16, Udyog Bihar,Phase IV, Gurgaon - 122015HaryanaIndiaPraveen.Maheshwari@in.airtel.comHuawei Technologies Co., LtdQ13 Huawei CampusNo. 156 Beiqing RoadHai-dian DistrictBeijing100095Chinariver.huang@huawei.comHuawei Technologies Co., LtdStockholmSwedenloa@mail01.huawei.comHuawei Technologies Co., LtdQ14 Huawei CampusNo. 156 Beiqing RoadHai-dian DistrictBeijing100095Chinamach.chen@huawei.comStratumThis document describes an IP/MPLS network that has an infrastructure
that can be separated into two or more strata. For the implementation
described in this document, the infrastructure has been separated into
two strata: one for the "Hard Pipes", called the "Hard Pipe Stratum",
and one for the normal IP/MPLS traffic, called the "Normal IP/MPLS
Stratum".This document introduces the concept of a Hard Pipe --
an MPLS Label Switched Path (LSP) or a pseudowire (PW) with a bandwidth
that is guaranteed and can neither be exceeded nor infringed upon.The Hard Pipe stratum does not use statistical multiplexing; for the
LSPs and PWs set up within this stratum, the bandwidth is guaranteed end
to end.The document does not specify any new protocol or
procedures. It does explain how the MPLS standards implementation has
been deployed and operated to meet the requirements from operators that
offer traditional Virtual Leased Line (VLL) services.IP leased line services, Ethernet Private Line (EPL), and Time-Division Multiplexed (TDM) leased line services are commonly offered by
operators worldwide.There are customers, e.g., many enterprises, that insist on TDM
leased line services. They do so regardless of the fact that the same
operators often offer IP leased line services and EPL services at a
lower price and with a guaranteed bandwidth.Today we see a trend that TDM (in particular, Synchronous Digital
Hierarchy / Synchronous Optical Network (SDH/SONET)) networks
are gradually carrying less and less traffic, and many operators want to shut
their TDM networks down to reduce costs.
In light of these trends, vendors and operators have built and deployed
the Hard Pipe service described in this document. It is a way to introduce
leased line service with the same characteristics as TDM leased line
services in IP/MPLS networks.
Even if leased line has been the initial motivation to define the
Hard Pipe technology, the Hard Pipe is by no means limited to support
leased line services. When guaranteed bandwidth is the priority, Virtual
Private Wire Services (VPWS), Virtual Private LAN Services (VPLS), L3
Virtual Private Networks (L3VPN), and IP-only Private LAN Services can be
mapped to a tunnel in the Hard Pipe stratum.EPL and Ethernet Private LAN (EPLAN) are out of scope for this
document.Virtual Leased Line service is used in examples throughout this document.The solution soon to be deployed has an Ethernet infrastructure
that has been split into two parallel logical networks -- two parallel
strata. The first stratum -- the Hard Pipe Stratum -- does not use
statistical multiplexing, and bandwidth is guaranteed end to end. The
second stratum -- the Normal IP/MPLS Stratum -- works as a normal IP/&wj;MPLS
network. The two strata share the same physical network, i.e., routers
and links, but the resource reserved for the Hard Pipe stratum will
never be preempted by the Normal IP/MPLS stratum.The routers will handle the traffic belonging to one stratum
differently from how traffic from the other stratum is handled. This separation
in traffic handling is based on support in hardware.The reader of this document is assumed to be familiar with RFC 3031
and RFC 5921 .This document has the following purposes: to introduce a two strata IP/MPLS network: the purpose of one
of the strata is to provide capabilities for services that are,
from a customer's point of view, functionally identical to TDM-like
leased lines; andto indicate how a router differentiates the traffic of the two
strata.CC: Continuity Check
CV: Connection Verification
L-label: Leased Line label
LSP: Label Switched Path
LSR: Label Switching Router
MPLS-TP: MPLS Transport Profile
NMS: Network Management System
OAM: Operations, Administration, and Maintenance
P: Provider Router
PE: Provider Edge Router
PW: Pseudowire
T-label: Tunnel label
TDM: Time-Division Multiplexing
tLDP: Targeted LDP
VLL: Virtual Leased Line
VPLS: Virtual Private LAN Service
VPWS: Virtual Private Wire Service
The concept of stratified or strata networks has been around for some
time. It appears to have different meaning in different contexts. The
way we use the concept is that we logically assign certain
characteristics to part of the network. The part of the network that has
the special characteristics form one stratum, and the "remainder" forms
a second stratum. The network described in this document uses a single
link-layer technology, Ethernet.In many cases, a whole physical interface is assigned to a single
hard stratum, especially in the scenario where there are many physical
links between two nodes.This document does not address the network configuration
possibilities for Hard Pipe and IP/MPLS strata in detail. There are
configuration options, the basic configuration is that one Hard Pipe
stratum and one IP/MPLS stratum are provisioned.However, it is also possible to provision more than one Hard Pipe
stratum, e.g., if customers want enhanced separation for their leased
line. Even though the main driver for the Hard Pipe technology is the
leased lines, any service for which an operator does not want to use
statistical multiplexing will benefit from using the Hard Pipes.Consider a network with 10 routers and all the links between are
10G Ethernet, such as shown in . This is the
network topology we've used for this model and also (with topology
variations) in our first deployment.In this document, we use the terms "traffic matrix" or "estimated
traffic matrix" to indicate an estimate of how much traffic will
flow between the ingress and egress (PE) nodes. This may be translated
into how much bandwidth is needed per link in the Hard Pipe
stratum.When the intention is to define a Hard Pipe stratum, it is, for
example, possible to start from an estimated traffic matrix to estimate
how much bandwidth to reserve on the links of the Ethernet link-layer
network for the Hard Pipes.Note that the implication is that the normal traffic gets the
remainder of the available bandwidth. Thus, the link-layer network will
be split into two logical networks, or two strata -- one stratum for the hardened pipe network and the other for the "normal" IP and
MPLS traffic. This is shown in Figures and .It is worth noting that even if the figures in this document are
drawn to indicate "bandwidth on the link", the only bandwidth
information that the nodes have available is the bandwidth assigned to
the Hard Pipe stratum and the Normal IP/MPLS stratum. All other
information is kept on the NMS/Controller. The NMS/Controller keeps a global bandwidth
resource table for the Hard Pipe stratum.Given that the starting point is the physical network in and the Hard Pipe stratum as defined in , the Normal IP/MPLS stratum will look as is shown in :
In this document, the concept of stratum network is used
to indicate basically parallel logical networks with
strictly separated resources.
Traffic sent over one stratum network can not infringe on traffic in
the other stratum network.In the case described here, all the traffic in the Hard Pipe
stratum is MPLS encapsulated. A number of the labels have been set
aside so other applications can't allocate them and so the routers
recognize them as belonging to the Hard Pipe application.When the strata are provisioned, the IP/MPLS stratum is set up exactly
as any other IP/MPLS network. The one small difference between
provisioning the Hard Pipe stratum and the IP/MPLS stratum is that no
overbooking is done for the Hard Pipe stratum.Overbooking and/or congestion in the IP/MPLS stratum can not affect
the Hard Pipe stratum.All labels used for the Hard Pipe stratum are "Configured Labels",
i.e., labels that are provisioned and reclaimed by management actions.
These management actions can be by manual actions or by an NMS/Controller or a
centralized controller. For the size of network being deployed, manual
configuration is not practical; we are both provisioning and
reclaiming a label from an NMS/Controller.If an operator wants to set up a leased line, it is first checked
if there is a path available in the Hard Pipe stratum that matches
the criteria (e.g., bandwidth) for the requested leased line. If such a path does exist, it is checked if there is a
matching MPLS tunnel available over that path. If such a tunnel exists, it is used to establish the
leased line by adding L-labels forming an LSP that are
carried by the tunnel. L-labels are known only by the
ingress and egress LSRs.
They are local to the endpoints the same way
that the label signaled by Targeted LDP (tLDP) is
local to the endpoints of a targeted session LSP. (Here, "Targeted LDP"
means LDP as defined in RFC 5036 , using Targeted Hello messages.)
At the same
time, the available bandwidth in the Hard Pipe stratum is
decremented by the bandwidth that is needed for the leased
line for every hop across this stratum in the global
resource table (for the Hard Pipe stratum).If such a tunnel does not exist, it can be established so
that the leased line can be set up as above. If the path does not exist (not enough bandwidth in the Hard Pipe
stratum for the leased line), available bandwidth on the links
is checked to see if the stratum can be expanded to accommodate
such a path. If the Hard Pipe stratum can be expanded, this is done
and the tunnel for the leased line is established as
described above. It is likely that
other modifications of the Hard Pipe stratum, e.g.,
consolidating already set up Hard IP tunnels on to existing
links so that room for new leased lines are created, may have
implications that go well outside the leased line service,
and it is currently not viewed as a fully automated
operation.If it is not possible to expand the Hard Pipe stratum to
accommodate the new path, set up of the leased line will
need to be declined.Thus, given the existence of a viable Hard Pipe stratum, leased lines
are configured in two very simple steps. First, establish a hop-by-hop
tunnel (T-labels), and second, configure the leased lines (L-labels). The
T-labels need to be configured on both the PE and P routers while L-labels
only need to be configured on the PE routers.Note that L-labels may be used for normal IP service , BGP/MPLS VPNs , or PWs .The system as described here generates a very small amount of state,
and most of it is kept in the NMS/Controller.The only configured information that is actually kept on the LSRs
is the information needed for the label swapping procedures, i.e.,
incoming label to outgoing label and port, and whether the label
belongs to the set of labels that are set aside for the Hard Pipe
stratum tunnels; andthe bandwidth available for the Hard Pipe stratum and the
Normal IP/MPLS stratum.The following state needs to be kept in the NMS/Controller: the topology and bandwidth resources available in the Hard Pipe
network; see .the total and available bandwidth per link in the Hard Pipe
network; see .the T-label mappings; see .the L-label mappings; see .the reserved bandwidth, as well as other constraints and the
path per L-label.The annotations given below are neither a programming guideline nor
an indication how this architecture could be implemented. It is rather
an indication of how much data needs to be saved for each stratum and
leased line, as well as where this data could be stored.Considering the Hard Pipe stratum as it has been outlined in , there is actually some additional information
related to the Hard Pipe stratum that not is shown in the figure.Looking explicitly on the link between LSR J and K we find:The annotation [4,0]G means that 4G is allocated to the stratum on
the link between J and K, and of these, 0G has been allocated to a
service.If we were to allocate two tunnels labels from the labels that have
been configured to work within the Hard Pipe stratum, the resource view
would look like this:Note that allocating the tunnel labels does not reserve bandwidth
for the tunnel from the Hard Pipe stratum.When the L-labels are assigned, this will
consume bandwidth; so we need to keep track of the bandwidth per
leased line and the total of bandwidth allocated from the Hard Pipe
stratum.The annotation for the link between J and K could look like
this:The line [4,1.5]G, T1, L1 [.5], L2 [.5], T2, L1 [.5] would be
interpreted as follows:
The Hard Pipe stratum link between nodes J and K has 4G bandwidth allocated; of the total
bandwidth, 1.5G is allocated for leased lines.
Tunnel label T1 carries two leased lines, each of 0.5G, and tunnel
label T2 carries a third leased line of 0.5G.
Note that it is not necessary to keep this information in the nodes;
it is held within the NMS/Controller. Also, it is not necessary to keep
the bandwidth per leased line, but some operations are simplified
(e.g., removing a leased line) if this is done.
Consider the case where an operator wants to set up a leased line
of 0.4G from F to G in the Hard Pipe stratum in .Since there are no constraints other than bandwidth and ingress and
egress PEs, the shortest path will be chosen. A tunnel will be configured
from F to G over the nodes F, H, J, K, L, and G, and a Leased
Line label (a) will be configured on F and G, and the available
resources will be recalculated.A second leased line of 0.3G between the same PEs is easily configured
by adding a new Leased Line label (b) at the ingress and egress PEs.After these operations, a view of the Hard Pipe stratum resources
(available bandwidth) would look like this:If the operator now wishes to establish a new leased line with the
criteria being that it should originate from F and terminate at G, have 0.4G
bandwidth, and pass through node E, then analysis of the Hard Pipe stratum
(after establishing the first two listed lines) and the criteria for the
new leased line would give the following: The existing tunnel cannot be used since it does not pass
through E; a new tunnel need to be established.The hop from F to H cannot be used since the available bandwidth
is insufficient.
Since no existing tunnels meet the criteria requested, a new tunnel will
be set up from F, to B, C, J, K, L, E (the criteria to pass
through E), and to G.
A new L-label (c) to be carried over T2 will be configured on
F and G, and the available resources of the Hard Pipe stratum will be
recalculated.This leased line service uses the MPLS Transport Profile (MPLS-TP)
line protection as it is defined in RFC 6378
and is updated as specified in RFC 7271 and RFC 7324
The CV and CC are run over the tunnels between the Maintenance Entity Group End Points (MEP)
at each end, i.e., the entire tunnel is protected end to end.In general, all of the MPLS-TP Operations, Administration, and
Maintenance (OAM) as defined in RFC 6371 is
v applicable.The security considerations as defined in "Security
Framework for MPLS and GMPLS Networks" (RFC 5920 ) and
"MPLS Transport Profile (MPLS-TP) Security Framework" (RFC 6941 ) apply to this document.The authors want to thank Andy Malis for detailed technical and language review and for valuable comments.