rfc9342.original   rfc9342.txt 
Network Working Group G. Fioccola, Ed. Internet Engineering Task Force (IETF) G. Fioccola, Ed.
Internet-Draft Huawei Technologies Request for Comments: 9342 Huawei Technologies
Obsoletes: 8889 (if approved) M. Cociglio Obsoletes: 8889 M. Cociglio
Intended status: Standards Track Telecom Italia Category: Standards Track Telecom Italia
Expires: 30 March 2023 A. Sapio ISSN: 2070-1721 A. Sapio
Intel Corporation Intel Corporation
R. Sisto R. Sisto
Politecnico di Torino Politecnico di Torino
T. Zhou T. Zhou
Huawei Technologies Huawei Technologies
26 September 2022 December 2022
Clustered Alternate-Marking Method Clustered Alternate-Marking Method
draft-ietf-ippm-rfc8889bis-04
Abstract Abstract
This document generalizes and expands Alternate-Marking methodology This document generalizes and expands the Alternate-Marking
to measure any kind of unicast flow whose packets can follow several methodology to measure any kind of unicast flow whose packets can
different paths in the network that can result in a multipoint-to- follow several different paths in the network; this can result in a
multipoint network. The network clustering approach is presented multipoint-to-multipoint network. The network clustering approach is
and, for this reason, the technique here described is called presented and, for this reason, the technique described here is
"Clustered Alternate-Marking". This document obsoletes RFC 8889. called "Clustered Alternate Marking". This document obsoletes RFC
8889.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This is an Internet Standards Track document.
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document is a product of the Internet Engineering Task Force
and may be updated, replaced, or obsoleted by other documents at any (IETF). It represents the consensus of the IETF community. It has
time. It is inappropriate to use Internet-Drafts as reference received public review and has been approved for publication by the
material or to cite them other than as "work in progress." Internet Engineering Steering Group (IESG). Further information on
Internet Standards is available in Section 2 of RFC 7841.
This Internet-Draft will expire on 30 March 2023. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc9342.
Copyright Notice Copyright Notice
Copyright (c) 2022 IETF Trust and the persons identified as the Copyright (c) 2022 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/ Provisions Relating to IETF Documents
license-info) in effect on the date of publication of this document. (https://trustee.ietf.org/license-info) in effect on the date of
Please review these documents carefully, as they describe your rights publication of this document. Please review these documents
and restrictions with respect to this document. Code Components carefully, as they describe your rights and restrictions with respect
extracted from this document must include Revised BSD License text as to this document. Code Components extracted from this document must
described in Section 4.e of the Trust Legal Provisions and are include Revised BSD License text as described in Section 4.e of the
provided without warranty as described in the Revised BSD License. Trust Legal Provisions and are provided without warranty as described
in the Revised BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction
1.1. Summary of Changes from RFC 8889 . . . . . . . . . . . . 4 1.1. Summary of Changes from RFC 8889
1.2. Requirements Language . . . . . . . . . . . . . . . . . . 5 1.2. Requirements Language
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 2. Terminology
2.1. Correlation with RFC 5644 . . . . . . . . . . . . . . . . 6 2.1. Correlation with RFC 5644
3. Flow Classification . . . . . . . . . . . . . . . . . . . . . 6 3. Flow Classification
4. Extension of the Method to Multipoint Flows . . . . . . . . . 9 4. Extension of the Method to Multipoint Flows
4.1. Monitoring Network . . . . . . . . . . . . . . . . . . . 9 4.1. Monitoring Network
4.2. Network Packet Loss . . . . . . . . . . . . . . . . . . . 10 4.2. Network Packet Loss
5. Network Clustering . . . . . . . . . . . . . . . . . . . . . 11 5. Network Clustering
5.1. Algorithm for Clusters Partition . . . . . . . . . . . . 12 5.1. Algorithm for Clusters Partition
6. Multipoint Packet Loss Measurement . . . . . . . . . . . . . 14 6. Multipoint Packet-Loss Measurement
7. Multipoint Delay and Delay Variation . . . . . . . . . . . . 14 7. Multipoint Delay and Delay Variation
7.1. Delay Measurements on a Multipoint-Paths Basis . . . . . 15 7.1. Delay Measurements on a Multipoint-Paths Basis
7.1.1. Single-Marking Measurement . . . . . . . . . . . . . 15 7.1.1. Single-Marking Measurement
7.2. Delay Measurements on a Single-Packet Basis . . . . . . . 15 7.2. Delay Measurements on a Single-Packet Basis
7.2.1. Single- and Double-Marking Measurement . . . . . . . 15 7.2.1. Single- and Double-Marking Measurement
7.2.2. Hashing Selection Method . . . . . . . . . . . . . . 16 7.2.2. Hashing Selection Method
8. Synchronization and Timing . . . . . . . . . . . . . . . . . 17 8. Synchronization and Timing
9. Recommendations for Deployment . . . . . . . . . . . . . . . 19 9. Recommendations for Deployment
10. A Closed-Loop Performance-Management Approach . . . . . . . . 21 10. A Closed-Loop Performance-Management Approach
11. Security Considerations . . . . . . . . . . . . . . . . . . . 22 11. Security Considerations
12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 12. IANA Considerations
13. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 22 13. References
14. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23 13.1. Normative References
15. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 13.2. Informative References
15.1. Normative References . . . . . . . . . . . . . . . . . . 23 Appendix A. Example of Monitoring Network and Clusters Partition
15.2. Informative References . . . . . . . . . . . . . . . . . 23 Acknowledgements
Appendix A. Example of Monitoring Network and Clusters Contributors
Partition . . . . . . . . . . . . . . . . . . . . . . . . 24 Authors' Addresses
Appendix B. Changes Log . . . . . . . . . . . . . . . . . . . . 27
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 29
1. Introduction 1. Introduction
The Alternate-Marking Method, as described in The Alternate-Marking Method, as described in [RFC9341], is
[I-D.ietf-ippm-rfc8321bis], is applicable to a point-to-point path. applicable to a point-to-point path. The extension proposed in this
The extension proposed in this document applies to the most general document applies to the most general case of a multipoint-to-
case of a multipoint-to-multipoint path and enables flexible and multipoint path and enables flexible and adaptive performance
adaptive performance measurements in a managed network. measurements in a managed network.
The Alternate-Marking methodology consists in splitting the packet The Alternate-Marking methodology consists of splitting the packet
flow into marking blocks and the monitoring parameters are the packet flow into marking blocks, and the monitoring parameters are the
counters and the timestamps for each marking period. In some packet counters and the timestamps for each marking period. In some
applications of the Alternate-Marking method, a lot of flows and applications of the Alternate-Marking Method, a lot of flows and
nodes are to be monitored. Multipoint Alternate-Marking aims to nodes are to be monitored. Multipoint Alternate Marking aims to
reduce these values and makes the performance monitoring more reduce these values and makes the performance monitoring more
flexible in case a detailed analysis is not needed. For instance, by flexible in case a detailed analysis is not needed. For instance, by
considering n measurement points and m monitored flows, the order of considering n measurement points and m monitored flows, the order of
magnitude of the packet counters for each time interval is n*m*2 (1 magnitude of the packet counters for each time interval is n*m*2 (1
per color). The number of measurement points and monitored flows may per color). The number of measurement points and monitored flows may
vary and depends on the portion of the network we are monitoring vary and depends on the portion of the network we are monitoring
(core network, metro network, access network) and the granularity (core network, metro network, access network, etc.) and the
(for each service, each customer). So if both n and m are high granularity (for each service, each customer, etc.). So if both n
values, the packet counters increase a lot, and Multipoint Alternate- and m are high values, the packet counters increase a lot, and
Marking offers a tool to control these parameters. Multipoint Alternate Marking offers a tool to control these
parameters.
The approach presented in this document is applied only to unicast The approach presented in this document is applied only to unicast
flows and not to multicast. Broadcast, Unknown Unicast, and flows and not to multicast. Broadcast, Unknown Unicast, and
Multicast (BUM) traffic is not considered here, because traffic Multicast (BUM) traffic is not considered here, because traffic
replication is not covered by the Multipoint Alternate-Marking replication is not covered by the Multipoint Alternate-Marking
method. Furthermore, it can be applicable to anycast flows, and Method. Furthermore, it can be applicable to anycast flows, and
Equal-Cost Multipath (ECMP) paths can also be easily monitored with Equal-Cost Multipath (ECMP) paths can also be easily monitored with
this technique. this technique.
[I-D.ietf-ippm-rfc8321bis] applies to point-to-point unicast flows [RFC9341] applies to point-to-point unicast flows and BUM traffic.
and BUM traffic. For BUM traffic, the basic method of For BUM traffic, the basic method of [RFC9341] can be easily applied
[I-D.ietf-ippm-rfc8321bis] can easily be applied link by link and link by link; therefore, the multicast flow tree distribution can be
therefore split the multicast flow tree distribution into separate split into separate unicast point-to-point links.
unicast point-to-point links. While, this document and its Clustered
Alternate-Marking method apply to multipoint-to-multipoint unicast
flows, anycast, and ECMP flows.
Therefore, the Alternate-Marking method can be extended to any kind This document and its Clustered Alternate-Marking Method applies to
multipoint-to-multipoint unicast flows, anycast, and ECMP flows.
Therefore, the Alternate-Marking Method can be extended to any kind
of multipoint-to-multipoint paths, and the network-clustering of multipoint-to-multipoint paths, and the network-clustering
approach presented in this document is the formalization of how to approach presented in this document is the formalization of how to
implement this property and allow a flexible and optimized implement this property and allow a flexible and optimized
performance measurement support for network management in every performance measurement support for network management in every
situation. situation.
Without network clustering, it is possible to apply Alternate-Marking Without network clustering, it is possible to apply Alternate Marking
only for all the network or per single flow. Instead, with network only for all the network or per single flow. Instead, with network
clustering, it is possible to use the partition of the network into clustering, it is possible to partition the network into clusters at
clusters at different levels in order to provide the needed degree of different levels in order to provide the needed degree of detail. In
detail. In some circumstances, it is possible to monitor a some circumstances, it is possible to monitor a multipoint network by
multipoint network by monitoring the network clusters, without monitoring the network clusters, without examining in depth. In case
examining in depth. In case of problems (packet loss is measured, or of problems (packet loss is measured or the delay is too high), the
the delay is too high), the filtering criteria could be enhanced in filtering criteria could be enhanced in order to perform a detailed
order to perform a detailed analysis by using a different combination analysis by using a different combination of clusters up to a per-
of clusters up to a per-flow measurement as described in flow measurement as described in [RFC9341].
[I-D.ietf-ippm-rfc8321bis].
This approach fits very well with the Closed-Loop Network and This approach fits very well with the Closed-Loop Network and
Software-Defined Network (SDN) paradigm, where the SDN orchestrator Software-Defined Network (SDN) paradigm, where the SDN orchestrator
and the SDN controllers are the brains of the network and can manage and the SDN controllers are the brains of the network and can manage
flow control to the switches and routers and, in the same way, can flow control to the switches and routers and, in the same way, can
calibrate the performance measurements depending on the desired calibrate the performance measurements depending on the desired
accuracy. An SDN controller application can orchestrate how accuracy. An SDN controller application can orchestrate how
accurately the network performance monitoring is set up by applying accurately the network performance monitoring is set up by applying
the Multipoint Alternate-Marking as described in this document. the Multipoint Alternate Marking as described in this document.
It is important to underline that, as an extension of It is important to underline that, as an extension of [RFC9341], this
[I-D.ietf-ippm-rfc8321bis], this is a methodology document, so the is a methodology document, so the mechanism that can be used to
mechanism that can be used to transmit the counters and the transmit the counters and the timestamps is out of scope here.
timestamps is out of scope here.
This document assumes that the blocks are created according to a This document assumes that the blocks are created according to a
fixed timer as per [I-D.ietf-ippm-rfc8321bis]. Switching after a fixed timer as per [RFC9341]. Switching after a fixed number of
fixed number of packets is possible but it is out of scope here. packets is possible, but it is out of scope here.
Note that the fragmented packets' case can be managed with the Note that the fragmented packets' case can be managed with the
Alternate-Marking methodology and the same guidance provided in Alternate-Marking methodology, and the same guidance provided in
section 6 of [I-D.ietf-ippm-rfc8321bis] apply also in the case of Section 6 of [RFC9341] also applies in the case of Multipoint
Multipoint Alternate-Marking. Alternate Marking.
1.1. Summary of Changes from RFC 8889 1.1. Summary of Changes from RFC 8889
This document defines the Multipoint Alternate-Marking Method, This document defines the Multipoint Alternate-Marking Method,
addressing ambiguities and overtaking its experimental phase in the addressing ambiguities and overtaking its experimental phase in the
original specification [RFC8889]. original specification [RFC8889].
The relevant changes are: The relevant changes are:
* Added the recommendations about the different deployments in case * Added the recommendations about the different deployments in case
one or two flag bits are available for marking (Section 9). one or two flag bits are available for marking (Section 9).
* Changed the structure to improve the readability. * Changed the structure to improve readability.
* Removed the wording about the experimentation of the method and * Removed the wording about the experimentation of the method and
considerations that no longer apply. considerations that no longer apply.
* Revised the description of detailed aspects of the methodology, * Revised the description of detailed aspects of the methodology,
e.g. synchronization and timing. e.g., synchronization and timing.
It is important to note that all the changes are totally backward It is important to note that all the changes are totally backward
compatible with [RFC8889] and no new additional technique has been compatible with [RFC8889], and no new additional technique has been
introduced in this document compared to [RFC8889]. introduced in this document compared to [RFC8889].
1.2. Requirements Language 1.2. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP "OPTIONAL" in this document are to be interpreted as described in
14 [RFC2119] [RFC8174] when, and only when, they appear in all BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here. capitals, as shown here.
2. Terminology 2. Terminology
The definitions of the basic terms are identical to those found in The use of the basic terms are identical to those found in Alternate
Alternate-Marking [I-D.ietf-ippm-rfc8321bis]. It is to be remembered Marking [RFC9341]. It is to be remembered that [RFC9341] is valid
that [I-D.ietf-ippm-rfc8321bis] is valid for point-to-point unicast for point-to-point unicast flows and BUM traffic.
flows and BUM traffic.
The important new terms that need to be explained are listed below: The important new terms are explained below:
Multipoint Alternate-Marking: Extension to Multipoint Alternate Marking: Extension to [RFC9341], valid for
[I-D.ietf-ippm-rfc8321bis], valid for multipoint-to-multipoint multipoint-to-multipoint unicast flows, anycast, and ECMP flows.
unicast flows, anycast, and ECMP flows. It can also be referred It can also be referred to as "Clustered Alternate Marking".
to as Clustered Alternate-Marking.
Flow definition: The concept of flow is generalized in this Flow definition: The concept of flow is generalized in this
document. The identification fields are selected without any document. The identification fields are selected without any
constraints and, in general, the flow can be a multipoint-to- constraints and, in general, the flow can be a multipoint-to-
multipoint flow, as a result of aggregate point-to-point flows. multipoint flow, as a result of aggregate point-to-point flows.
Monitoring Network: Identified with the nodes of the network that Monitoring network: Identified with the nodes of the network that
are the measurement points (MPs) and the links that are the are the measurement points (MPs) and the links that are the
connections between MPs. The monitoring network graph depends on connections between MPs. The monitoring network graph depends on
the flow definition, so it can represent a specific flow or the the flow definition, so it can represent a specific flow or the
entire network topology as aggregate of all the flows. Each node entire network topology as aggregate of all the flows. Each node
of the monitoring network cannot be both a source and a of the monitoring network cannot be both a source and a
destination of the flow. destination of the flow.
Cluster: Smallest identifiable non-trivial subnetwork of the Cluster: Smallest identifiable non-trivial subnetwork of the entire
entire monitoring network graph that still satisfies the condition monitoring network graph that still satisfies the condition that
that the number of packets that go in is the same as the number the number of packets that go in is the same as the number that go
that go out. A cluster partition algorithm, such as that found in out. A cluster partition algorithm, such as that found in
Section 5.1, can be applied to split the monitoring network into Section 5.1, can be applied to split the monitoring network into
clusters. clusters.
Multipoint metrics: Packet loss, delay and delay variation are Multipoint metrics: Packet loss, delay, and delay variation are
extended to the case of multipoint flows. It is possible to extended to the case of multipoint flows. It is possible to
compute these metrics on the basis of multipoint paths in order to compute these metrics on the basis of multipoint paths in order to
associate the measurements to a cluster, a combination of associate the measurements to a cluster, a combination of
clusters, or the entire monitored network. For delay and delay clusters, or the entire monitored network. For delay and delay
variation, it is also possible to define the metrics on a single- variation, it is also possible to define the metrics on a single-
packet basis, and it means that the multipoint path is used to packet basis, and it means that the multipoint path is used to
easily couple packets between input and output nodes of a easily couple packets between input and output nodes of a
multipoint path. multipoint path.
The next section highlights the correlation with the terms used in The next section highlights the correlation with the terms used in
RFC 5644 [RFC5644]. [RFC5644].
2.1. Correlation with RFC 5644 2.1. Correlation with RFC 5644
RFC 5644 [RFC5644] is limited to active measurements using a single [RFC5644] is limited to active measurements using a single source
source packet or stream. Its scope is also limited to observations packet or stream. Its scope is also limited to observations of
of corresponding packets along the path (spatial metric) and at one corresponding packets along the path (spatial metric) and at one or
or more destinations (one-to-group) along the path. more destinations (one-to-group) along the path.
Instead, the scope of this memo is to define multiparty metrics for Instead, the scope of this memo is to define multiparty metrics for
passive and hybrid measurements in a group-to-group topology with passive and hybrid measurements in a group-to-group topology with
multiple sources and destinations. multiple sources and destinations.
RFC 5644 [RFC5644] introduces metric names that can be reused here [RFC5644] introduces metric names that can be reused here but have to
but have to be extended and rephrased to be applied to the Alternate- be extended and rephrased to be applied to the Alternate-Marking
Marking schema: schema:
a. the multiparty metrics are not only one-to-group metrics but can a. the multiparty metrics are not only one-to-group metrics but can
be also group-to-group metrics; also be group-to-group metrics;
b. the spatial metrics, used for measuring the performance of b. the spatial metrics, used for measuring the performance of
segments of a source to destination path, are applied here to segments of a source-to-destination path, are applied here to
clusters. clusters.
3. Flow Classification 3. Flow Classification
A unicast flow is identified by all the packets having a set of A unicast flow is identified by all the packets having a set of
common characteristics. This definition is inspired by RFC 7011 common characteristics. This definition is inspired by [RFC7011].
[RFC7011].
As an example, by considering a flow as all the packets sharing the As an example, by considering a flow as all the packets sharing the
same source IP address or the same destination IP address, it is easy same source IP address or the same destination IP address, it is easy
to understand that the resulting pattern will not be a point-to-point to understand that the resulting pattern will not be a point-to-point
connection, but a point-to-multipoint or multipoint-to-point connection but a point-to-multipoint or multipoint-to-point
connection. connection.
In general, a flow can be defined by a set of selection rules used to In general, a flow can be defined by a set of selection rules used to
match a subset of the packets processed by the network device. These match a subset of the packets processed by the network device. These
rules specify a set of Layer 3 and Layer 4 header fields rules specify a set of Layer 3 and Layer 4 header fields
(identification fields) and the relative values that must be found in (identification fields) and the relative values that must be found in
matching packets. matching packets.
The choice of the identification fields directly affects the type of The choice of the identification fields directly affects the type of
paths that the flow would follow in the network. In fact, it is paths that the flow would follow in the network. In fact, it is
possible to relate a set of identification fields with the pattern of possible to relate a set of identification fields with the pattern of
the resulting graphs, as listed in Figure 1. the resulting graphs, as listed in Figure 1.
A TCP 5-tuple usually identifies flows following either a single path A TCP 5-tuple usually identifies flows following either a single path
or a point-to-point multipath (in the case of load balancing). On or a point-to-point multipath (in the case of load balancing). On
the contrary, a single source address selects aggregate flows the contrary, a single source address selects aggregate flows
following a point-to-multipoint, while a multipoint-to-point can be following a point-to-multipoint path, while a multipoint-to-point
the result of a matching on a single destination address. In the path can be the result of a matching on a single destination address.
case where a selection rule and its reverse are used for In the case where a selection rule and its reverse are used for
bidirectional measurements, they can correspond to a point-to- bidirectional measurements, they can correspond to a point-to-
multipoint in one direction and a multipoint-to-point in the opposite multipoint path in one direction and a multipoint-to-point path in
direction. the opposite direction.
So the flows to be monitored are selected into the monitoring points So the flows to be monitored are selected into the monitoring points
using packet selection rules, which can also change the pattern of using packet selection rules, which can also change the pattern of
the monitored network. the monitored network.
Note that, more generally, the flow can be defined at different Note that, more generally, the flow can be defined at different
levels based on the potential encapsulation, and additional levels based on the potential encapsulation, and additional
conditions that are not in the packet header can also be included as conditions that are not in the packet header can also be included as
part of matching criteria. part of matching criteria.
The Alternate-Marking method is applicable only to a single path (and The Alternate-Marking Method is applicable only to a single path (and
partially to a one-to-one multipath), so the extension proposed in partially to a one-to-one multipath), so the extension proposed in
this document is suitable also for the most general case of this document is suitable also for the most general case of
multipoint-to-multipoint, which embraces all the other patterns of multipoint-to-multipoint, which embraces all the other patterns in
Figure 1. Figure 1.
point-to-point single path point-to-point single path
+------+ +------+ +------+ +------+ +------+ +------+
---<> R1 <>----<> R2 <>----<> R3 <>--- ---<> R1 <>----<> R2 <>----<> R3 <>---
+------+ +------+ +------+ +------+ +------+ +------+
point-to-point multipath point-to-point multipath
+------+ +------+
<> R2 <> <> R2 <>
skipping to change at page 9, line 32 skipping to change at line 392
Figure 1: Flow Classification Figure 1: Flow Classification
The case of unicast flow is considered in Figure 1. The anycast flow The case of unicast flow is considered in Figure 1. The anycast flow
is also covered, since it is only a special case of a unicast flow if is also covered, since it is only a special case of a unicast flow if
routing is stable throughout the measurement period. Furthermore, an routing is stable throughout the measurement period. Furthermore, an
ECMP flow is in scope by definition, since it is a point-to- ECMP flow is in scope by definition, since it is a point-to-
multipoint unicast flow. multipoint unicast flow.
4. Extension of the Method to Multipoint Flows 4. Extension of the Method to Multipoint Flows
By using the Alternate-Marking method, only point-to-point paths can By using the Alternate-Marking Method, only point-to-point paths can
be monitored. To have an IP (TCP/UDP) flow that follows a point-to- be monitored. To have an IP (TCP/UDP) flow that follows a point-to-
point path, in general we have to define, with a specific value, 5 point path, in general we have to define, with a specific value, 5
identification fields (IP Source, IP Destination, Transport Protocol, identification fields (IP Source, IP Destination, Transport Protocol,
Source Port, Destination Port). Source Port, and Destination Port).
Multipoint Alternate-Marking enables the performance measurement for Multipoint Alternate Marking enables the performance measurement for
multipoint flows selected by identification fields without any multipoint flows selected by identification fields without any
constraints (even the entire network production traffic). It is also constraints (even the entire network production traffic). It is also
possible to use multiple marking points for the same monitored flow. possible to use multiple marking points for the same monitored flow.
4.1. Monitoring Network 4.1. Monitoring Network
The monitoring network is deduced from the production network by The monitoring network is deduced from the production network by
identifying the nodes of the graph that are the measurement points, identifying the nodes of the graph that are the measurement points
and the links that are the connections between measurement points. and the links that are the connections between measurement points.
It can be modeled as a set of nodes and a set of directed arcs which It can be modeled as a set of nodes and a set of directed arcs that
connect pairs of nodes. connect pairs of nodes.
There are some techniques that can help with the building of the There are some techniques that can help with the building of the
monitoring network (as an example, see [I-D.ietf-ippm-route]). In monitoring network (as an example, see [RFC9198]). In general, there
general, there are different options: the monitoring network can be are different options: the monitoring network can be obtained by
obtained by considering all the possible paths for the traffic or considering all the possible paths for the traffic or periodically
periodically checking the traffic (e.g. daily, weekly, monthly) and checking the traffic (e.g., daily, weekly, and monthly) and updating
updating the graph as appropriate, but this is up to the Network the graph as appropriate, but this is up to the Network Management
Management System (NMS) configuration. System (NMS) configuration.
So a graph model of the monitoring network can be built according to So a graph model of the monitoring network can be built according to
the Alternate-Marking method: the monitored interfaces and links are the Alternate-Marking Method, where the monitored interfaces and
identified. Only the measurement points and links where the traffic links are identified. Only the measurement points and links where
has flowed have to be represented in the graph. the traffic has flowed have to be represented in the graph.
A simple example of a monitoring network graph is showed in A simple example of a monitoring network graph is shown in
Appendix A. Appendix A.
Each monitoring point is characterized by the packet counter that Each monitoring point is characterized by the packet counter that
refers only to a marking period of the monitored flow. Also, it is refers only to a marking period of the monitored flow. Also, it is
assumed that there be a monitoring point at all possible egress assumed that there is a monitoring point at all possible egress
points of the multipoint monitored network. points of the multipoint monitored network.
The same is also applicable for the delay, but it will be described The same is also applicable for the delay, but it will be described
in the following sections. in the following sections.
The rest of the document assumes that the traffic is going from left The rest of the document assumes that the traffic is going from left
to right in order to simplify the explanation. But the analysis done to right in order to simplify the explanation. But the analysis done
for one direction applies equally to all directions. for one direction applies equally to all directions.
4.2. Network Packet Loss 4.2. Network Packet Loss
Since all the packets of the considered flow leaving the network have Since all the packets of the considered flow leaving the network have
previously entered the network, the number of packets counted by all previously entered the network, the number of packets counted by all
the input nodes is always greater than, or equal to, the number of the input nodes is always greater than, or equal to, the number of
packets counted by all the output nodes. It is assumed that routing packets counted by all the output nodes. It is assumed that routing
is stable during the measurement period while packet fragmentation is stable during the measurement period while packet fragmentation
must be handled as described in [I-D.ietf-ippm-rfc8321bis]. must be handled as described in [RFC9341].
In the case of no packet loss occurring in the marking period, if all In the case of no packet loss occurring in the marking period, if all
the input and output points of the network domain to be monitored are the input and output points of the network domain to be monitored are
measurement points, the sum of the number of packets on all the measurement points, the sum of the number of packets on all the
ingress interfaces equals the number on egress interfaces for the ingress interfaces equals the number on egress interfaces for the
monitored flow. In this circumstance, if no packet loss occurs, the monitored flow. In this circumstance, if no packet loss occurs, the
intermediate measurement points only have the task of splitting the intermediate measurement points only have the task of splitting the
measurement. measurement.
It is possible to define the Network Packet Loss of one monitored It is possible to define the network packet loss of one monitored
flow for a single period. In a packet network, the number of lost flow for a single period. In a packet network, the number of lost
packets is the number of packets counted by the input nodes minus the packets is the number of packets counted by the input nodes minus the
number of packets counted by the output nodes. This is true for number of packets counted by the output nodes. This is true for
every packet flow in each marking period. every packet flow in each marking period.
The monitored network packet loss with n input nodes and m output The monitored network packet loss with n input nodes and m output
nodes is given by: nodes is given by:
PL = (PI1 + PI2 +...+ PIn) - (PO1 + PO2 +...+ POm) PL = (PI1 + PI2 +...+ PIn) - (PO1 + PO2 +...+ POm)
where: where:
PL is the network packet loss (number of lost packets) * PL is the network packet loss (number of lost packets);
PIi is the number of packets flowed through the i-th input node in * PIi is the number of packets flowed through the i-th input node in
this period this period; and
POj is the number of packets flowed through the j-th output node in * POj is the number of packets flowed through the j-th output node
this period in this period.
The equation is applied on a per-time-interval basis and a per-flow The equation is applied on a per-time-interval basis and a per-flow
basis: basis:
The reference interval is the Alternate-Marking period, as defined * The reference interval is the Alternate-Marking period, as defined
in [I-D.ietf-ippm-rfc8321bis]. in [RFC9341].
The flow definition is generalized here. Indeed, as described * The flow definition is generalized here. Indeed, as described
before, a multipoint packet flow is considered, and the before, a multipoint packet flow is considered, and the
identification fields can be selected without any constraints. identification fields can be selected without any constraints.
5. Network Clustering 5. Network Clustering
The previous equation of Section 4.2 can determine the number of The previous equation of Section 4.2 can determine the number of
packets lost globally in the monitored network, exploiting only the packets lost globally in the monitored network, exploiting only the
data provided by the counters in the input and output nodes. data provided by the counters in the input and output nodes.
In addition, it is possible to leverage the data provided by the In addition, it is possible to leverage the data provided by the
skipping to change at page 12, line 42 skipping to change at line 534
clusters based on the topological information so that they are clusters based on the topological information so that they are
applicable to all the possible flows in the monitored network. applicable to all the possible flows in the monitored network.
Note that, in case of translation or encapsulation, the cluster Note that, in case of translation or encapsulation, the cluster
properties must also be invariant. properties must also be invariant.
5.1. Algorithm for Clusters Partition 5.1. Algorithm for Clusters Partition
A simple algorithm can be applied in order to split the monitoring A simple algorithm can be applied in order to split the monitoring
network into clusters. This can be done for each direction network into clusters. This can be done for each direction
separately, indeed a node cannot be both a source and a destination. separately; indeed, a node cannot be both a source and a destination.
The clusters partition is based on the monitoring network graph, The clusters partition is based on the monitoring network graph,
which can be valid for a specific flow or can also be general and which can be valid for a specific flow or can also be general and
valid for the entire network topology. valid for the entire network topology.
It is a two-step algorithm: It is a two-step algorithm:
* Group the links where there is the same starting node; * Group the links where there is the same starting node;
* Join the grouped links with at least one ending node in common. * Join the grouped links with at least one ending node in common.
skipping to change at page 13, line 17 skipping to change at line 557
the different links if they have the same starting node. Note that the different links if they have the same starting node. Note that
it is possible to start from any link, and the procedure will work. it is possible to start from any link, and the procedure will work.
Following this classification, the second step implies eventually Following this classification, the second step implies eventually
joining the groups classified in the first step by looking at the joining the groups classified in the first step by looking at the
ending nodes. If different groups have at least one common ending ending nodes. If different groups have at least one common ending
node, they are put together and belong to the same set. After the node, they are put together and belong to the same set. After the
application of the two steps of the algorithm, each one of the application of the two steps of the algorithm, each one of the
composed sets of links, together with the endpoint nodes, constitutes composed sets of links, together with the endpoint nodes, constitutes
a cluster. a cluster.
A simple application of the clusters partition is showed in A simple application of the clusters partition is shown in
Appendix A. Appendix A.
The algorithm, as applied in the example of a point-to-multipoint The algorithm, as applied in the example of a point-to-multipoint
network, works for the more general case of multipoint-to-multipoint network, works for the more general case of a multipoint-to-
network in the same way. It should be highlighted that for a multipoint network in the same way. It should be highlighted that
multipoint-to-multipoint network the multiple sources MUST mark for a multipoint-to-multipoint network, the multiple sources MUST
coherently the traffic and MUST be synchronized with all the other mark the traffic coherently and MUST be synchronized with all the
nodes according to the timing requirements detailed in Section 8. other nodes according to the timing requirements detailed in
Section 8.
When the clusters partition is done, the calculation of packet loss, When the clusters partition is done, the calculation of packet loss,
delay and delay variation can be made on a cluster basis. Note that delay, and delay variation can be made on a cluster basis. Note that
the packet counters for each marking period permit calculating the the packet counters for each marking period permit calculating the
packet rate on a cluster basis, so Committed Information Rate (CIR) packet rate on a cluster basis, so Committed Information Rate (CIR)
and Excess Information Rate (EIR) could also be deduced on a cluster and Excess Information Rate (EIR) could also be deduced on a cluster
basis. basis.
Obviously, by combining some clusters in a new connected subnetwork Obviously, by combining some clusters in a new connected subnetwork,
the packet-loss rule is still true. So it is also possible to the packet-loss rule is still true. So it is also possible to
consider combinations of clusters if and where it suits. consider combinations of clusters if and where it suits.
In this way, in a very large network, there is no need to configure In this way, in a very large network, there is no need to configure
detailed filter criteria to inspect the traffic. It is possible to detailed filter criteria to inspect the traffic. It is possible to
check a multipoint network and, in case of problems, go deep with a check a multipoint network and, in case of problems, go deep with a
step-by-step cluster analysis, but only for the cluster or step-by-step cluster analysis, but only for the cluster or
combination of clusters where the problem happens. combination of clusters where the problem happens.
In summary, once a flow is defined, the algorithm to build the In summary, once a flow is defined, the algorithm to build the
clusters partition is based on topological information; therefore, it clusters partition is based on topological information; therefore, it
considers all the possible links and nodes that could potentially be considers all the possible links and nodes that could potentially be
crossed by the given flow, even if there is no traffic. So, if the crossed by the given flow, even if there is no traffic. So if the
flow does not enter or traverse all the nodes, the counters have a flow does not enter or traverse all the nodes, the counters have a
non-zero value for the involved nodes and a zero value for the other non-zero value for the involved nodes and a zero value for the other
nodes without traffic; but in the end, all the formulas are still nodes without traffic; but in the end, all the formulas are still
valid. valid.
The algorithm described above is an iterative clustering algorithm The algorithm described above is an iterative clustering algorithm
since it executes steps in iterations, but it is also possible to since it executes steps in iterations, but it is also possible to
apply a recursive clustering algorithm as detailed in apply a recursive clustering algorithm as detailed in
[IEEE-ACM-ToN-MPNPM]. [IEEE-ACM-TON-MPNPM].
The complete and mathematical analysis of the possible algorithms for The complete and mathematical analysis of the possible algorithms for
clusters partition, including the considerations in terms of the clusters partition, including the considerations in terms of
efficiency and a comparison between the different methods, is in the efficiency and a comparison between the different methods, is in the
paper [IEEE-ACM-ToN-MPNPM]. paper [IEEE-ACM-TON-MPNPM].
6. Multipoint Packet Loss Measurement 6. Multipoint Packet-Loss Measurement
The Network Packet Loss, defined in Section 4.2, valid for the entire The network packet loss, defined in Section 4.2, valid for the entire
monitored flow, can easily be extended to each multipoint path (e.g., monitored flow, can easily be extended to each multipoint path (e.g.,
the whole multipoint network, a cluster, or a combination of the whole multipoint network, a cluster, or a combination of
clusters). In this way it is possible to calculate Multipoint Packet clusters). In this way, it is possible to calculate Multipoint
Loss that is representative of a multipoint path. Packet Loss that is representative of a multipoint path.
The same equation of Section 4.2 can be applied to a generic The same equation of Section 4.2 can be applied to a generic
multipoint path like a cluster or a combination of clusters, where multipoint path like a cluster or a combination of clusters, where
the number of packets are those entering and leaving the multipoint the number of packets are those entering and leaving the multipoint
path. path.
By applying the algorithm described in Section 5.1, it is possible to By applying the algorithm described in Section 5.1, it is possible to
split the monitoring network into clusters. Then, packet loss can be split the monitoring network into clusters. Then, packet loss can be
measured on a cluster basis for each single period by considering the measured on a cluster basis for each single period by considering the
counters of the input and output nodes that belong to the specific counters of the input and output nodes that belong to the specific
cluster. This can be done for every packet flow in each marking cluster. This can be done for every packet flow in each marking
period. period.
7. Multipoint Delay and Delay Variation 7. Multipoint Delay and Delay Variation
The same line of reasoning can be applied to delay and delay The same line of reasoning can be applied to delay and delay
variation. The delay measurement methods defined in variation. The delay measurement methods defined in [RFC9341] can be
[I-D.ietf-ippm-rfc8321bis] can be extended to the case of multipoint extended to the case of multipoint flows. It is important to
flows. It is important to highlight that both delay and delay- highlight that both delay and delay-variation measurements make sense
variation measurements make sense in a multipoint path. The delay in a multipoint path. The delay variation is calculated by
variation is calculated by considering the same packets selected for considering the same packets selected for measuring the delay.
measuring the delay.
In general, it is possible to perform delay and delay-variation In general, it is possible to perform delay and delay-variation
measurements on the basis of multipoint paths or single packets: measurements on the basis of multipoint paths or single packets:
* Delay measurements on the basis of multipoint paths mean that the * Delay measurements on the basis of multipoint paths mean that the
delay value is representative of an entire multipoint path (e.g., delay value is representative of an entire multipoint path (e.g.,
the whole multipoint network, a cluster, or a combination of the whole multipoint network, a cluster, or a combination of
clusters). clusters).
* Delay measurements on a single-packet basis mean that it is * Delay measurements on a single-packet basis mean that it is
skipping to change at page 15, line 27 skipping to change at line 663
or the entire monitored network. or the entire monitored network.
The average latency can be measured as the difference between the The average latency can be measured as the difference between the
weighted averages of the mean timestamps of the sets of output and weighted averages of the mean timestamps of the sets of output and
input nodes. This means that, in the calculation, it is possible to input nodes. This means that, in the calculation, it is possible to
weigh the timestamps with the number of packets for each endpoint. weigh the timestamps with the number of packets for each endpoint.
Note that, since the one-way delay value is representative of a Note that, since the one-way delay value is representative of a
multipoint path, it is possible to calculate the two-way delay of a multipoint path, it is possible to calculate the two-way delay of a
multipoint path by summing the one-way delays of the two directions, multipoint path by summing the one-way delays of the two directions,
similarly to [I-D.ietf-ippm-rfc8321bis]. similarly to [RFC9341].
7.2. Delay Measurements on a Single-Packet Basis 7.2. Delay Measurements on a Single-Packet Basis
7.2.1. Single- and Double-Marking Measurement 7.2.1. Single- and Double-Marking Measurement
Delay and delay-variation measurements associated with only one Delay and delay-variation measurements associated with only one
picked packet per period, both single and double marked, cannot be picked packet per period, both single and double marked, cannot be
easily performed in a multipoint scenario since there are some easily performed in a multipoint scenario since there are some
limitations: limitations:
Single marking based on the first/last packet of the interval does * Single Marking based on the first/last packet of the interval does
not work properly. Indeed, by considering a point-to-multipoint not work properly. Indeed, by considering a point-to-multipoint
scenario, it is not possible to recognize which path the first scenario, it is not possible to recognize which path the first
packet of each block takes over the multipoint flow in order to packet of each block takes over the multipoint flow in order to
correlate it. This is also true for the general case of the correlate it. This is also true for the general case of the
multipoint-to-multipoint scenario. multipoint-to-multipoint scenario.
Double marking or multiplexed marking works but only through * Double Marking or multiplexed marking works but only through
statistical means. In a point-to-multipoint scenario, by statistical means. In a point-to-multipoint scenario, by
selecting only a single packet with the second marking for each selecting only a single packet with the second marking for each
block, it is possible to follow and calculate the delay for that block, it is possible to follow and calculate the delay for that
picked packet. But the measurement can only be done for a single picked packet. But the measurement can only be done for a single
path in each marking period. To traverse all the paths of the path in each marking period. To traverse all the paths of the
multipoint flow, it can theoretically be done by continuing the multipoint flow, it can theoretically be done by continuing the
measurement for the following marking periods and expect to span measurement for the following marking periods and expect to span
all the paths. In the general case of a multipoint-to-multipoint all the paths. In the general case of a multipoint-to-multipoint
path, it is also needed to take into account the multiple source path, it is also needed to take into account the multiple source
nodes which complicate the correlation of the samples. In this nodes that complicate the correlation of the samples. In this
case, it can be possible to select the second marked packet only case, it can be possible to select the second marked packet only
for a source node at a time for each block and cover the remaining for a source node at a time for each block and cover the remaining
source nodes one by one in the next marking periods. source nodes one by one in the next marking periods.
Note that, since the one-way delay measurement is done on a single- Note that, since the one-way delay measurement is done on a single-
packet basis, it is always possible to calculate the two-way delay packet basis, it is always possible to calculate the two-way delay,
but it is not immediate since it is necessary to couple the but it is not immediate since it is necessary to couple the
measurement on each single path with the opposite direction. In this measurement on each single path with the opposite direction. In this
case the NMS can do the calculation. case, the NMS can do the calculation.
If a delay measurement is performed for more than one picked packet If a delay measurement is performed for more than one picked packet
and for all the paths of the multipoint flow in the same marking and for all the paths of the multipoint flow in the same marking
period, neither the single- nor the double-marking method are period, neither the Single- nor the Double-Marking Method are
applicable in the multipoint scenario. The packets follow different applicable in the multipoint scenario. The packets follow different
paths and it becomes very difficult to correlate marked packets in a paths, and it becomes very difficult to correlate marked packets in a
multipoint-to-multipoint path if there are more than one per period. multipoint-to-multipoint path if there are more than one per period.
A desirable option is to monitor simultaneously all the paths of a A desirable option is to monitor simultaneously all the paths of a
multipoint path in the same marking period. For this purpose, multipoint path in the same marking period. For this purpose,
hashing can be used, as reported in the next section. hashing can be used, as reported in the next section.
7.2.2. Hashing Selection Method 7.2.2. Hashing Selection Method
RFCs 5474 [RFC5474] and 5475 [RFC5475] introduce sampling and Sampling and filtering techniques for IP packet selection are
filtering techniques for IP packet selection. introduced in [RFC5474] and [RFC5475].
The hash-based selection methodologies for delay measurement can work The hash-based selection methodologies for delay measurement can work
in a multipoint-to-multipoint path and can be used either coupled to in a multipoint-to-multipoint path and can be used either coupled to
mean delay or stand-alone. mean delay or standalone.
[IEEE-Network-PNPM] introduces how to use the hash method (RFC 5474 [IEEE-NETWORK-PNPM] introduces how to use the hash method (see
[RFC5474] and RFC 5475 [RFC5475]) combined with the Alternate-Marking [RFC5474] and [RFC5475]) combined with the Alternate-Marking Method
method for point-to-point flows. It is also called Mixed Hashed for point-to-point flows. It is also called "Mixed Hashed Marking"
Marking because it refers to the conjunction of the marking method because it refers to the conjunction of the marking method and the
and the hashing technique. It involves only the single marking, hashing technique. It involves only the Single Marking; indeed, it
indeed it is supposed that double marking is not used with hashing. is supposed that Double Marking is not used with hashing. The
The coupling of the single marking with the hashing selection allows coupling of the Single Marking with the hashing selection allows
choosing a simplified hash function since the alternation of blocks choosing a simplified hash function since the alternation of blocks
gives temporal boundaries for the hashing samples. The marking gives temporal boundaries for the hashing samples. The marking
batches anchor the samples selected with hashing and this eases the batches anchor the samples selected with hashing, and this eases the
correlation of the hashing packets along the path. For example, in correlation of the hashing packets along the path. For example, in
case a hashed sample is lost, it is confined to the considered block case a hashed sample is lost, it is confined to the considered block
without affecting the identification of the samples for the following without affecting the identification of the samples for the following
blocks. blocks.
Using the hash-based sampling, the number of samples in each block Using the hash-based sampling, the number of samples in each block
may vary a lot because it depends on the packet rate that is may vary a lot because it depends on the packet rate that is
variable. A dynamic approach can help to have an almost fixed number variable. A dynamic approach can help to have an almost fixed number
of samples for each marking period, and this is a better option for of samples for each marking period, and this is a better option for
making regular measurements over time. In the hash-based sampling, making regular measurements over time. In the hash-based sampling,
Alternate-Marking is used to create periods, so that hash-based Alternate Marking is used to create periods, so that hash-based
samples are divided into batches, which allows anchoring the selected samples are divided into batches, which allows anchoring the selected
samples to their period. Moreover, in a dynamic hash-based sampling, samples to their period. Moreover, in a dynamic hash-based sampling,
it can be possible to dynamically adapt the length of the hash value it can be possible to dynamically adapt the length of the hash value
to meet the current packet rate, so that the number of samples is to meet the current packet rate, so that the number of samples is
bounded in each marking period. bounded in each marking period.
In a multipoint environment, the hashing selection may be the In a multipoint environment, the hashing selection may be the
solution for performing delay measurements on specific packets and solution for performing delay measurements on specific packets and
overcoming the single- and double-marking limitations. overcoming the Single- and Double-Marking limitations.
8. Synchronization and Timing 8. Synchronization and Timing
It is important to consider the timing aspects, since out-of-order It is important to consider the timing aspects, since out-of-order
packets happen and have to be handled as well, as described in packets happen and have to be handled as well, as described in
[I-D.ietf-ippm-rfc8321bis]. [RFC9341].
However, in a multisource situation, an additional issue has to be However, in a multisource situation, an additional issue has to be
considered. With multipoint path, the egress nodes will receive considered. With multipoint path, the egress nodes will receive
alternate marked packets in random order from different ingress alternate marked packets in random order from different ingress
nodes, and this must not affect the measurement. nodes, and this must not affect the measurement.
So, if we analyze a multipoint-to-multipoint path with more than one So, if we analyze a multipoint-to-multipoint path with more than one
marking node, it is important to recognize the reference measurement marking node, it is important to recognize the reference measurement
interval. In general, the measurement interval for describing the interval. In general, the measurement interval for describing the
results is the interval of the marking node that is more aligned with results is the interval of the marking node that is more aligned with
skipping to change at page 18, line 7 skipping to change at line 782
time -> start stop time -> start stop
T(R1) |-------------| T(R1) |-------------|
T(R2) |-------------| T(R2) |-------------|
T(R3) |------------| T(R3) |------------|
Figure 2: Measurement Interval Figure 2: Measurement Interval
In Figure 2, it is assumed that the node with the earliest clock (R1) In Figure 2, it is assumed that the node with the earliest clock (R1)
identifies the right starting and ending times of the measurement, identifies the right starting and ending times of the measurement,
but it is just an assumption, and other possibilities could occur. but it is just an assumption and other possibilities could occur. So
So, in this case, T(R1) is the measurement interval, and its in this case, T(R1) is the measurement interval, and its recognition
recognition is essential in order to make comparisons with other is essential in order to make comparisons with other active/passive/
active/passive/hybrid Packet Loss metrics. hybrid packet-loss metrics.
Regarding the timing constraints of the methodology, Regarding the timing constraints of the methodology, [RFC9341]
[I-D.ietf-ippm-rfc8321bis] already describes two contributions that already describes two contributions that are taken into account: the
are taken into account: the clock error between network devices and clock error between network devices and the network delay between the
the network delay between the measurement points. measurement points.
When we expand to a multipoint environment, we have to consider that When we expand to a multipoint environment, we have to consider that
there are more marking nodes that mark the traffic based on there are more marking nodes that mark the traffic based on
synchronized clock time. But, due to different synchronization synchronized clock time. But, due to different synchronization
issues that may happen, the marking batches can be of different issues that may happen, the marking batches can be of different
lengths and with different offsets when they get mixed in a lengths and with different offsets when they get mixed in a
multipoint flow. According to [I-D.ietf-ippm-rfc8321bis], the multipoint flow. According to [RFC9341], the maximum clock skew
maximum clock skew between the network devices is A. Therefore, the between the network devices is A. Therefore, the additional gap that
additional gap that results between the multiple sources can be results between the multiple sources can be incorporated into A.
incorporated into A.
...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB... ...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB...
|<======================================>| |<======================================>|
| L | | L |
...=========>|<==================><==================>|<==========... ...=========>|<==================><==================>|<==========...
| L/2 L/2 | | L/2 L/2 |
|<====>| |<====>| |<====>| |<====>|
d | | d d | | d
|<========================>| |<========================>|
available counting interval available counting interval
Figure 3: Timing Aspects Figure 3: Timing Aspects
Moreover, it is assumed that the multipoint path can be modeled with Moreover, it is assumed that the multipoint path can be modeled with
a normal distribution, otherwise it is necessary to reformulate based a normal distribution; otherwise, it is necessary to reformulate
on the type of distribution. Under this assumption, the definition based on the type of distribution. Under this assumption, the
of the guard band d is still applicable as defined in definition of the guard band d is still applicable as defined in
[I-D.ietf-ippm-rfc8321bis] and is given by: [RFC9341] and is given by:
d = A + D_avg + 3*D_stddev, d = A + D_avg + 3*D_stddev,
where A is the clock accuracy, D_avg is the average value of the where A is the clock accuracy, D_avg is the average value of the
network delay, and D_stddev is the standard deviation of the delay. network delay, and D_stddev is the standard deviation of the delay.
As shown in Figure 3 and according to [I-D.ietf-ippm-rfc8321bis], the As shown in Figure 3 and according to [RFC9341], the condition that
condition that must be satisfied to enable the method to function must be satisfied to enable the method to function properly is that
properly is that the available counting interval must be > 0, and the available counting interval must be > 0, and that means:
that means:
L - 2d > 0. L - 2d > 0.
This formula needs to be verified for each measurement point on the This formula needs to be verified for each measurement point on the
multipoint path. multipoint path.
Note that the timing considerations are valid for both packet loss Note that the timing considerations are valid for both packet loss
and delay measurements. and delay measurements.
9. Recommendations for Deployment 9. Recommendations for Deployment
The methodology described in the previous sections can be applied to The methodology described in the previous sections can be applied to
various performance measurement problems, as also explained in various performance measurement problems, as also explained in
[I-D.ietf-ippm-rfc8321bis]. [RFC8889] reports experimental examples [RFC9341]. [RFC8889] reports experimental examples and
and [IEEE-Network-PNPM] also includes some information about the [IEEE-NETWORK-PNPM] also includes some information about the
deployment experience. deployment experience.
Different deployments are possible using one flag bit, two flag bits Different deployments are possible using one flag bit, two flag bits,
or hashing selection: or the hashing selection:
One flag: packet loss measurement MUST be done as described in One flag: packet-loss measurement MUST be done as described in
Section 6 by applying the network clustering partition described Section 6 by applying the network clustering partition described
in Section 5. Delay measurement MUST be done according to the in Section 5. Delay measurement MUST be done according to the
Mean delay calculation representative of the multipoint path, as mean delay calculation representative of the multipoint path, as
described in Section 7.1.1. Single-marking method based on the described in Section 7.1.1. A Single-Marking Method based on the
first/last packet of the interval cannot be applied, as mentioned first/last packet of the interval cannot be applied, as mentioned
in Section 7.2.1. in Section 7.2.1.
Two flags: packet loss measurement MUST be done as described in Two flags: packet-loss measurement MUST be done as described in
Section 6 by applying the network clustering partition described Section 6 by applying the network clustering partition described
in Section 5. Delay measurement SHOULD be done on a single packet in Section 5. Delay measurement SHOULD be done on a single-packet
basis according to double-marking method Section 7.2.1. In this basis according to the Double-Marking Method (Section 7.2.1). In
case the Mean delay calculation (Section 7.1.1) MAY also be used this case, the mean delay calculation (Section 7.1.1) MAY also be
as a representative value of a multipoint path. The choice used as a representative value of a multipoint path. The choice
depends on the kind of information that is needed, as further depends on the kind of information that is needed, as further
detailed below. detailed below.
One flag with hash-based selection: packet loss measurement MUST One flag with hash-based selection: packet-loss measurement MUST be
be done as described in Section 6 by applying the network done as described in Section 6 by applying the network clustering
clustering partition described in Section 5. Hash-based selection partition described in Section 5. Hash-based selection
methodologies, introduced in Section 7.2.2, MUST be used for delay methodologies, introduced in Section 7.2.2, MUST be used for delay
measurement. measurement.
Similarly to [I-D.ietf-ippm-rfc8321bis], there are some operational Similarly to [RFC9341], there are some operational guidelines to
guidelines to consider for the purpose of deciding to follow the consider when deciding which recommendation to use (i.e., one flag or
recommendations above and use one or two flags or one flag with hash- two flags or one flag with hash-based selection.
based selection.
The Multipoint Alternate-Marking method utilizes specific flags in * The Multipoint Alternate-Marking Method utilizes specific flags in
the packet header, so an important factor is the number of flags the packet header, so an important factor is the number of flags
available for the implementation. Indeed, if there is only one available for the implementation. Indeed, if there is only one
flag available there is no other way, while if two flags are flag available, there is no other way, while if two flags are
available the option with two flags can be considered in available, the option with two flags can be considered in
comparison with the option of one flag with hash-based selection. comparison with the option of one flag with hash-based selection.
The duration of the Alternate-Marking period affects the frequency * The duration of the Alternate-Marking period affects the frequency
of the measurement and this is a parameter that can be decided on of the measurement, and this is a parameter that can be decided on
the basis of the required temporal sampling. But it cannot be the basis of the required temporal sampling. But it cannot be
freely chosen, as explained in Section 8. freely chosen, as explained in Section 8.
The Multipoint Alternate-Marking methodologies enable packet loss, * The Multipoint Alternate-Marking methodologies enable packet loss,
delay and delay variation calculation, but in accordance with the delay, and delay variation calculation, but in accordance with the
method used (e.g. single-marking or double-marking or hashing method used (e.g., Single Marking, Double Marking, or hashing
selection), there is a different kind of information that can be selection), there is a different kind of information that can be
derived. For example, to get measurements on a multipoint-paths derived. For example, to get measurements on a multipoint-paths
basis, one flag can be used. To get measurements on a single- basis, one flag can be used. To get measurements on a single-
packet basis, two flags are preferred. For this reason, the type packet basis, two flags are preferred. For this reason, the type
of data needed in the specific scenario is an additional element of data needed in the specific scenario is an additional element
to take into account. to take into account.
The Multipoint Alternate-Marking methods imply different * The Multipoint Alternate-Marking Methods imply different
computational load depending on the method employed. Therefore, computational load depending on the method employed. Therefore,
the available computational resources on the measurement points the available computational resources on the measurement points
can also influence the choice. As an example, mean delay can also influence the choice. As an example, mean delay
calculation may require more processing and it may not be the best calculation may require more processing, and it may not be the
option to minimize the computational load. best option to minimize the computational load.
The experiment with Multipoint Alternate-Marking methodologies The experiment with Multipoint Alternate-Marking methodologies
confirmed the benefits of the Alternate-Marking methodology confirmed the benefits of the Alternate-Marking methodology [RFC9341]
([I-D.ietf-ippm-rfc8321bis]), as its extension to the general case of as its extension to the general case of multipoint-to-multipoint
multipoint-to-multipoint scenarios. scenarios.
The Multipoint Alternate-Marking Method MUST only be applied to The Multipoint Alternate-Marking Method MUST only be applied to
controlled domains, as per [I-D.ietf-ippm-rfc8321bis]. controlled domains, as per [RFC9341].
10. A Closed-Loop Performance-Management Approach 10. A Closed-Loop Performance-Management Approach
The Multipoint Alternate-Marking framework that is introduced in this The Multipoint Alternate-Marking framework that is introduced in this
document adds flexibility to Performance Management (PM), because it document adds flexibility to Performance Management (PM), because it
can reduce the order of magnitude of the packet counters. This can reduce the order of magnitude of the packet counters. This
allows an SDN orchestrator to supervise, control, and manage PM in allows an SDN orchestrator to supervise, control, and manage PM in
large networks. large networks.
The monitoring network can be considered as a whole or split into The monitoring network can be considered as a whole or split into
clusters that are the smallest subnetworks (group-to-group segments), clusters that are the smallest subnetworks (group-to-group segments),
maintaining the packet-loss property for each subnetwork. The maintaining the packet-loss property for each subnetwork. The
clusters can also be combined in new, connected subnetworks at clusters can also be combined in new, connected subnetworks at
different levels, depending on the detail we want to achieve. different levels, depending on the detail we want to achieve.
An SDN controller or a Network Management System (NMS) can calibrate An SDN controller or an NMS can calibrate performance measurements,
performance measurements, since they are aware of the network since they are aware of the network topology. They can start without
topology. They can start without examining in depth. In case of examining in depth. In case of necessity (packet loss is measured or
necessity (packet loss is measured, or the delay is too high), the the delay is too high), the filtering criteria could be immediately
filtering criteria could be immediately reconfigured in order to reconfigured in order to perform a partition of the network by using
perform a partition of the network by using clusters and/or different clusters and/or different combinations of clusters. In this way, the
combinations of clusters. In this way, the problem can be localized problem can be localized in a specific cluster or a single
in a specific cluster or a single combination of clusters, and a more combination of clusters, and a more detailed analysis can be
detailed analysis can be performed step by step by successive performed step by step by successive approximation up to a point-to-
approximation up to a point-to-point flow detailed analysis. This is point flow detailed analysis. This is the so-called "closed loop".
the so-called "closed loop".
This approach can be called "network zooming" and can be performed in This approach can be called "network zooming" and can be performed in
two different ways: two different ways:
1) change the traffic filter and select more detailed flows; 1. change the traffic filter and select more detailed flows;
2) activate new measurement points by defining more specified 2. activate new measurement points by defining more specified
clusters. clusters.
The network-zooming approach implies that some filters or rules are The network-zooming approach implies that some filters or rules are
changed and that therefore there is a transient time to wait once the changed; therefore, there is a transient time to wait once the new
new network configuration takes effect. This time can be determined network configuration takes effect. This time can be determined by
by the Network Orchestrator/Controller, based on the network the network orchestrator/controller, based on the network conditions.
conditions.
For example, if the network zooming identifies the performance For example, if the network zooming identifies the performance
problem for the traffic coming from a specific source, we need to problem for the traffic coming from a specific source, we need to
recognize the marked signal from this specific source node and its recognize the marked signal from this specific source node and its
relative path. For this purpose, we can activate all the available relative path. For this purpose, we can activate all the available
measurement points and better specify the flow filter criteria (i.e., measurement points and better specify the flow filter criteria (i.e.,
5-tuple). As an alternative, it can be enough to select packets from 5-tuple). As an alternative, it can be enough to select packets from
the specific source for delay measurements; in this case, it is the specific source for delay measurements; in this case, it is
possible to apply the hashing technique, as mentioned in the previous possible to apply the hashing technique, as mentioned in the previous
sections. sections.
[I-D.song-opsawg-ifit-framework] defines an architecture where the [OPSAWG-IFIT-FRAMEWORK] defines an architecture where the centralized
centralized Data Collector and Network Management can apply the data collector and network management can apply the intelligent and
intelligent and flexible Alternate-Marking algorithm as previously flexible Alternate-Marking algorithm as previously described.
described.
As for [I-D.ietf-ippm-rfc8321bis], it is possible to classify the As for [RFC9341], it is possible to classify the traffic and mark a
traffic and mark a portion of the total traffic. For each period, portion of the total traffic. For each period, the packet rate and
the packet rate and bandwidth are calculated from the number of bandwidth are calculated from the number of packets. In this way,
packets. In this way, the network orchestrator becomes aware if the the network orchestrator becomes aware if the traffic rate surpasses
traffic rate surpasses limits. In addition, more precision can be limits. In addition, more precision can be obtained by reducing the
obtained by reducing the marking period; indeed, some implementations marking period; indeed, some implementations use a marking period of
use a marking period of 1 sec or less. 1 sec or less.
In addition, an SDN controller could also collect the measurement In addition, an SDN controller could also collect the measurement
history. history.
It is important to mention that the Multipoint Alternate-Marking It is important to mention that the Multipoint Alternate-Marking
framework also helps Traffic Visualization. Indeed, this methodology framework also helps Traffic Visualization. Indeed, this methodology
is very useful for identifying which path or cluster is crossed by is very useful for identifying which path or cluster is crossed by
the flow. the flow.
11. Security Considerations 11. Security Considerations
This document specifies a method of performing measurements that does This document specifies a method of performing measurements that does
not directly affect Internet security or applications that run on the not directly affect Internet security or applications that run on the
Internet. However, implementation of this method must be mindful of Internet. However, implementation of this method must be mindful of
security and privacy concerns, as explained in security and privacy concerns, as explained in [RFC9341].
[I-D.ietf-ippm-rfc8321bis].
12. IANA Considerations 12. IANA Considerations
This document has no IANA actions. This document has no IANA actions.
13. Contributors 13. References
Greg Mirsky Ericsson Email: gregimirsky@gmail.com
Tal Mizrahi Huawei Technologies Email: tal.mizrahi.phd@gmail.com
Xiao Min ZTE Corp. Email: xiao.min2@zte.com.cn
14. Acknowledgements
The authors would like to thank Martin Duke and Tommy Pauly for their
assistance and their detailed and precious reviews.
15. References
15.1. Normative References
[I-D.ietf-ippm-rfc8321bis] 13.1. Normative References
Fioccola, G., Cociglio, M., Mirsky, G., Mizrahi, T., and
T. Zhou, "Alternate-Marking Method", Work in Progress,
Internet-Draft, draft-ietf-ippm-rfc8321bis-03, 25 July
2022, <https://www.ietf.org/archive/id/draft-ietf-ippm-
rfc8321bis-03.txt>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. <https://www.rfc-editor.org/info/rfc2119>.
[RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. [RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.
Raspall, "Sampling and Filtering Techniques for IP Packet Raspall, "Sampling and Filtering Techniques for IP Packet
Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009, Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009,
<https://www.rfc-editor.org/info/rfc5475>. <https://www.rfc-editor.org/info/rfc5475>.
[RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance [RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance
Metrics (IPPM): Spatial and Multicast", RFC 5644, Metrics (IPPM): Spatial and Multicast", RFC 5644,
DOI 10.17487/RFC5644, October 2009, DOI 10.17487/RFC5644, October 2009,
<https://www.rfc-editor.org/info/rfc5644>. <https://www.rfc-editor.org/info/rfc5644>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>. May 2017, <https://www.rfc-editor.org/info/rfc8174>.
15.2. Informative References [RFC9341] Fioccola, G., Ed., Cociglio, M., Mirsky, G., Mizrahi, T.,
and T. Zhou, "Alternate-Marking Method", RFC 9341,
[I-D.ietf-ippm-route] DOI 10.17487/RFC9341, December 2022,
Alvarez-Hamelin, J. I., Morton, A., Fabini, J., Pignataro, <https://www.rfc-editor.org/info/rfc9341>.
C., and R. Geib, "Advanced Unidirectional Route Assessment
(AURA)", Work in Progress, Internet-Draft, draft-ietf-
ippm-route-10, 13 August 2020,
<https://www.ietf.org/archive/id/draft-ietf-ippm-route-
10.txt>.
[I-D.song-opsawg-ifit-framework] 13.2. Informative References
Song, H., Qin, F., Chen, H., Jin, J., and J. Shin, "A
Framework for In-situ Flow Information Telemetry", Work in
Progress, Internet-Draft, draft-song-opsawg-ifit-
framework-18, 6 September 2022,
<https://www.ietf.org/archive/id/draft-song-opsawg-ifit-
framework-18.txt>.
[IEEE-ACM-ToN-MPNPM] [IEEE-ACM-TON-MPNPM]
IEEE/ACM TRANSACTION ON NETWORKING, "Multipoint Passive Cociglio, M., Fioccola, G., Marchetto, G., Sapio, A., and
Monitoring in Packet Networks", R. Sisto, "Multipoint Passive Monitoring in Packet
DOI 10.1109/TNET.2019.2950157, 2019, Networks", IEEE/ACM Transactions on Networking, Vol. 27,
Issue 6, DOI 10.1109/TNET.2019.2950157, December 2019,
<https://doi.org/10.1109/TNET.2019.2950157>. <https://doi.org/10.1109/TNET.2019.2950157>.
[IEEE-Network-PNPM] [IEEE-NETWORK-PNPM]
IEEE Network, "AM-PM: Efficient Network Telemetry using Mizrahi, T., Navon, G., Fioccola, G., Cociglio, M., Chen,
Alternate Marking", DOI 10.1109/MNET.2019.1800152, 2019, M., and G. Mirsky, "AM-PM: Efficient Network Telemetry
using Alternate Marking", IEEE Network, Vol. 33, Issue 4,
DOI 10.1109/MNET.2019.1800152, July 2019,
<https://doi.org/10.1109/MNET.2019.1800152>. <https://doi.org/10.1109/MNET.2019.1800152>.
[OPSAWG-IFIT-FRAMEWORK]
Song, H., Qin, F., Chen, H., Jin, J., and J. Shin, "A
Framework for In-situ Flow Information Telemetry", Work in
Progress, Internet-Draft, draft-song-opsawg-ifit-
framework-19, 24 October 2022,
<https://datatracker.ietf.org/doc/html/draft-song-opsawg-
ifit-framework-19>.
[RFC5474] Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A., [RFC5474] Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A.,
Grossglauser, M., and J. Rexford, "A Framework for Packet Grossglauser, M., and J. Rexford, "A Framework for Packet
Selection and Reporting", RFC 5474, DOI 10.17487/RFC5474, Selection and Reporting", RFC 5474, DOI 10.17487/RFC5474,
March 2009, <https://www.rfc-editor.org/info/rfc5474>. March 2009, <https://www.rfc-editor.org/info/rfc5474>.
[RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken,
"Specification of the IP Flow Information Export (IPFIX) "Specification of the IP Flow Information Export (IPFIX)
Protocol for the Exchange of Flow Information", STD 77, Protocol for the Exchange of Flow Information", STD 77,
RFC 7011, DOI 10.17487/RFC7011, September 2013, RFC 7011, DOI 10.17487/RFC7011, September 2013,
<https://www.rfc-editor.org/info/rfc7011>. <https://www.rfc-editor.org/info/rfc7011>.
[RFC8889] Fioccola, G., Ed., Cociglio, M., Sapio, A., and R. Sisto, [RFC8889] Fioccola, G., Ed., Cociglio, M., Sapio, A., and R. Sisto,
"Multipoint Alternate-Marking Method for Passive and "Multipoint Alternate-Marking Method for Passive and
Hybrid Performance Monitoring", RFC 8889, Hybrid Performance Monitoring", RFC 8889,
DOI 10.17487/RFC8889, August 2020, DOI 10.17487/RFC8889, August 2020,
<https://www.rfc-editor.org/info/rfc8889>. <https://www.rfc-editor.org/info/rfc8889>.
[RFC9198] Alvarez-Hamelin, J., Morton, A., Fabini, J., Pignataro,
C., and R. Geib, "Advanced Unidirectional Route Assessment
(AURA)", RFC 9198, DOI 10.17487/RFC9198, May 2022,
<https://www.rfc-editor.org/info/rfc9198>.
Appendix A. Example of Monitoring Network and Clusters Partition Appendix A. Example of Monitoring Network and Clusters Partition
Figure 4 shows a simple example of a monitoring network graph: Figure 4 shows a simple example of a monitoring network graph:
+------+ +------+
<> R6 <>--- <> R6 <>---
/ +------+ / +------+
+------+ +------+ / +------+ +------+ /
<> R2 <>---<> R4 <> <> R2 <>---<> R4 <>
/ +------+ \ +------+ \ / +------+ \ +------+ \
skipping to change at page 25, line 35 skipping to change at line 1098
+------+ +------+
Figure 4: Monitoring Network Graph Figure 4: Monitoring Network Graph
In the monitoring network graph example, it is possible to identify In the monitoring network graph example, it is possible to identify
the clusters partition by applying this two-step algorithm described the clusters partition by applying this two-step algorithm described
in Section 5.1. in Section 5.1.
The first step identifies the following groups: The first step identifies the following groups:
1. Group 1: (R1-R2), (R1-R3), (R1-R10) Group 1: (R1-R2), (R1-R3), (R1-R10)
2. Group 2: (R2-R4), (R2-R5) Group 2: (R2-R4), (R2-R5)
3. Group 3: (R3-R5), (R3-R9) Group 3: (R3-R5), (R3-R9)
4. Group 4: (R4-R6), (R4-R7) Group 4: (R4-R6), (R4-R7)
5. Group 5: (R5-R8) Group 5: (R5-R8)
And then, the second step builds the clusters partition (in Then, the second step builds the clusters partition (in particular,
particular, we can underline that Groups 2 and 3 connect together, we can underline that Groups 2 and 3 connect together, since R5 is in
since R5 is in common): common):
1. Cluster 1: (R1-R2), (R1-R3), (R1-R10) Cluster 1: (R1-R2), (R1-R3), (R1-R10)
2. Cluster 2: (R2-R4), (R2-R5), (R3-R5), (R3-R9) Cluster 2: (R2-R4), (R2-R5), (R3-R5), (R3-R9)
3. Cluster 3: (R4-R6), (R4-R7)
4. Cluster 4: (R5-R8) Cluster 3: (R4-R6), (R4-R7)
The flow direction here considered is from left to right. For the Cluster 4: (R5-R8)
The flow direction considered here is from left to right. For the
opposite direction, the same reasoning can be applied, and in this opposite direction, the same reasoning can be applied, and in this
example, you get the same clusters partition. example, you get the same clusters partition.
In the end, the following 4 clusters are obtained: In the end, the following 4 clusters are obtained:
Cluster 1 Cluster 1
+------+ +------+
<> R2 <>--- <> R2 <>---
/ +------+ / +------+
/ /
skipping to change at page 27, line 31 skipping to change at line 1188
<> R8 <>--- <> R8 <>---
+------+ +------+
Figure 5: Clusters Example Figure 5: Clusters Example
There are clusters with more than two nodes as well as two-node There are clusters with more than two nodes as well as two-node
clusters. In the two-node clusters, the loss is on the link (Cluster clusters. In the two-node clusters, the loss is on the link (Cluster
4). In more-than-two-node clusters, the loss is on the cluster, but 4). In more-than-two-node clusters, the loss is on the cluster, but
we cannot know in which link (Cluster 1, 2, or 3). we cannot know in which link (Cluster 1, 2, or 3).
Appendix B. Changes Log Acknowledgements
Changes from RFC 8889 in draft-fioccola-rfc8889bis-00 include:
* Minor editorial changes
* Removed section on "Examples of application"
Changes in draft-fioccola-rfc8889bis-01 include:
* Considerations on BUM traffic
* Reference to RFC8321bis for the fragmentation part
* Revised section on "Delay Measurements on a Single-Packet Basis"
* Revised section on "Timing Aspects"
Changes in draft-fioccola-rfc8889bis-02 include:
* Clarified the formula in the section on "Timing Aspects" to be
aligned with RFC 8321
* Considerations on two-way delay measurements in both sections 8.1
and 8.2 on delay measurements
* Clarified in section 4.1 on "Monitoring Network" that the
description is done for one direction but it can easily be
extended to all direction
* New section on "Results of the Multipoint Alternate Marking
Experiment"
Changes in draft-fioccola-rfc8889bis-03 include:
* Moved and renamed section on "Timing Aspects" as "Synchronization
and Timing"
* Renamed old section on "Multipoint Packet Loss" as "Network Packet
Loss"
* New section on "Multipoint Packet Loss Measurement"
* Renamed section on "Multipoint Performance Measurement" as
"Extension of the Method to Multipoint Flows"
Changes in draft-fioccola-rfc8889bis-04/draft-ietf-ippm-rfc8889bis-00
include:
* Revised section 5.1 on "Algorithm for Clusters Partition"
Changes in draft-ietf-ippm-rfc8889bis-01 include:
* New section on "Summary of Changes from RFC 8889"
Changes in draft-ietf-ippm-rfc8889bis-02 include:
* Revised sections on "Single- and Double-Marking Measurement",
"Hashing Selection Method" and "Synchronization and Timing"
* Revised references The authors would like to thank Martin Duke and Tommy Pauly for their
assistance and their detailed and valuable reviews.
Changes in draft-ietf-ippm-rfc8889bis-03 include: Contributors
* Comments addressed from Last Call review Greg Mirsky
Ericsson
Email: gregimirsky@gmail.com
* Renamed section 9 as "Recommendations for Deployment" Tal Mizrahi
Changes in draft-ietf-ippm-rfc8889bis-04 include: Huawei Technologies
Email: tal.mizrahi.phd@gmail.com
* Comments addressed from Last Call review Xiao Min
ZTE Corp.
Email: xiao.min2@zte.com.cn
Authors' Addresses Authors' Addresses
Giuseppe Fioccola (editor) Giuseppe Fioccola (editor)
Huawei Technologies Huawei Technologies
Riesstrasse, 25 Riesstrasse, 25
80992 Munich 80992 Munich
Germany Germany
Email: giuseppe.fioccola@huawei.com Email: giuseppe.fioccola@huawei.com
 End of changes. 139 change blocks. 
426 lines changed or deleted 345 lines changed or added

This html diff was produced by rfcdiff 1.48.