rfc9342xml2.original.xml   rfc9342.xml 
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
<!-- A set of on-line citation libraries are maintained on the xml2rfc web site.
The next line defines an entity named RFC2629, which contains the necessary
XML
for the reference element, and is used much later in the file. This XML co
ntains an
anchor (also RFC2629) which can be used to cross-reference this item in the
text.
You can also use local file names instead of a URI. The environment variab
le
XML_LIBRARY provides a search path of directories to look at to locate a
relative path name for the file. There has to be one entity for each item t
o be
referenced. -->
<!ENTITY RFC2234 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC
.2234.xml">
<!ENTITY RFC2629 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC
.2629.xml">
<!ENTITY RFC4234 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC
.4234.xml">
<!ENTITY RFC5575 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC
.5575.xml">
<!-- There is also a library of current Internet Draft citations. It isn't a go
od idea to
actually use one for the template because it might have disappeared when yo
u come to test
this template. This is the form of the entity definition
&lt;!ENTITY I-D.mrose-writing-rfcs SYSTEM
"http://xml.resource.org/public/rfc/bibxml3/reference.I-D.mrose-writing-rfc
s.xml">
corresponding to a draft filename draft-mrose-writing-rfcs-nn.txt. The cita
tion will be
to the most recent draft in the sequence, and is updated roughly hourly on
the web site.
For working group drafts, the same principle applies: file name starts draf
t-ietf-wgname-..
and entity file is reference.I-D.ietf-wgname-... The corresponding entity
name is
I-D.ietf-wgname-... (I-D.mrose-writing-rfcs for the other example). Of cou
rse this doesn't
change when the draft version changes.
-->
<!-- Fudge for XMLmind which doesn't have this built in -->
<!ENTITY nbsp "&#160;">
]>
<!-- Extra statement used by XSLT processors to control the output style. -->
<?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?>
<!-- Processing Instructions can be placed here but if you are editing
with XMLmind (and maybe other XML editors) they are better placed
after the rfc element start tag as shown below. -->
<!-- Information about the document.
category values: std, bcp, info, exp, and historic
For Internet-Drafts, specify attribute "ipr".
(ipr values are: full3667, noModification3667, noDerivatives3667),
Also for Internet-Drafts, can specify values for
attributes "docName" and, if relevant, "iprExtract". Note
that the value for iprExtract is the anchor attribute
value of a section (such as a MIB specification) that can be
extracted for separate publication, and is only
useful whenhe value of "ipr" is not "full3667". -->
<!-- TODO: verify which attributes are specified only
by the RFC editor. It appears that attributes
"number", "obsoletes", "updates", and "seriesNo"
are specified by the RFC editor (and not by
the document author). -->
<rfc
category="std"
ipr="trust200902"
docName="draft-ietf-ippm-rfc8889bis-04"
obsoletes="8889" >
<!-- Processing Instructions- PIs (for a complete list and description,
see file http://xml.resource.org/authoring/README.html and below... --
>
<!-- Some of the more generally applicable PIs that most I-Ds might want to
use -->
<!-- Try to enforce the ID-nits conventions and DTD validity -->
<?rfc strict="yes" ?>
<!-- Items used when reviewing the document -->
<?rfc comments="no" ?> <!-- Controls display of <cref> elements -->
<?rfc inline="no" ?> <!-- When no, put comments at end in comments sectio
n,
otherwise, put inline -->
<?rfc editing="no" ?> <!-- When yes, insert editing marks: editing marks c
onsist of a
string such as <29> printed in the blank line a
t the
beginning of each paragraph of text. -->
<!-- Create Table of Contents (ToC) and set some options for it.
Note the ToC may be omitted for very short documents,but idnits insists
on a ToC
if the document has more than 15 pages. -->
<?rfc toc="yes"?>
<?rfc tocompact="yes"?> <!-- If "yes" eliminates blank lines before main sect
ion entries. -->
<?rfc tocdepth="3"?> <!-- Sets the number of levels of sections/subsection
s... in ToC -->
<!-- Choose the options for the references. <!DOCTYPE rfc [
Some like symbolic tags in the references (and citations) and others pr <!ENTITY nbsp "&#160;">
efer <!ENTITY zwsp "&#8203;">
numbers. The RFC Editor always uses symbolic tags. <!ENTITY nbhy "&#8209;">
The tags used are the anchor attributes of the references. --> <!ENTITY wj "&#8288;">
<?rfc symrefs="yes"?> ]>
<?rfc sortrefs="yes" ?> <!-- If "yes", causes the references to be sorted in
order of tags.
This doesn't have any effect unless symrefs is
"yes" also. -->
<!-- These two save paper: Just setting compact to "yes" makes savings by no <rfc xmlns:xi="http://www.w3.org/2001/XInclude" submissionType="IETF" category="
t starting each std" consensus="true" docName="draft-ietf-ippm-rfc8889bis-04" number="9342" ipr=
main section on a new page but does not omit the blank lines between li "trust200902" obsoletes="8889" updates="" xml:lang="en" tocInclude="true" tocDep
st items. th="3" symRefs="true" sortRefs="true" version="3">
If subcompact is also "yes" the blank lines between list items are also <front>
omitted. -->
<?rfc compact="yes" ?>
<?rfc subcompact="no" ?>
<!-- end of list of popular I-D processing instructions -->
<!-- ***** FRONT MATTER ***** --> <title abbrev="Clustered Alternate-Marking Method">Clustered Alternate-Marki
<front> ng Method</title>
<!-- The abbreviated title is used in the page header - it is only necessary <seriesInfo name="RFC" value="9342"/>
if the
full title is longer than 42 characters -->
<title abbrev="ClustAltMark">Clustered Alternate-Marking Method</title>
<!-- add 'role="editor"' below for the editors if appropriate -->
<author role="editor" fullname="Giuseppe Fioccola" initials="G." surname= "Fioccola"> <author role="editor" fullname="Giuseppe Fioccola" initials="G." surname= "Fioccola">
<organization>Huawei Technologies</organization> <organization>Huawei Technologies</organization>
<address> <address>
<postal> <postal>
<street>Riesstrasse, 25</street> <street>Riesstrasse, 25</street>
<city>Munich</city> <city>Munich</city>
<code>80992</code> <code>80992</code>
<country>Germany</country> <country>Germany</country>
</postal> </postal>
<email>giuseppe.fioccola@huawei.com</email> <email>giuseppe.fioccola@huawei.com</email>
</address> </address>
</author> </author>
<author fullname="Mauro Cociglio" initials="M." surname="Cociglio"> <author fullname="Mauro Cociglio" initials="M." surname="Cociglio">
<organization>Telecom Italia</organization> <organization>Telecom Italia</organization>
<address> <address>
<postal> <postal>
<street></street> <street/>
<city></city> <city/>
<code></code> <code/>
<country></country> <country/>
</postal> </postal>
<email>mauro.cociglio@outlook.com</email> <email>mauro.cociglio@outlook.com</email>
</address> </address>
</author> </author>
<author fullname="Amedeo Sapio" initials="A." surname="Sapio">
<author fullname="Amedeo Sapio" initials="A." surname="Sapio"> <organization>Intel Corporation</organization>
<organization>Intel Corporation</organization>
<address> <address>
<postal> <postal>
<street>4750 Patrick Henry Dr.</street> <street>4750 Patrick Henry Dr.</street>
<city>Santa Clara</city> <city>Santa Clara</city>
<region>CA</region> <region>CA</region>
<code>95054</code> <code>95054</code>
<country>USA</country> <country>United States of America</country>
</postal> </postal>
<email>amedeo.sapio@intel.com</email> <email>amedeo.sapio@intel.com</email>
</address> </address>
</author> </author>
<author fullname="Riccardo Sisto" initials="R." surname="Sisto">
<author fullname="Riccardo Sisto" initials="R." surname="Sisto"> <organization>Politecnico di Torino</organization>
<organization>Politecnico di Torino</organization>
<address> <address>
<postal> <postal>
<street>Corso Duca degli Abruzzi, 24</street> <street>Corso Duca degli Abruzzi, 24</street>
<city>Torino</city> <city>Torino</city>
<code>10129</code> <code>10129</code>
<country>Italy</country> <country>Italy</country>
</postal> </postal>
<email>riccardo.sisto@polito.it</email> <email>riccardo.sisto@polito.it</email>
</address> </address>
</author> </author>
<author fullname="Tianran Zhou" initials="T." surname="Zhou">
<author fullname="Tianran Zhou" initials="T." surname="Zhou">
<organization>Huawei Technologies</organization> <organization>Huawei Technologies</organization>
<address> <address>
<postal> <postal>
<street>156 Beiqing Rd.</street> <street>156 Beiqing Rd.</street>
<city>Beijing</city> <city>Beijing</city>
<code>100095</code> <code>100095</code>
<region/> <region/>
<country>China</country> <country>China</country>
</postal> </postal>
<email>zhoutianran@huawei.com</email> <email>zhoutianran@huawei.com</email>
</address> </address>
</author> </author>
<date year="2022" month="December" />
<date year="2022"/> <!-- month="March" is no longer necessary <area>tsv</area>
note also, day="30" is optional --> <workgroup>ippm</workgroup>
<!-- WARNING: If the month and year are the current ones, xml2rfc will fill <keyword>Multipoint</keyword>
in the day for <keyword>Cluster</keyword>
you. If only the year is specified, xml2rfc will fill in the current da <keyword>Performance</keyword>
y and month <keyword>Measurement</keyword>
irrespective of the day. This silliness should be fixed in v1.31. --> <keyword>Monitoring</keyword>
<keyword>Passive</keyword>
<!-- Meta-data Declarations --> <keyword>Hybrid</keyword>
<keyword>Loss</keyword>
<!-- Notice the use of &amp; as an escape for & which would otherwise <keyword>Delay</keyword>
start an entity declaration, whereas we want a literal &. --> <keyword>Delay Variation</keyword>
<keyword>Hashing</keyword>
<area></area> <keyword>Closed-Loop</keyword>
<!-- WG name at the upperleft corner of the doc,
IETF fine for individual submissions. You can also
omit this element in which case in defaults to "Network Working Group"
-
a hangover from the ancient history of the IETF! -->
<workgroup></workgroup>
<!-- The DTD allows multiple area and workgroup elements but only the first
one has any
effect on output. -->
<!-- You can add <keyword/> elements here. They will be incorporated into H
TML output
files in a meta tag but they have no effect on text or nroff output. --
>
<abstract> <abstract>
<t>This document generalizes and expands Alternate-Marking methodology to measure <t>This document generalizes and expands the Alternate-Marking methodology to measure
any kind of unicast flow whose packets can follow several different p aths in the any kind of unicast flow whose packets can follow several different p aths in the
network that can result in a multipoint-to-multipoint network. network; this can result in a multipoint-to-multipoint network.
The network clustering approach is presented and, for this reason , The network clustering approach is presented and, for this reason ,
the technique here described is called "Clustered Alternate-Marki ng". the technique described here is called "Clustered Alternate Marki ng".
This document obsoletes RFC 8889.</t> This document obsoletes RFC 8889.</t>
</abstract> </abstract>
</front> </front>
<middle> <middle>
<section numbered="true" toc="default">
<section title="Introduction"> <name>Introduction</name>
<t>The Alternate-Marking Method, as described in <xref target="I-D.ietf-ip <t>The Alternate-Marking Method, as described in <xref target="RFC9341" fo
pm-rfc8321bis" format="default"/>, rmat="default"/>,
is applicable to a point-to-point path. The extension proposed in this document applies to the most general case is applicable to a point-to-point path. The extension proposed in this document applies to the most general case
of a multipoint-to-multipoint path and enables flexible and adaptive pe rformance measurements in a managed network.</t> of a multipoint-to-multipoint path and enables flexible and adaptive perfo rmance measurements in a managed network.</t>
<t>The Alternate-Marking methodology consists in splitting the packet flow into marking blocks and <t>The Alternate-Marking methodology consists of splitting the packet flow into marking blocks, and
the monitoring parameters are the packet counters and the timestamps fo r each marking period. the monitoring parameters are the packet counters and the timestamps fo r each marking period.
In some applications of the Alternate-Marking method, a lot of flows an In some applications of the Alternate-Marking Method, a lot of flows an
d nodes are to be monitored. d nodes are to be monitored.
Multipoint Alternate-Marking aims to reduce these values and makes the Multipoint Alternate Marking aims to reduce these values and makes the
performance monitoring more performance monitoring more
flexible in case a detailed analysis is not needed. flexible in case a detailed analysis is not needed.
For instance, by considering n measurement points and m monitored flows, t he order of magnitude For instance, by considering n measurement points and m monitored flows, t he order of magnitude
of the packet counters for each time interval is n*m*2 (1 per color). of the packet counters for each time interval is n*m*2 (1 per color).
The number of measurement points and monitored flows may vary and depen The number of measurement points and monitored flows may vary and depends
ds on the portion of the network on the portion of the network
we are monitoring (core network, metro network, access network) and the gr we are monitoring (core network, metro network, access network, etc.) and
anularity the granularity
(for each service, each customer). So if both n and m are high values, the (for each service, each customer, etc.). So if both n and m are high value
packet counters s, the packet counters
increase a lot, and Multipoint Alternate-Marking offers a tool to control increase a lot, and Multipoint Alternate Marking offers a tool to control
these parameters.</t> these parameters.</t>
<t>The approach presented in this document is applied only to unicast flow s and not to multicast. <t>The approach presented in this document is applied only to unicast flow s and not to multicast.
Broadcast, Unknown Unicast, and Multicast (BUM) traffic is not considered here, Broadcast, Unknown Unicast, and Multicast (BUM) traffic is not considered here,
because traffic replication is not covered by the Multipoint Alternate- Marking method. because traffic replication is not covered by the Multipoint Alternate- Marking Method.
Furthermore, it can be applicable to anycast flows, and Equal-Cost Mult ipath (ECMP) Furthermore, it can be applicable to anycast flows, and Equal-Cost Mult ipath (ECMP)
paths can also be easily monitored with this technique.</t> paths can also be easily monitored with this technique.</t>
<t><xref target="RFC9341" format="default"/> applies to point-to-point uni
cast flows and BUM traffic.
For BUM traffic, the basic method of <xref target="RFC9341" format="def
ault"/> can be easily
applied link by link; therefore, the multicast flow tree distribution c
an be split into separate unicast point-to-point links.</t>
<t><xref target="I-D.ietf-ippm-rfc8321bis" format="default"/> applies to p <t>This document and its Clustered Alternate-Marking Method applies to mul
oint-to-point unicast flows and BUM traffic. tipoint-to-multipoint
For BUM traffic, the basic method of <xref target="I-D.ietf-ippm-rfc832 unicast flows, anycast, and ECMP flows. Therefore, the Alternate-Marking M
1bis" format="default"/> can easily ethod can be extended to any kind of
be applied link by link and therefore split the multicast flow tree dis
tribution into separate unicast point-to-point links.
While, this document and its Clustered Alternate-Marking method apply t
o multipoint-to-multipoint
unicast flows, anycast, and ECMP flows.</t>
<t>Therefore, the Alternate-Marking method can be extended to any kind of
multipoint-to-multipoint paths, and the network-clustering approach presen ted in this document multipoint-to-multipoint paths, and the network-clustering approach presen ted in this document
is the formalization of how to implement this property and allow a flex ible and is the formalization of how to implement this property and allow a flex ible and
optimized performance measurement support for network management in eve ry situation.</t> optimized performance measurement support for network management in eve ry situation.</t>
<t>Without network clustering, it is possible to apply Alternate Marking o
<t>Without network clustering, it is possible to apply Alternate-Marking o nly for all
nly for all the network or per single flow. Instead, with network clustering, it is po
the network or per single flow. Instead, with network clustering, it is po ssible to partition
ssible to use the partition the network into clusters at different levels in order to provide the need
of the network into clusters at different levels in order to provide the n ed degree of detail.
eeded degree of detail.
In some circumstances, it is possible to monitor a multipoint network b y monitoring the network clusters, In some circumstances, it is possible to monitor a multipoint network b y monitoring the network clusters,
without examining in depth. In case of problems (packet loss is measur ed, or the delay is too high), without examining in depth. In case of problems (packet loss is measur ed or the delay is too high),
the filtering criteria could be enhanced in order to perform a detailed analysis by using a different the filtering criteria could be enhanced in order to perform a detailed analysis by using a different
combination of clusters up to a per-flow measurement as described in <x combination of clusters up to a per-flow measurement as described in <x
ref target="I-D.ietf-ippm-rfc8321bis" format="default"/>.</t> ref target="RFC9341" format="default"/>.</t>
<t>This approach fits very well with the Closed-Loop Network and Software-
<t>This approach fits very well with the Closed-Loop Network and Softwa Defined Network (SDN) paradigm,
re-Defined Network (SDN) paradigm,
where the SDN orchestrator and the SDN controllers are the brains of the n etwork and can manage flow control where the SDN orchestrator and the SDN controllers are the brains of the n etwork and can manage flow control
to the switches and routers and, in the same way, can calibrate the perfor mance measurements to the switches and routers and, in the same way, can calibrate the perfor mance measurements
depending on the desired accuracy. An SDN controller application can orche strate how accurately the network depending on the desired accuracy. An SDN controller application can orche strate how accurately the network
performance monitoring is set up by applying the Multipoint Alternate-Mark performance monitoring is set up by applying the Multipoint Alternate Mark
ing as described in this document.</t> ing as described in this document.</t>
<t>It is important to underline that, as an extension of <xref target="RFC
<t>It is important to underline that, as an extension of <xref target="I-D 9341" format="default"/>,
.ietf-ippm-rfc8321bis" format="default"/>,
this is a methodology document, so the mechanism that can be used to tr ansmit the counters and the timestamps is out of scope here.</t> this is a methodology document, so the mechanism that can be used to tr ansmit the counters and the timestamps is out of scope here.</t>
<t>This document assumes that the blocks are created according to a fixed
<t>This document assumes that the blocks are created according to a fix timer as per <xref target="RFC9341" format="default"/>.
ed timer as per <xref target="I-D.ietf-ippm-rfc8321bis" format="default"/>. Switching after a fixed number of packets is possible, but it is out of
Switching after a fixed number of packets is possible but it is out of scope here.</t>
scope here.</t> <t>Note that the fragmented packets' case can be managed with the Alternat
e-Marking methodology, and the same guidance provided in
<t>Note that the fragmented packets' case can be managed with the Alternat <xref target="RFC9341" sectionFormat="of" section="6" format="default"/
e-Marking methodology and the same guidance provided in > also applies in the case of Multipoint Alternate Marking.</t>
section 6 of <xref target="I-D.ietf-ippm-rfc8321bis" format="default"/> <section numbered="true" toc="default">
apply also in the case of Multipoint Alternate-Marking.</t> <name>Summary of Changes from RFC 8889</name>
<t>This document defines the Multipoint Alternate-Marking Method, addres
<section title="Summary of Changes from RFC 8889"> sing
<t>This document defines the Multipoint Alternate-Marking Method, addre
ssing
ambiguities and overtaking its experimental phase in the original speci fication ambiguities and overtaking its experimental phase in the original speci fication
<xref target="RFC8889"></xref>.</t> <xref target="RFC8889" format="default"/>.</t>
<t>The relevant changes are:</t>
<t>The relevant changes are:<list style="symbols"> <ul spacing="normal">
<li>Added the recommendations about the different deployments in case
<t>Added the recommendations about the different deployments in c one or two flag bits
ase one or two flag bits are available for marking (<xref target="finding" format="default
are available for marking (<xref target="finding"></xref>).</t> "/>).</li>
<li>Changed the structure to improve readability.</li>
<t>Changed the structure to improve the readability.</t> <li>Removed the wording about the experimentation of the method and co
nsiderations that no longer apply.</li>
<t>Removed the wording about the experimentation of the method an <li>Revised the description of detailed aspects of the methodology, e.
d considerations that no longer apply.</t> g., synchronization and timing.</li>
</ul>
<t>Revised the description of detailed aspects of the methodology, e.g. <t>It is important to note that all the changes are totally backward com
synchronization and timing.</t> patible with <xref target="RFC8889" format="default"/>,
</list></t> and no new additional technique has been introduced in this document co
mpared to <xref target="RFC8889" format="default"/>.</t>
<t>It is important to note that all the changes are totally backward co </section>
mpatible with <xref target="RFC8889"></xref> <section numbered="true" toc="default">
and no new additional technique has been introduced in this document co
mpared to <xref target="RFC8889"></xref>.</t>
</section>
<section title="Requirements Language">
<t>The key words &quot;MUST&quot;, &quot;MUST NOT&quot;, <name>Requirements Language</name>
&quot;REQUIRED&quot;, &quot;SHALL&quot;, &quot;SHALL NOT&quot;, <t>
&quot;SHOULD&quot;, &quot;SHOULD NOT&quot;, &quot;RECOMMENDED&quot;, The key words "<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>", "<bcp14>REQU
&quot;NOT RECOMMENDED&quot;, &quot;MAY&quot;, and &quot;OPTIONAL& IRED</bcp14>", "<bcp14>SHALL</bcp14>", "<bcp14>SHALL
quot; NOT</bcp14>", "<bcp14>SHOULD</bcp14>", "<bcp14>SHOULD NOT</bcp14>", "<bcp14>
in this document are to be interpreted as described in BCP 14 RECOMMENDED</bcp14>", "<bcp14>NOT RECOMMENDED</bcp14>",
<xref target="RFC2119"></xref> <xref target="RFC8174"></xref> "<bcp14>MAY</bcp14>", and "<bcp14>OPTIONAL</bcp14>" in this document are to
when, and only when, they appear in all capitals, as shown here.< be interpreted as
/t> described in BCP&nbsp;14 <xref target="RFC2119"/> <xref target="RFC8174"/>
when, and only when, they appear in all capitals, as shown here.
</t>
</section>
</section> </section>
</section> <section anchor="terms" numbered="true" toc="default">
<name>Terminology</name>
<section anchor="terms" title="Terminology">
<t>The definitions of the basic terms are identical to those found in Alt
ernate-Marking <xref target="I-D.ietf-ippm-rfc8321bis" format="default"/>.
It is to be remembered that <xref target="I-D.ietf-ippm-rfc8321bis" forma
t="default"/> is valid for point-to-point unicast flows and BUM traffic.</t>
<t>The important new terms that need to be explained are listed below:<li
st>
<t>Multipoint Alternate-Marking: Extension to <xref target="I-D.ietf-ippm <t>The use of the basic terms are identical to those found in Alternate Ma
-rfc8321bis" format="default"/>, valid for multipoint-to-multipoint rking <xref target="RFC9341" format="default"/>.
unicast flows, anycast, and ECMP flows. It can also be referred to as Clu It is to be remembered that <xref target="RFC9341" format="default"/> is v
stered Alternate-Marking.</t> alid for point-to-point unicast flows and BUM traffic.</t>
<t>Flow definition: The concept of flow is generalized in this document. <t>The important new terms are explained below:</t>
The identification fields are selected without any constraints
and, in general, the flow can be a multipoint-to-multipoint flow, as a re
sult of aggregate point-to-point flows.</t>
<t>Monitoring Network: Identified with the nodes of the network that are <dl>
the measurement points (MPs) and <dt>Multipoint Alternate Marking:
the links that are the connections between MPs. The monitoring network gr </dt>
aph depends on the flow definition, so it can <dd>Extension to <xref target="RFC9341" format="default"/>, valid for
represent a specific flow or the entire network topology as aggregate of multipoint-to-multipoint unicast flows, anycast, and ECMP flows. It
all the flows. Each node of the monitoring network can also be referred to as "Clustered Alternate Marking".
cannot be both a source and a destination of the flow.</t> </dd>
<t>Cluster: Smallest identifiable non-trivial subnetwork of the entire mo <dt>Flow definition:
nitoring network graph that still satisfies the condition </dt>
that the number of packets that go in is the same as the number that go o <dd>The concept of flow is generalized in this document. The
ut. identification fields are selected without any constraints and, in
A cluster partition algorithm, such as that found in <xref target="nclust general, the flow can be a multipoint-to-multipoint flow, as a result
ering_algo"></xref>, can be applied to split the monitoring network of aggregate point-to-point flows.
into clusters.</t> </dd>
<t>Multipoint metrics: Packet loss, delay and delay variation are extende <dt>Monitoring network:
d to the case of multipoint flows. </dt>
It is possible to compute these metrics on the basis of multipoint paths <dd>Identified with the nodes of the network that are the measurement
in order to associate the measurements points (MPs) and the links that are the connections between MPs. The
to a cluster, a combination of clusters, or the entire monitored network. monitoring network graph depends on the flow definition, so it can
For delay and delay variation, represent a specific flow or the entire network topology as aggregate
it is also possible to define the metrics on a single-packet basis, and i of all the flows. Each node of the monitoring network cannot be both a
t means that the multipoint path is used source and a destination of the flow.
to easily couple packets between input and output nodes of a multipoint p </dd>
ath.</t>
</list></t> <dt>Cluster:
</dt>
<dd>Smallest identifiable non-trivial subnetwork of the entire
monitoring network graph that still satisfies the condition that the
number of packets that go in is the same as the number that go out. A
cluster partition algorithm, such as that found in <xref
target="nclustering_algo" format="default"/>, can be applied to split
the monitoring network into clusters.
</dd>
<t>The next section highlights the correlation with the terms used in <xr <dt>Multipoint metrics:
ef target="RFC5644">RFC 5644</xref>.</t> </dt>
<dd>Packet loss, delay, and delay variation are extended to the case
of multipoint flows. It is possible to compute these metrics on the
basis of multipoint paths in order to associate the measurements to a
cluster, a combination of clusters, or the entire monitored
network. For delay and delay variation, it is also possible to define
the metrics on a single-packet basis, and it means that the multipoint
path is used to easily couple packets between input and output nodes
of a multipoint path.
</dd>
</dl>
<section title="Correlation with RFC 5644"> <t>The next section highlights the correlation with the terms used in <xref targ
<t><xref target="RFC5644" format="default">RFC 5644</xref> is limited to et="RFC5644" format="default"/>.</t>
active measurements <section numbered="true" toc="default">
<name>Correlation with RFC 5644</name>
<t><xref target="RFC5644" format="default"/> is limited to active measur
ements
using a single source packet or stream. Its scope is also limited to observ ations of corresponding packets along the using a single source packet or stream. Its scope is also limited to observ ations of corresponding packets along the
path (spatial metric) and at one or more destinations (one-to-group) along the path.</t> path (spatial metric) and at one or more destinations (one-to-group) along the path.</t>
<t>Instead, the scope of this memo is to define multiparty metrics for p
<t>Instead, the scope of this memo is to define multiparty metrics for p assive and hybrid
assive and hybrid
measurements in a group-to-group topology with multiple sources and dest inations.</t> measurements in a group-to-group topology with multiple sources and dest inations.</t>
<t><xref target="RFC5644" format="default"/> introduces metric names tha
<t><xref target="RFC5644" format="default">RFC 5644</xref> introduces me t can be reused here
tric names that can be reused here
but have to be extended and rephrased to be applied to the Alternate-Mar king schema:</t> but have to be extended and rephrased to be applied to the Alternate-Mar king schema:</t>
<t><list style="letters"> <ol spacing="normal" type="a">
<t>the multiparty metrics are not only one-to-group metrics but can be also <li>the multiparty metrics are not only one-to-group metrics but can a
group-to-group lso be group-to-group
metrics;</t> metrics;</li>
<t>the spatial metrics, used for measuring the performance of segments of a <li>the spatial metrics, used for measuring the performance of segment
source to destination path, s of a source-to-destination path,
are applied here to clusters.</t> are applied here to clusters.</li>
</list></t> </ol>
</section> </section>
</section> </section>
<section numbered="true" toc="default">
<section title="Flow Classification"> <name>Flow Classification</name>
<t>A unicast flow is identified by all the packets having a set of common characteristics. <t>A unicast flow is identified by all the packets having a set of common characteristics.
This definition is inspired by <xref target="RFC7011" format="default"> This definition is inspired by <xref target="RFC7011" format="default"/
RFC 7011</xref>.</t> >.</t>
<t>As an example, by considering a flow as all the packets sharing the sam e <t>As an example, by considering a flow as all the packets sharing the sam e
source IP address or the same destination IP address, it is easy to und erstand source IP address or the same destination IP address, it is easy to und erstand
that the resulting pattern will not be a point-to-point connection, that the resulting pattern will not be a point-to-point connection
but a point-to-multipoint or multipoint-to-point connection.</t> but a point-to-multipoint or multipoint-to-point connection.</t>
<t>In general, a flow can be defined by a set of selection rules used to m atch <t>In general, a flow can be defined by a set of selection rules used to m atch
a subset of the packets processed by the network device. These rules a subset of the packets processed by the network device. These rules
specify a set of Layer 3 and Layer 4 header fields (identification fiel ds) specify a set of Layer 3 and Layer 4 header fields (identification fiel ds)
and the relative values that must be found in matching packets.</t> and the relative values that must be found in matching packets.</t>
<t>The choice of the identification fields directly affects the type of
<t>The choice of the identification fields directly affects the type of
paths that the flow would follow in the network. In fact, it is paths that the flow would follow in the network. In fact, it is
possible to relate a set of identification fields with the pattern possible to relate a set of identification fields with the pattern
of the resulting graphs, as listed in <xref target="Flows" />.</t> of the resulting graphs, as listed in <xref target="Flows" format="defa
ult"/>.</t>
<t>A TCP 5-tuple usually identifies flows following either a single pat h <t>A TCP 5-tuple usually identifies flows following either a single pat h
or a point-to-point multipath (in the case of load balancing). On the c ontrary, or a point-to-point multipath (in the case of load balancing). On the c ontrary,
a single source address selects aggregate flows following a point-to-mu a single source address selects aggregate flows following a point-to-mu
ltipoint, ltipoint path,
while a multipoint-to-point can be the result of a matching on a single while a multipoint-to-point path can be the result of a matching on a s
ingle
destination address. destination address.
In the case where a selection rule and its reverse are used for bidirec tional measurements, In the case where a selection rule and its reverse are used for bidirec tional measurements,
they can correspond to a point-to-multipoint in one direction they can correspond to a point-to-multipoint path in one direction
and a multipoint-to-point in the opposite direction.</t> and a multipoint-to-point path in the opposite direction.</t>
<t>So the flows to be monitored are selected into the monitoring points
<t>So the flows to be monitored are selected into the monitoring points
using packet selection rules, which can also change the pattern of the monitored using packet selection rules, which can also change the pattern of the monitored
network.</t> network.</t>
<t>Note that, more generally, the flow can be defined at different levels based <t>Note that, more generally, the flow can be defined at different levels based
on the potential encapsulation, and additional conditions that are not in the packet header on the potential encapsulation, and additional conditions that are not in the packet header
can also be included as part of matching criteria.</t> can also be included as part of matching criteria.</t>
<t>The Alternate-Marking Method is applicable only to a single path (and
<t>The Alternate-Marking method is applicable only to a single path (an
d
partially to a one-to-one multipath), so the extension proposed in partially to a one-to-one multipath), so the extension proposed in
this document is suitable also for the most general case of multipoint- to-multipoint, this document is suitable also for the most general case of multipoint- to-multipoint,
which embraces all the other patterns of <xref target="Flows" />.</t> which embraces all the other patterns in <xref target="Flows" format="d
efault"/>.</t>
<figure anchor="Flows" title="Flow Classification"> <figure anchor="Flows">
<name>Flow Classification</name>
<artwork name="" type="" align="left" alt=""><![CDATA[ <artwork name="" type="" align="left" alt=""><![CDATA[
point-to-point single path point-to-point single path
+------+ +------+ +------+ +------+ +------+ +------+
---<> R1 <>----<> R2 <>----<> R3 <>--- ---<> R1 <>----<> R2 <>----<> R3 <>---
+------+ +------+ +------+ +------+ +------+ +------+
point-to-point multipath point-to-point multipath
+------+ +------+
<> R2 <> <> R2 <>
/ +------+ \ / +------+ \
/ \ / \
skipping to change at line 469 skipping to change at line 364
+------+ \ +------+ \
+------+ \ +------+ +------+ \ +------+
---<> R2 <> <> R7 <>--- ---<> R2 <> <> R7 <>---
+------+ \ / +------+ +------+ \ / +------+
\ +------+ / \ +------+ /
<> R5 <> <> R5 <>
/ +------+ \ / +------+ \
+------+ / \ +------+ +------+ / \ +------+
---<> R3 <> <> R8 <>--- ---<> R3 <> <> R8 <>---
+------+ +------+ +------+ +------+
]]></artwork> ]]></artwork>
</figure> </figure>
<t>The case of unicast flow is considered in <xref target="Flows" format="
<t>The case of unicast flow is considered in <xref target="Flows"/>. default"/>.
The anycast flow is also covered, since it is only a special case of a unicast flow if routing is The anycast flow is also covered, since it is only a special case of a unicast flow if routing is
stable throughout the measurement period. stable throughout the measurement period.
Furthermore, an ECMP flow is in scope by definition, since it is a poin t-to-multipoint unicast flow.</t> Furthermore, an ECMP flow is in scope by definition, since it is a poin t-to-multipoint unicast flow.</t>
</section> </section>
<section numbered="true" toc="default">
<name>Extension of the Method to Multipoint Flows</name>
<section title="Extension of the Method to Multipoint Flows"> <t>By using the Alternate-Marking Method, only point-to-point paths can be
monitored.
<t>By using the Alternate-Marking method, only point-to-point paths can b
e monitored.
To have an IP (TCP/UDP) flow that follows a point-to-point path, in ge neral we have to define, To have an IP (TCP/UDP) flow that follows a point-to-point path, in ge neral we have to define,
with a specific value, 5 identification fields (IP Source, IP Dest with a specific value, 5 identification fields (IP Source, IP Destinat
ination, Transport Protocol, ion, Transport Protocol,
Source Port, Destination Port).</t> Source Port, and Destination Port).</t>
<t>Multipoint Alternate Marking enables the performance measurement for mu
<t>Multipoint Alternate-Marking enables the performance measurement for ltipoint flows
multipoint flows
selected by identification fields without any constraints (even the entire network production traffic). selected by identification fields without any constraints (even the entire network production traffic).
It is also possible to use multiple marking points for the same monitor ed flow.</t> It is also possible to use multiple marking points for the same monitor ed flow.</t>
<section numbered="true" toc="default">
<section title="Monitoring Network"> <name>Monitoring Network</name>
<t>The monitoring network is deduced from the production network by iden
<t>The monitoring network is deduced from the production network tifying
by identifying the nodes of the graph that are the measurement points and the links
the nodes of the graph that are the measurement points, and the links that are the
that are the
connections between measurement points. It can be modeled as a set of nodes and connections between measurement points. It can be modeled as a set of nodes and
a set of directed arcs which connect pairs of nodes.</t> a set of directed arcs that connect pairs of nodes.</t>
<t>There are some techniques that can help with the building of the moni toring network (as an example, <t>There are some techniques that can help with the building of the moni toring network (as an example,
see <xref target="I-D.ietf-ippm-route" format="default"/>). In genera l, there are different options: see <xref target="RFC9198" format="default"/>). In general, there are different options:
the monitoring network can be obtained by considering all the possible the monitoring network can be obtained by considering all the possible
paths for the traffic or periodically checking the traffic (e.g. daily, paths for the traffic or periodically checking the traffic (e.g., daily,
weekly, monthly) and updating the graph as appropriate, weekly, and monthly) and updating the graph as appropriate,
but this is up to the Network Management System (NMS) configuration.< /t> but this is up to the Network Management System (NMS) configuration.< /t>
<t>So a graph model of the monitoring network can be built according to the Alternate-Marking <t>So a graph model of the monitoring network can be built according to the Alternate-Marking
method: the monitored interfaces and links are identified. Only the m easurement points and links Method, where the monitored interfaces and links are identified. Only the measurement points and links
where the traffic has flowed have to be represented in the graph.</t> where the traffic has flowed have to be represented in the graph.</t>
<t>A simple example of a monitoring network graph is shown in <xref targ
<t>A simple example of a monitoring network graph is showed in <x et="Appendix" format="default"/>.</t>
ref target="Appendix"/>.</t> <t>Each monitoring point is characterized by the packet counter
<t>Each monitoring point is characterized by the packet counter
that refers only to a marking period of the monitored flow. that refers only to a marking period of the monitored flow.
Also, it is assumed that there be a monitoring point at all possi ble egress points Also, it is assumed that there is a monitoring point at all possi ble egress points
of the multipoint monitored network.</t> of the multipoint monitored network.</t>
<t>The same is also applicable for the delay, but it will be described
<t>The same is also applicable for the delay, but it will be desc
ribed
in the following sections.</t> in the following sections.</t>
<t>The rest of the document assumes that the traffic is going from left
<t>The rest of the document assumes that the traffic is going fro to right
m left to right
in order to simplify the explanation. But the analysis done for o ne direction applies in order to simplify the explanation. But the analysis done for o ne direction applies
equally to all directions.</t> equally to all directions.</t>
</section> </section>
<section anchor="Nploss" numbered="true" toc="default">
<section anchor="Nploss" title="Network Packet Loss"> <name>Network Packet Loss</name>
<t>Since all the packets of the considered flow leaving the <t>Since all the packets of the considered flow leaving the
network have previously entered the network, the number of network have previously entered the network, the number of
packets counted by all the input nodes is always greater than, packets counted by all the input nodes is always greater than,
or equal to, the number of packets counted by all the output nodes. or equal to, the number of packets counted by all the output nodes.
It is assumed that routing is stable during the measurement perio d It is assumed that routing is stable during the measurement perio d
while packet fragmentation must be handled as described in while packet fragmentation must be handled as described in
<xref target="I-D.ietf-ippm-rfc8321bis" format="default"/>.</t> <xref target="RFC9341" format="default"/>.</t>
<t>In the case of no packet loss occurring in the marking period,
<t>In the case of no packet loss occurring in the marking period,
if all the input and output points of the network domain to be mo nitored if all the input and output points of the network domain to be mo nitored
are measurement points, the sum of the number of packets on all t he are measurement points, the sum of the number of packets on all t he
ingress interfaces equals the number on egress interfaces for the ingress interfaces equals the number on egress interfaces for the
monitored flow. In this circumstance, if no packet loss occurs, monitored flow. In this circumstance, if no packet loss occurs,
the intermediate measurement points only have the task of splitting the intermediate measurement points only have the task of splitting
the measurement.</t> the measurement.</t>
<t>It is possible to define the network packet loss of one monitored flo
<t>It is possible to define the Network Packet Loss of one monitored flow w
for a single period. In a packet network, the number of lost packets is the for a single period. In a packet network, the number of lost packets is the
number of packets counted by the input nodes minus the number of pack ets number of packets counted by the input nodes minus the number of pack ets
counted by the output nodes. This is true for every packet flow counted by the output nodes. This is true for every packet flow
in each marking period.</t> in each marking period.</t>
<t>The monitored network packet loss with n input nodes and m output nod
<t>The monitored network packet loss with n input nodes and m output es
nodes
is given by:</t> is given by:</t>
<artwork><![CDATA[
<t>PL = (PI1 + PI2 +...+ PIn) - (PO1 + PO2 +...+ POm)</t> PL = (PI1 + PI2 +...+ PIn) - (PO1 + PO2 +...+ POm)
]]></artwork>
<t>where:</t> <t>where:</t>
<ul>
<t>PL is the network packet loss (number of lost packets)</t> <li>PL is the network packet loss (number of lost packets);
</li>
<t>PIi is the number of packets flowed through the i-th input node in <li>PIi is the number of packets flowed through the i-th
this period</t> input node in this period; and
</li>
<t>POj is the number of packets flowed through the j-th output node i <li>POj is the number of packets flowed through the
n j-th output node in this period.
this period</t> </li>
</ul>
<t>The equation is applied on a per-time-interval basis and a per-flo <t>The equation is applied on a per-time-interval basis and a per-flow b
w basis:<list> asis:</t>
<ul empty="false" spacing="normal">
<t>The reference interval is the Alternate-Marking period, as de <li>The reference interval is the Alternate-Marking period, as defined
fined in <xref target="RFC9341" format="default"/>.</li>
in <xref target="I-D.ietf-ippm-rfc8321bis" format="default"/>.</t> <li>The flow definition is generalized here. Indeed, as described befo
re, a multipoint packet flow
<t>The flow definition is generalized here. Indeed, as described is considered, and the identification fields can be selected without
before, a multipoint packet flow any constraints.</li>
is considered, and the identification fields can be selected without </ul>
any constraints.</t> </section>
</list></t>
</section>
</section> </section>
<section anchor="nclustering" numbered="true" toc="default">
<section anchor="nclustering" title="Network Clustering"> <name>Network Clustering</name>
<t>The previous equation of <xref target="Nploss"></xref> can determine th <t>The previous equation of <xref target="Nploss" format="default"/> can d
e number of packets lost etermine the number of packets lost
globally in the monitored network, exploiting only the data provided by the counters globally in the monitored network, exploiting only the data provided by the counters
in the input and output nodes.</t> in the input and output nodes.</t>
<t>In addition, it is possible to leverage the data provided by the other
<t>In addition, it is possible to leverage the data provided by the oth counters
er counters
in the network to converge on the smallest identifiable subnetworks whe re the losses occur.</t> in the network to converge on the smallest identifiable subnetworks whe re the losses occur.</t>
<t>As defined in <xref target="terms" format="default"/>, a cluster is a n
<t>As defined in <xref target="terms"></xref>, a cluster is a non-trivi on-trivial subnetwork of the
al subnetwork of the
entire monitoring network graph that still satisfies the condition that the number of packets that entire monitoring network graph that still satisfies the condition that the number of packets that
go in is the same as the number that go out, if no packet loss occurs. go in is the same as the number that go out, if no packet loss occurs.
According to this definition, a cluster should contain all the arcs ema nating from its input nodes According to this definition, a cluster should contain all the arcs ema nating from its input nodes
and all the arcs terminating at its output nodes. This ensures that we can count all the packets and all the arcs terminating at its output nodes. This ensures that we can count all the packets
(and only those) exiting an input node again at the output node, whatev er path they follow.</t> (and only those) exiting an input node again at the output node, whatev er path they follow.</t>
<t>As for the entire monitoring network graph, the cluster is defined on a per-flow basis. <t>As for the entire monitoring network graph, the cluster is defined on a per-flow basis.
In a completely monitored network (a network where every network interf ace is monitored), In a completely monitored network (a network where every network interf ace is monitored),
each network device corresponds to a cluster, and each physical link co rresponds to two each network device corresponds to a cluster, and each physical link co rresponds to two
clusters (one for each device).</t> clusters (one for each device).</t>
<t>Clusters can have different sizes depending on the flow-filtering crite
<t>Clusters can have different sizes depending on the flow-filtering cr ria adopted.</t>
iteria adopted.</t> <t>Moreover, sometimes clusters can be optionally simplified. For example,
when two monitored interfaces
<t>Moreover, sometimes clusters can be optionally simplified. For examp
le, when two monitored interfaces
are divided by a single router (one is the input interface, the other i s the output interface, are divided by a single router (one is the input interface, the other i s the output interface,
and the router has only these two interfaces), instead of counting exac tly twice, upon entering and leaving, and the router has only these two interfaces), instead of counting exac tly twice, upon entering and leaving,
it is possible to consider a single measurement point. In this case, it is possible to consider a single measurement point. In this case,
we do not care about the internal packet loss of the router.</t> we do not care about the internal packet loss of the router.</t>
<t>It is worth highlighting that it might also be convenient to define clu sters based on the topological <t>It is worth highlighting that it might also be convenient to define clu sters based on the topological
information so that they are applicable to all the possible flows in th e monitored network.</t> information so that they are applicable to all the possible flows in th e monitored network.</t>
<t>Note that, in case of translation or encapsulation, the cluster propert
<t>Note that, in case of translation or encapsulation, the cluster prop ies must also be invariant.</t>
erties must also be invariant.</t> <section anchor="nclustering_algo" numbered="true" toc="default">
<name>Algorithm for Clusters Partition</name>
<section anchor="nclustering_algo" title="Algorithm for Clusters Partition <t>A simple algorithm can be applied in order to split the monitoring ne
"> twork into clusters.
This can be done for each direction separately; indeed, a node cannot b
<t>A simple algorithm can be applied in order to split the monitoring n e both a source
etwork into clusters.
This can be done for each direction separately, indeed a node cannot be
both a source
and a destination. The clusters partition is based on the monitoring ne twork graph, and a destination. The clusters partition is based on the monitoring ne twork graph,
which can be valid for a specific flow or can also be general and valid for the entire network topology.</t> which can be valid for a specific flow or can also be general and valid for the entire network topology.</t>
<t>It is a two-step algorithm:</t>
<t>It is a two-step algorithm:<list style="symbols"> <ul spacing="normal">
<t>Group the links where there is the same starting node;</t> <li>Group the links where there is the same starting node;</li>
<t>Join the grouped links with at least one ending node in common.</t> <li>Join the grouped links with at least one ending node in common.</l
</list></t> i>
</ul>
<t>Considering that the links are unidirectional, the first step <t>Considering that the links are unidirectional, the first step
implies listing all the links as connections between two nodes and implies listing all the links as connections between two nodes and
grouping the different links if they have the same starting node. grouping the different links if they have the same starting node.
Note that it is possible to start from any link, and the procedure Note that it is possible to start from any link, and the procedure
will work. Following this classification, the second step implies will work. Following this classification, the second step implies
eventually joining the groups classified in the first step by looking a t the ending nodes. eventually joining the groups classified in the first step by looking a t the ending nodes.
If different groups have at least one common ending node, they are put together and belong to the same set. If different groups have at least one common ending node, they are put together and belong to the same set.
After the application of the two steps of the algorithm, each one of th e composed sets of links, After the application of the two steps of the algorithm, each one of th e composed sets of links,
together with the endpoint nodes, constitutes a cluster.</t> together with the endpoint nodes, constitutes a cluster.</t>
<t>A simple application of the clusters partition is shown in <xref targ
<t>A simple application of the clusters partition is showed in <xref ta et="Appendix" format="default"/>.</t>
rget="Appendix"/>.</t> <t>The algorithm, as applied in the example of a point-to-multipoint net
work,
<t>The algorithm, as applied in the example of a point-to-multipoint ne works for the more general case of a multipoint-to-multipoint network i
twork, n the same way.
works for the more general case of multipoint-to-multipoint network in It should be highlighted that for a multipoint-to-multipoint network, t
the same way. he multiple sources
It should be highlighted that for a multipoint-to-multipoint network th <bcp14>MUST</bcp14> mark the traffic coherently and <bcp14>MUST</bcp14>
e multiple sources be synchronized with all the other nodes
MUST mark coherently the traffic and MUST be synchronized with all the according to the timing requirements detailed in <xref target="sync-tim
other nodes ing" format="default"/>.</t>
according to the timing requirements detailed in <xref target="sync-tim <t>When the clusters partition is done, the calculation of packet loss,
ing"></xref>.</t> delay, and delay variation
<t>When the clusters partition is done, the calculation of packet loss,
delay and delay variation
can be made on a cluster basis. can be made on a cluster basis.
Note that the packet counters for each marking period permit calculatin g the packet rate Note that the packet counters for each marking period permit calculatin g the packet rate
on a cluster basis, so Committed Information Rate (CIR) and Excess Info rmation Rate (EIR) on a cluster basis, so Committed Information Rate (CIR) and Excess Info rmation Rate (EIR)
could also be deduced on a cluster basis.</t> could also be deduced on a cluster basis.</t>
<t>Obviously, by combining some clusters in a new connected subnetwork,
<t>Obviously, by combining some clusters in a new connected subnetwork the packet-loss rule is still true.
the packet-loss rule is still true.
So it is also possible to consider combinations of clusters if and wher e it suits.</t> So it is also possible to consider combinations of clusters if and wher e it suits.</t>
<t>In this way, in a very large network, there is no need to configure d
<t>In this way, in a very large network, there is no need to configure etailed
detailed
filter criteria to inspect the traffic. It is possible to check a multi point network and, filter criteria to inspect the traffic. It is possible to check a multi point network and,
in case of problems, go deep with a step-by-step cluster analysis, in case of problems, go deep with a step-by-step cluster analysis,
but only for the cluster or combination of clusters where the problem h appens.</t> but only for the cluster or combination of clusters where the problem h appens.</t>
<t>In summary, once a flow is defined, the algorithm to build the cluste
<t>In summary, once a flow is defined, the algorithm to build the clusters rs partition
partition
is based on topological information; therefore, it considers all the po ssible links and is based on topological information; therefore, it considers all the po ssible links and
nodes that could potentially be crossed by the given flow, even if ther e is no traffic. nodes that could potentially be crossed by the given flow, even if ther e is no traffic.
So, if the flow does not enter or traverse all the nodes, the counters have a non-zero So if the flow does not enter or traverse all the nodes, the counters h ave a non-zero
value for the involved nodes and a zero value for the other nodes witho ut traffic; value for the involved nodes and a zero value for the other nodes witho ut traffic;
but in the end, all the formulas are still valid.</t> but in the end, all the formulas are still valid.</t>
<t>The algorithm described above is an iterative clustering algorithm si
<t>The algorithm described above is an iterative clustering algorithm sinc nce it executes steps in iterations,
e it executes steps in iterations,
but it is also possible to apply a recursive clustering algorithm as de tailed in but it is also possible to apply a recursive clustering algorithm as de tailed in
<xref target="IEEE-ACM-ToN-MPNPM" format="default"/>.</t> <xref target="IEEE-ACM-TON-MPNPM" format="default"/>.</t>
<t>The complete and mathematical analysis of the possible algorithms for
<t>The complete and mathematical analysis of the possible algorithms for c the clusters
lusters
partition, including the considerations in terms of efficiency and a co mparison partition, including the considerations in terms of efficiency and a co mparison
between the different methods, is in the paper <xref target="IEEE-ACM-T oN-MPNPM" format="default"/>.</t> between the different methods, is in the paper <xref target="IEEE-ACM-T ON-MPNPM" format="default"/>.</t>
</section> </section>
</section> </section>
<section anchor="Mploss" numbered="true" toc="default">
<section anchor="Mploss" title="Multipoint Packet Loss Measurement"> <name>Multipoint Packet-Loss Measurement</name>
<t>The Network Packet Loss, defined in <xref target="Nploss"></xref>, vali <t>The network packet loss, defined in <xref target="Nploss" format="defau
d for the lt"/>, valid for the
entire monitored flow, can easily be extended to each multipoint path entire monitored flow, can easily be extended to each multipoint path
(e.g., the whole multipoint network, a cluster, or a combination of clu sters). (e.g., the whole multipoint network, a cluster, or a combination of clu sters).
In this way it is possible to calculate Multipoint Packet Loss that is In this way, it is possible to calculate Multipoint Packet Loss that is
representative of a multipoint path.</t> representative of a multipoint path.</t>
<t>The same equation of <xref target="Nploss" format="default"/> can be ap
<t>The same equation of <xref target="Nploss"></xref> can be applied to plied to a generic multipoint path
a generic multipoint path
like a cluster or a combination of clusters, where the number of packet s are those entering and leaving like a cluster or a combination of clusters, where the number of packet s are those entering and leaving
the multipoint path.</t> the multipoint path.</t>
<t>By applying the algorithm described in <xref target="nclustering_algo"
<t>By applying the algorithm described in <xref target="nclustering_alg format="default"/>, it is possible to split
o"></xref>, it is possible to split
the monitoring network into clusters. Then, packet loss can be measured on a cluster basis for each single period the monitoring network into clusters. Then, packet loss can be measured on a cluster basis for each single period
by considering the counters of the input and output nodes that belong t o the specific cluster. by considering the counters of the input and output nodes that belong t o the specific cluster.
This can be done for every packet flow in each marking period.</t> This can be done for every packet flow in each marking period.</t>
</section> </section>
<section numbered="true" toc="default">
<section title="Multipoint Delay and Delay Variation"> <name>Multipoint Delay and Delay Variation</name>
<t>The same line of reasoning can be applied to delay and delay variation. <t>The same line of reasoning can be applied to delay and delay variation.
The delay measurement methods defined in <xref target="I-D.ietf-ippm-rfc83 21bis" format="default"/> The delay measurement methods defined in <xref target="RFC9341" format="de fault"/>
can be extended to the case of multipoint flows. can be extended to the case of multipoint flows.
It is important to highlight that both delay and delay-variation measureme nts It is important to highlight that both delay and delay-variation measureme nts
make sense in a multipoint path. The delay variation is calculated by cons idering make sense in a multipoint path. The delay variation is calculated by cons idering
the same packets selected for measuring the delay.</t> the same packets selected for measuring the delay.</t>
<t>In general, it is possible to perform delay and delay-variation measure ments <t>In general, it is possible to perform delay and delay-variation measure ments
on the basis of multipoint paths or single packets:<list style="symbols"> on the basis of multipoint paths or single packets:</t>
<ul spacing="normal">
<t>Delay measurements on the basis of multipoint paths mean that <li>Delay measurements on the basis of multipoint paths mean that the de
the delay value is representative lay value is representative
of an entire multipoint path (e.g., the whole multipoint network, a c luster, or a combination of an entire multipoint path (e.g., the whole multipoint network, a c luster, or a combination
of clusters).</t> of clusters).</li>
<li>Delay measurements on a single-packet basis mean that it is possible
<t>Delay measurements on a single-packet basis mean that it is po to use
ssible to use
a multipoint path just to easily couple packets between input and out put nodes of a multipoint path, a multipoint path just to easily couple packets between input and out put nodes of a multipoint path,
as described in the following sections.</t> as described in the following sections.</li>
</list></t> </ul>
<section anchor="Mdelay" numbered="true" toc="default">
<section anchor="Mdelay" title="Delay Measurements on a Multipoint-Paths B <name>Delay Measurements on a Multipoint-Paths Basis</name>
asis"> <section anchor="M_single-marking" numbered="true" toc="default">
<name>Single-Marking Measurement</name>
<section anchor="M_single-marking" title="Single-Marking Measurem <t>Mean delay and mean delay-variation measurements can also be genera
ent"> lized
<t>Mean delay and mean delay-variation measurements can also be generali
zed
to the case of multipoint flows. It is possible to compute the av erage to the case of multipoint flows. It is possible to compute the av erage
one-way delay of packets in one block, a cluster, or the entire one-way delay of packets in one block, a cluster, or the entire
monitored network.</t> monitored network.</t>
<t>The average latency can be measured as the difference
<t>The average latency can be measured as the difference
between the weighted averages of the mean timestamps of the between the weighted averages of the mean timestamps of the
sets of output and input nodes. This means that, in the calculati on, sets of output and input nodes. This means that, in the calculati on,
it is possible to weigh the timestamps with the number of packets it is possible to weigh the timestamps with the number of packets
for each endpoint.</t> for each endpoint.</t>
<t>Note that, since the one-way delay value is representative of a mul
<t>Note that, since the one-way delay value is representative of tipoint path,
a multipoint path,
it is possible to calculate the two-way delay of a multipoint pat h by summing it is possible to calculate the two-way delay of a multipoint pat h by summing
the one-way delays of the two directions, similarly to <xref targ the one-way delays of the two directions, similarly to <xref targ
et="I-D.ietf-ippm-rfc8321bis" format="default"/>.</t> et="RFC9341" format="default"/>.</t>
</section>
</section>
</section> </section>
<section numbered="true" toc="default">
<section title="Delay Measurements on a Single-Packet Basis"> <name>Delay Measurements on a Single-Packet Basis</name>
<section anchor="p_sd-marking" numbered="true" toc="default">
<section anchor="p_sd-marking" title="Single- and Double-Marking Measure <name>Single- and Double-Marking Measurement</name>
ment"> <t>Delay and delay-variation measurements associated with only one pic
<t>Delay and delay-variation measurements associated with only one picke ked packet per period,
d packet per period,
both single and double marked, cannot be easily performed in a mu ltipoint scenario both single and double marked, cannot be easily performed in a mu ltipoint scenario
since there are some limitations:<list style="hanging"> since there are some limitations:</t>
<t>Single marking based on the first/last packet of the interval
does not work properly.
Indeed, by considering a point-to-multipoint scenario, it is not
possible to recognize
which path the first packet of each block takes over the multipo
int flow in order to correlate it.
This is also true for the general case of the multipoint-to-mult
ipoint scenario.</t>
<t>Double marking or multiplexed marking works but only through
statistical means.
In a point-to-multipoint scenario, by selecting only a single pa
cket with the second marking
for each block, it is possible to follow and calculate the delay
for that picked packet.
But the measurement can only be done for a single path in each m
arking period.
To traverse all the paths of the multipoint flow, it can theoret
ically be done by continuing
the measurement for the following marking periods and expect to
span all the paths.
In the general case of a multipoint-to-multipoint path, it is al
so needed to take into account
the multiple source nodes which complicate the correlation of th
e samples. In this case,
it can be possible to select the second marked packet only for a
source node at a time for each block
and cover the remaining source nodes one by one in the next mark
ing periods.</t>
</list></t> <ul>
<li>Single Marking based on the first/last packet of the
interval does not work properly. Indeed, by considering a
point-to-multipoint scenario, it is not possible to
recognize which path the first packet of each block takes
over the multipoint flow in order to correlate it. This is
also true for the general case of the
multipoint-to-multipoint scenario.
</li>
<t>Note that, since the one-way delay measurement is done on a si <li>Double Marking or multiplexed marking works but only
ngle-packet basis, it is always possible through statistical means. In a point-to-multipoint
to calculate the two-way delay but it is not immediate since it i scenario, by selecting only a single packet with the second
s necessary to couple the measurement marking for each block, it is possible to follow and
on each single path with the opposite direction. In this case the calculate the delay for that picked packet. But the
NMS can do the calculation.</t> measurement can only be done for a single path in each
marking period. To traverse all the paths of the multipoint
flow, it can theoretically be done by continuing the
measurement for the following marking periods and expect to
span all the paths. In the general case of a
multipoint-to-multipoint path, it is also needed to take
into account the multiple source nodes that complicate the
correlation of the samples. In this case, it can be possible
to select the second marked packet only for a source node at
a time for each block and cover the remaining source nodes
one by one in the next marking periods.
</li>
</ul>
<t>If a delay measurement is performed for more than one picked p <t>Note that, since the one-way delay measurement is done on a single-
acket and for all the paths of the packet basis, it is always possible
multipoint flow in the same marking period, neither the single- n to calculate the two-way delay, but it is not immediate since it
or the double-marking method are is necessary to couple the measurement
applicable in the multipoint scenario. The packets follow differe on each single path with the opposite direction. In this case, th
nt paths and it becomes very difficult e NMS can do the calculation.</t>
<t>If a delay measurement is performed for more than one picked packet
and for all the paths of the
multipoint flow in the same marking period, neither the Single- n
or the Double-Marking Method are
applicable in the multipoint scenario. The packets follow differe
nt paths, and it becomes very difficult
to correlate marked packets in a multipoint-to-multipoint path if there are more than one per period.</t> to correlate marked packets in a multipoint-to-multipoint path if there are more than one per period.</t>
<t>A desirable option is to monitor simultaneously all the paths of a
<t>A desirable option is to monitor simultaneously all the paths multipoint path
of a multipoint path
in the same marking period. For this purpose, hashing can be used , as reported in the next section.</t> in the same marking period. For this purpose, hashing can be used , as reported in the next section.</t>
</section> </section>
<section anchor="p_hashing" numbered="true" toc="default">
<section anchor="p_hashing" title="Hashing Selection Method"> <name>Hashing Selection Method</name>
<t>Sampling and filtering techniques for IP packet selection are intro
<t>RFCs <xref target="RFC5474">5474</xref> and <xref target="RFC5 duced in <xref target="RFC5474"/> and <xref target="RFC5475" format="default"/>.
475">5475</xref> </t>
introduce sampling and filtering techniques for IP packet selecti <t>The hash-based selection methodologies for delay measurement can wo
on.</t> rk in a multipoint-to-multipoint path
and can be used either coupled to mean delay or standalone.</t>
<t>The hash-based selection methodologies for delay measurement c <t><xref target="IEEE-NETWORK-PNPM" format="default"/> introduces how
an work in a multipoint-to-multipoint path to use the hash method
and can be used either coupled to mean delay or stand-alone.</t> (see <xref target="RFC5474" format="default"/> and <xref target="
RFC5475" format="default"/>)
<t><xref target="IEEE-Network-PNPM"/> introduces how to use the hash met combined with the Alternate-Marking Method for point-to-point flo
hod ws.
(<xref target="RFC5474">RFC 5474</xref> and <xref target="RFC5475 It is also called "Mixed Hashed Marking" because it refers to the conjun
">RFC 5475</xref>) ction of the marking method and
combined with the Alternate-Marking method for point-to-point flo the hashing technique. It involves only the Single Marking; indee
ws. d, it is supposed that Double Marking
It is also called Mixed Hashed Marking because it refers to the conjunct is not used with hashing. The coupling of the Single Marking with
ion of the marking method and the hashing selection allows choosing
the hashing technique. It involves only the single marking, indee
d it is supposed that double marking
is not used with hashing. The coupling of the single marking with
the hashing selection allows choosing
a simplified hash function since the alternation of blocks gives temporal boundaries for the hashing samples. a simplified hash function since the alternation of blocks gives temporal boundaries for the hashing samples.
The marking batches anchor the samples selected with hashing and this eases the correlation of the hashing The marking batches anchor the samples selected with hashing, and this eases the correlation of the hashing
packets along the path. For example, in case a hashed sample is l ost, it is confined to the considered block packets along the path. For example, in case a hashed sample is l ost, it is confined to the considered block
without affecting the identification of the samples for the follo wing blocks.</t> without affecting the identification of the samples for the follo wing blocks.</t>
<t>Using the hash-based sampling, the number of samples in each block
<t>Using the hash-based sampling, the number of samples in each block ma may vary a lot because it depends on
y vary a lot because it depends on
the packet rate that is variable. A dynamic approach can help to have an almost fixed number the packet rate that is variable. A dynamic approach can help to have an almost fixed number
of samples for each marking period, and this is a better option for maki ng regular measurements over time. of samples for each marking period, and this is a better option for maki ng regular measurements over time.
In the hash-based sampling, Alternate-Marking is used to create p eriods, so that hash-based In the hash-based sampling, Alternate Marking is used to create p eriods, so that hash-based
samples are divided into batches, which allows anchoring the selected sa mples to their period. samples are divided into batches, which allows anchoring the selected sa mples to their period.
Moreover, in a dynamic hash-based sampling, it can be possible to dynamically adapt the length of the hash value Moreover, in a dynamic hash-based sampling, it can be possible to dynamically adapt the length of the hash value
to meet the current packet rate, so that the number of samples is bounded in each marking period.</t> to meet the current packet rate, so that the number of samples is bounded in each marking period.</t>
<t>In a multipoint environment, the hashing selection may be the solut
<t>In a multipoint environment, the hashing selection may be the solutio ion
n
for performing delay measurements on specific packets and overcom ing the for performing delay measurements on specific packets and overcom ing the
single- and double-marking limitations.</t> Single- and Double-Marking limitations.</t>
</section> </section>
</section> </section>
</section> </section>
<section anchor="sync-timing" numbered="true" toc="default">
<section anchor="sync-timing" title="Synchronization and Timing"> <name>Synchronization and Timing</name>
<t>It is important to consider the timing aspects, since out-of-order packe <t>It is important to consider the timing aspects, since out-of-order pack
ts happen and have ets happen and have
to be handled as well, as described in <xref target="I-D.ietf-ippm-rfc83 to be handled as well, as described in <xref target="RFC9341" format="de
21bis" format="default"/>.</t> fault"/>.</t>
<t>However, in a multisource situation, an additional issue has to be cons
<t>However, in a multisource situation, an additional issue has to be co idered. With multipoint path,
nsidered. With multipoint path,
the egress nodes will receive alternate marked packets in random order f rom different ingress nodes, the egress nodes will receive alternate marked packets in random order f rom different ingress nodes,
and this must not affect the measurement.</t> and this must not affect the measurement.</t>
<t>So, if we analyze a multipoint-to-multipoint path with more than one ma
<t>So, if we analyze a multipoint-to-multipoint path with more than one rking node,
marking node,
it is important to recognize the reference measurement interval. In general , the measurement it is important to recognize the reference measurement interval. In general , the measurement
interval for describing the results is the interval of the marking node that is more aligned with interval for describing the results is the interval of the marking node that is more aligned with
the start of the measurement, as reported in <xref target="measint" />.< the start of the measurement, as reported in <xref target="measint" form
/t> at="default"/>.</t>
<t>Note that the mark switching approach based on a fixed timer is conside
<t>Note that the mark switching approach based on a fixed timer is consi red in this document.</t>
dered in this document.</t> <figure anchor="measint">
<name>Measurement Interval</name>
<figure anchor="measint" title="Measurement Interval">
<artwork name="" type="" align="left" alt=""><![CDATA[ <artwork name="" type="" align="left" alt=""><![CDATA[
time -> start stop time -> start stop
T(R1) |-------------| T(R1) |-------------|
T(R2) |-------------| T(R2) |-------------|
T(R3) |------------| T(R3) |------------|
]]></artwork> ]]></artwork>
</figure> </figure>
<t>In <xref target="measint" format="default"/>, it is assumed that the no
<t>In <xref target="measint"/>, it is assumed that the node with the de with the
earliest clock (R1) identifies the right starting and ending times of the measurement, earliest clock (R1) identifies the right starting and ending times of the measurement,
but it is just an assumption, and other possibilities could occur. but it is just an assumption and other possibilities could occur.
So, in this case, T(R1) is the measurement interval, and its recognitio So in this case, T(R1) is the measurement interval, and its recognition
n is essential is essential
in order to make comparisons with other active/passive/hybrid Packet Lo in order to make comparisons with other active/passive/hybrid packet-lo
ss metrics.</t> ss metrics.</t>
<t>Regarding the timing constraints of the methodology, <xref target="RFC9
<t>Regarding the timing constraints of the methodology, <xref target="I 341" format="default"/>
-D.ietf-ippm-rfc8321bis" format="default"/>
already describes two contributions that are taken into account: the cl ock error between network devices and already describes two contributions that are taken into account: the cl ock error between network devices and
the network delay between the measurement points.</t> the network delay between the measurement points.</t>
<t>When we expand to a multipoint environment, we have to consider that th
<t>When we expand to a multipoint environment, we have to consider that ere are more marking nodes that mark the traffic
there are more marking nodes that mark the traffic
based on synchronized clock time. But, due to different synchronization issues that may happen, the marking batches based on synchronized clock time. But, due to different synchronization issues that may happen, the marking batches
can be of different lengths and with different offsets when they get mi xed in a multipoint flow. can be of different lengths and with different offsets when they get mi xed in a multipoint flow.
According to <xref target="I-D.ietf-ippm-rfc8321bis" format="default"/> , the maximum clock skew between the network devices According to <xref target="RFC9341" format="default"/>, the maximum clo ck skew between the network devices
is A. Therefore, the additional gap that results between the multiple s ources can be incorporated into A.</t> is A. Therefore, the additional gap that results between the multiple s ources can be incorporated into A.</t>
<figure anchor="timing">
<figure anchor="timing" title="Timing Aspects"> <name>Timing Aspects</name>
<artwork name="" type="" align="left" alt=""><![CDATA[ <artwork name="" type="" align="left" alt=""><![CDATA[
...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB... ...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB...
|<======================================>| |<======================================>|
| L | | L |
...=========>|<==================><==================>|<==========... ...=========>|<==================><==================>|<==========...
| L/2 L/2 | | L/2 L/2 |
|<====>| |<====>| |<====>| |<====>|
d | | d d | | d
|<========================>| |<========================>|
available counting interval available counting interval
]]></artwork> ]]></artwork>
</figure> </figure>
<t>Moreover, it is assumed that the multipoint path can be modeled with a
<t>Moreover, it is assumed that the multipoint path can be modeled with a normal distribution;
normal distribution, otherwise, it is necessary to reformulate based on the type of distribu
otherwise it is necessary to reformulate based on the type of distribut tion.
ion.
Under this assumption, the definition of the guard band d is still appl icable as defined in Under this assumption, the definition of the guard band d is still appl icable as defined in
<xref target="I-D.ietf-ippm-rfc8321bis" format="default"/> and is given <xref target="RFC9341" format="default"/> and is given by:</t>
by:</t> <artwork><![CDATA[
d = A + D_avg + 3*D_stddev,
<t>d = A + D_avg + 3*D_stddev,</t> ]]></artwork>
<t>where A is the clock accuracy, D_avg is the average value of the networ
<t>where A is the clock accuracy, D_avg is the average value of the net k delay,
work delay,
and D_stddev is the standard deviation of the delay.</t> and D_stddev is the standard deviation of the delay.</t>
<t>As shown in <xref target="timing" format="default"/> and according to <
<t>As shown in <xref target="timing"/> and according to <xref target="I xref target="RFC9341" format="default"/>,
-D.ietf-ippm-rfc8321bis" format="default"/>,
the condition that must be satisfied to enable the method to function p roperly is that the available counting interval the condition that must be satisfied to enable the method to function p roperly is that the available counting interval
must be &gt; 0, and that means:</t> must be &gt; 0, and that means:</t>
<artwork><![CDATA[
<t>L - 2d &gt; 0.</t> L - 2d > 0.
]]></artwork>
<t>This formula needs to be verified for each measurement point on the <t>This formula needs to be verified for each measurement point on the mul
multipoint path.</t> tipoint path.</t>
<t>Note that the timing considerations are valid for both packet loss and
<t>Note that the timing considerations are valid for both packet loss a delay measurements.</t>
nd delay measurements.</t>
</section> </section>
<section anchor="finding" title="Recommendations for Deployment"> <section anchor="finding" numbered="true" toc="default">
<t>The methodology described in the previous sections can be applied to <name>Recommendations for Deployment</name>
various performance measurement problems, as also explained in <xref ta <t>The methodology described in the previous sections can be applied to
rget="I-D.ietf-ippm-rfc8321bis" format="default"/>. various performance measurement problems, as also explained in <xref ta
<xref target="RFC8889"></xref> reports experimental examples and <xref rget="RFC9341" format="default"/>.
target="IEEE-Network-PNPM"/> also includes <xref target="RFC8889" format="default"/> reports experimental examples
and <xref target="IEEE-NETWORK-PNPM" format="default"/> also includes
some information about the deployment experience.</t> some information about the deployment experience.</t>
<t>Different deployments are possible using one flag bit, two flag bits, o r the hashing selection:</t>
<t>Different deployments are possible using one flag bit, two flag bits <dl>
or hashing selection:<list> <dt>One flag:
</dt>
<t>One flag: packet loss measurement MUST be done as described in <xre <dd>packet-loss measurement <bcp14>MUST</bcp14> be done as described
f target="Mploss"></xref> in <xref target="Mploss" format="default"/> by applying the network
by applying the network clustering partition described in <xref target clustering partition described in <xref target="nclustering"
="nclustering"></xref>. format="default"/>. Delay measurement <bcp14>MUST</bcp14> be done
Delay measurement MUST be done according to the Mean delay calculation according to the mean delay calculation representative of the
representative of the multipoint path, multipoint path, as described in <xref target="M_single-marking"
as described in <xref target="M_single-marking"></xref>. Single-markin format="default"/>. A Single-Marking Method based on the first/last
g method based on the first/last packet packet of the interval cannot be applied, as mentioned in <xref
of the interval cannot be applied, as mentioned in <xref target="p_sd- target="p_sd-marking" format="default"/>.
marking"></xref>.</t> </dd>
<t>Two flags: packet loss measurement MUST be done as described in <xr
ef target="Mploss"></xref>
by applying the network clustering partition described in <xref target
="nclustering"></xref>.
Delay measurement SHOULD be done on a single packet basis according to
double-marking method <xref target="p_sd-marking"></xref>.
In this case the Mean delay calculation (<xref target="M_single-markin
g"></xref>) MAY also be used
as a representative value of a multipoint path. The choice depends on the
kind of information that is needed,
as further detailed below.</t>
<t>One flag with hash-based selection: packet loss measurement MUST be
done as described in <xref target="Mploss"></xref>
by applying the network clustering partition described in <xref target
="nclustering"></xref>.
Hash-based selection methodologies, introduced in <xref target="p_hash
ing"></xref>, MUST be used for delay measurement.</t>
</list></t>
<t>Similarly to <xref target="I-D.ietf-ippm-rfc8321bis" format="default <dt>Two flags:
"/>, there are some operational guidelines to consider </dt>
for the purpose of deciding to follow the recommendations above and use <dd>packet-loss measurement <bcp14>MUST</bcp14> be done as described
one or two flags or one flag with hash-based selection.<list> in <xref target="Mploss" format="default"/> by applying the network
clustering partition described in <xref target="nclustering"
format="default"/>. Delay measurement <bcp14>SHOULD</bcp14> be done
on a single-packet basis according to the Double-Marking Method (<xref
target="p_sd-marking" format="default"/>). In this case, the mean delay
calculation (<xref target="M_single-marking" format="default"/>)
<bcp14>MAY</bcp14> also be used as a representative value of a
multipoint path. The choice depends on the kind of information that is
needed, as further detailed below.
</dd>
<t>The Multipoint Alternate-Marking method utilizes specific flags in <dt>One flag with hash-based selection:
the packet header, so an important factor is the number of </dt>
flags available for the implementation. Indeed, if there is only one f <dd>packet-loss measurement <bcp14>MUST</bcp14> be done as described
lag available there is no other way, while if two flags are available in <xref target="Mploss" format="default"/> by applying the network
the option with two flags can be considered in comparison with the opt clustering partition described in <xref target="nclustering"
ion of one flag with hash-based selection.</t> format="default"/>. Hash-based selection methodologies, introduced in
<xref target="p_hashing" format="default"/>, <bcp14>MUST</bcp14> be
used for delay measurement.
</dd>
<t>The duration of the Alternate-Marking period affects the frequency </dl>
of the measurement and this is a parameter that can be
decided on the basis of the required temporal sampling. But it cannot
be freely chosen, as explained in <xref target="sync-timing"/>.</t>
<t>The Multipoint Alternate-Marking methodologies enable packet loss, <t>Similarly to <xref target="RFC9341" format="default"/>, there are some operat
delay and delay variation calculation, but in accordance with ional guidelines to consider
the method used (e.g. single-marking or double-marking or hashing sele when deciding which recommendation to use (i.e., one flag or two flags
ction), there is a different kind of information that can be derived. or one flag with hash-based selection.</t>
<ul empty="false" spacing="normal">
<li>The Multipoint Alternate-Marking Method utilizes specific flags in t
he packet header, so an important factor is the number of
flags available for the implementation. Indeed, if there is only one f
lag available, there is no other way, while if two flags are available,
the option with two flags can be considered in comparison with the opt
ion of one flag with hash-based selection.</li>
<li>The duration of the Alternate-Marking period affects the frequency o
f the measurement, and this is a parameter that can be
decided on the basis of the required temporal sampling. But it cannot
be freely chosen, as explained in <xref target="sync-timing" format="default"/>.
</li>
<li>The Multipoint Alternate-Marking methodologies enable packet loss, d
elay, and delay variation calculation, but in accordance with
the method used (e.g., Single Marking, Double Marking, or hashing sele
ction), there is a different kind of information that can be derived.
For example, to get measurements on a multipoint-paths basis, one flag can be used. To get measurements on a single-packet basis, For example, to get measurements on a multipoint-paths basis, one flag can be used. To get measurements on a single-packet basis,
two flags are preferred. For this reason, the type of data needed in t two flags are preferred. For this reason, the type of data needed in t
he specific scenario is an additional element to take into account.</t> he specific scenario is an additional element to take into account.</li>
<li>The Multipoint Alternate-Marking Methods imply different computation
<t>The Multipoint Alternate-Marking methods imply different computatio al load depending on the method employed. Therefore, the available
nal load depending on the method employed. Therefore, the available
computational resources on the measurement points can also influence t he choice. As an example, mean delay calculation may computational resources on the measurement points can also influence t he choice. As an example, mean delay calculation may
require more processing and it may not be the best option to minimize require more processing, and it may not be the best option to minimize
the computational load.</t> the computational load.</li>
</ul>
</list></t> <t>The experiment with Multipoint Alternate-Marking methodologies confirme
d the benefits of the Alternate-Marking methodology
<t>The experiment with Multipoint Alternate-Marking methodologies confi <xref target="RFC9341" format="default"/> as its extension to the gener
rmed the benefits of the Alternate-Marking methodology al case of
(<xref target="I-D.ietf-ippm-rfc8321bis" format="default"/>), as its ex
tension to the general case of
multipoint-to-multipoint scenarios.</t> multipoint-to-multipoint scenarios.</t>
<t>The Multipoint Alternate-Marking Method <bcp14>MUST</bcp14> only be app
lied to controlled domains,
as per <xref target="RFC9341" format="default"/>.</t>
</section>
<section numbered="true" toc="default">
<name>A Closed-Loop Performance-Management Approach</name>
<t>The Multipoint Alternate-Marking Method MUST only be applied to cont <t>The Multipoint Alternate-Marking framework that is introduced in this d
rolled domains, ocument adds flexibility
as per <xref target="I-D.ietf-ippm-rfc8321bis" format="default"/>.</t>
</section>
<section title="A Closed-Loop Performance-Management Approach">
<t>The Multipoint Alternate-Marking framework that is introduced in this
document adds flexibility
to Performance Management (PM), because it can reduce the order of magnit ude of the packet counters. to Performance Management (PM), because it can reduce the order of magnit ude of the packet counters.
This allows an SDN orchestrator to supervise, control, and manage PM in l arge networks.</t> This allows an SDN orchestrator to supervise, control, and manage PM in l arge networks.</t>
<t>The monitoring network can be considered as a whole or split into clust
<t>The monitoring network can be considered as a whole or split into cluster ers that are the
s that are the
smallest subnetworks (group-to-group segments), maintaining the smallest subnetworks (group-to-group segments), maintaining the
packet-loss property for each subnetwork. The clusters can also be packet-loss property for each subnetwork. The clusters can also be
combined in new, connected subnetworks at different levels, depending on combined in new, connected subnetworks at different levels, depending on
the detail we want to achieve.</t> the detail we want to achieve.</t>
<t>An SDN controller or an NMS can calibrate performance measurements,
<t>An SDN controller or a Network Management System (NMS) can calibrate p
erformance measurements,
since they are aware of the network topology. They can start without exam ining in depth. In case of necessity since they are aware of the network topology. They can start without exam ining in depth. In case of necessity
(packet loss is measured, or the delay is too high), the filtering criter ia could be immediately reconfigured (packet loss is measured or the delay is too high), the filtering criteri a could be immediately reconfigured
in order to perform a partition of the network by using clusters and/or d ifferent combinations of clusters. in order to perform a partition of the network by using clusters and/or d ifferent combinations of clusters.
In this way, the problem can be localized in a specific cluster or a sing le combination of clusters, In this way, the problem can be localized in a specific cluster or a sing le combination of clusters,
and a more detailed analysis can be performed step by step by successive approximation up to and a more detailed analysis can be performed step by step by successive approximation up to
a point-to-point flow detailed analysis. This is the so-called "closed lo op".</t> a point-to-point flow detailed analysis. This is the so-called "closed lo op".</t>
<t>This approach can be called "network zooming" and can be performed in t wo different ways:</t>
<t>This approach can be called "network zooming" and can be performed in <ol>
two different ways:</t> <li>change the traffic filter and select more detailed flows;
<t>1) change the traffic filter and select more detailed flows;</t> </li>
<t>2) activate new measurement points by defining more specified clusters.
</t>
<t>The network-zooming approach implies that some filters or rules are <li>activate new measurement points by defining more specified clusters.
changed and that therefore there is a </li>
transient time to wait once the new network configuration takes effect. T
his time can be determined
by the Network Orchestrator/Controller, based on the network conditions.<
/t>
</ol>
<t>The network-zooming approach implies that some filters or rules are cha
nged; therefore, there is a
transient time to wait once the new network configuration takes effect. T
his time can be determined
by the network orchestrator/controller, based on the network conditions.<
/t>
<t>For example, if the network zooming identifies the performance problem for the traffic coming from <t>For example, if the network zooming identifies the performance problem for the traffic coming from
a specific source, we need to recognize the marked signal from this speci fic source node and its relative path. a specific source, we need to recognize the marked signal from this speci fic source node and its relative path.
For this purpose, we can activate all the available measurement points For this purpose, we can activate all the available measurement points
and better specify the flow filter criteria and better specify the flow filter criteria
(i.e., 5-tuple). As an alternative, it can be enough to select packets (i.e., 5-tuple). As an alternative, it can be enough to select packets
from the specific source for delay measurements; from the specific source for delay measurements;
in this case, it is possible to apply the hashing technique, as mentioned in the previous sections.</t> in this case, it is possible to apply the hashing technique, as mentioned in the previous sections.</t>
<t><xref target="I-D.song-opsawg-ifit-framework" format="default"/> define
<t><xref target="I-D.song-opsawg-ifit-framework" format="default"/> defin s an architecture where the centralized
es an architecture where the centralized data collector and network management can apply the intelligent and flexi
Data Collector and Network Management can apply the intelligent and flexi ble Alternate-Marking algorithm
ble Alternate-Marking algorithm
as previously described.</t> as previously described.</t>
<t>As for <xref target="RFC9341" format="default"/>, it is possible to cla
<t>As for <xref target="I-D.ietf-ippm-rfc8321bis" format="default"/>, it ssify the traffic and mark a portion of
is possible to classify the traffic and mark a portion of
the total traffic. For each period, the packet rate and bandwidth are cal culated from the number of packets. the total traffic. For each period, the packet rate and bandwidth are cal culated from the number of packets.
In this way, the network orchestrator becomes aware if the traffic rate s urpasses limits. In this way, the network orchestrator becomes aware if the traffic rate s urpasses limits.
In addition, more precision can be obtained by reducing the marking perio d; indeed, some implementations use In addition, more precision can be obtained by reducing the marking perio d; indeed, some implementations use
a marking period of 1 sec or less.</t> a marking period of 1 sec or less.</t>
<t>In addition, an SDN controller could also collect the measurement histo
<t>In addition, an SDN controller could also collect the measurement hist ry.</t>
ory.</t> <t>It is important to mention that the Multipoint Alternate-Marking framew
ork also helps Traffic Visualization.
<t>It is important to mention that the Multipoint Alternate-Marking frame
work also helps Traffic Visualization.
Indeed, this methodology is very useful for identifying which path or clu ster is crossed by the flow.</t> Indeed, this methodology is very useful for identifying which path or clu ster is crossed by the flow.</t>
</section> </section>
<section numbered="true" toc="default">
<section title="Security Considerations"> <name>Security Considerations</name>
<t>This document specifies a method of performing measurements that does n ot <t>This document specifies a method of performing measurements that does n ot
directly affect Internet security or applications that run on the Internet . directly affect Internet security or applications that run on the Internet .
However, implementation of this method must be mindful of security However, implementation of this method must be mindful of security
and privacy concerns, as explained in <xref target="I-D.ietf-ippm-rfc8321b is" format="default"/>.</t> and privacy concerns, as explained in <xref target="RFC9341" format="defau lt"/>.</t>
</section> </section>
<section anchor="IANA" numbered="true" toc="default">
<section anchor="IANA" title="IANA Considerations"> <name>IANA Considerations</name>
<t>This document has no IANA actions.</t> <t>This document has no IANA actions.</t>
</section> </section>
<section anchor="Contributors" title="Contributors">
<t>Greg Mirsky<vspace blankLines="0" /> Ericsson<vspace
blankLines="0" /> Email: gregimirsky@gmail.com</t>
<t>Tal Mizrahi<vspace blankLines="0" /> Huawei Technologies<vspace
blankLines="0" /> Email: tal.mizrahi.phd@gmail.com</t>
<t>Xiao Min<vspace blankLines="0" /> ZTE Corp.<vspace
blankLines="0" /> Email: xiao.min2@zte.com.cn</t>
</section>
<section anchor="Acknowledgements" title="Acknowledgements">
<t>The authors would like to thank Martin Duke and Tommy Pauly
for their assistance and their detailed and precious reviews.</t>
</section>
<!-- Possibly a 'Contributors' section ... -->
</middle> </middle>
<!-- *****BACK MATTER ***** -->
<back> <back>
<!-- References split to informative and normative -->
<references title="Normative References">
<?rfc include='reference.RFC.2119'?> <displayreference target="I-D.song-opsawg-ifit-framework" to="OPSAWG-IFIT-FRAM
EWORK"/>
<?rfc include='reference.RFC.8174'?>
<?rfc include='reference.RFC.5475'?>
<?rfc include='reference.RFC.5644'?>
<?rfc include='reference.I-D.ietf-ippm-rfc8321bis'?> <references>
<name>References</name>
<references>
<name>Normative References</name>
</references> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2119.
xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8174.
xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5475.
xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5644.
xml"/>
<references title="Informative References"> <!-- [I-D.ietf-ippm-rfc8321bis] in AUTH48-DONE as of 12/13/22; companion docume
<!-- A reference written by by an organization not a persoN. --> nt RFC YYY1 -->
<reference anchor='RFC9341' target='https://www.rfc-editor.org/info/rfc9341'>
<front>
<title>Alternate-Marking Method</title>
<author initials="G." surname="Fioccola" fullname="Giuseppe Fioccola" role="edit
or">
<organization>Huawei Technologies</organization>
</author>
<author initials="M." surname="Cociglio" fullname="Mauro Cociglio">
<organization>Telecom Italia</organization>
</author>
<author initials="G." surname="Mirsky" fullname="Greg Mirsky">
<organization>Ericsson</organization>
</author>
<author initials="T." surname="Mizrahi" fullname="Tal Mizrahi">
<organization>Huawei Technologies</organization>
</author>
<author initials="T." surname="Zhou" fullname="Tianran Zhou">
<organization>Huawei Technologies</organization>
</author>
<date month="December" year="2022"/>
</front>
<seriesInfo name="RFC" value="9341"/>
<seriesInfo name="DOI" value="10.17487/RFC9341"/>
</reference>
<?rfc include='reference.RFC.8889'?> </references>
<references>
<name>Informative References</name>
<?rfc include='reference.RFC.5474'?> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8889.
xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5474.
xml"/>
<reference anchor='IEEE-Network-PNPM'> <reference anchor="IEEE-NETWORK-PNPM">
<front> <front>
<title>AM-PM: Efficient Network Telemetry using Alternate Marking</title> <title>AM-PM: Efficient Network Telemetry using Alternate Marking</t
<author> itle>
<organization>IEEE Network</organization> <author surname="Mizrahi" initials="T">
</author> <organization/>
<date year='2019' /> </author>
</front> <author surname="Navon" initials="G">
<seriesInfo name='DOI' value='10.1109/MNET.2019.1800152'/> <organization/>
</reference> </author>
<author surname="Fioccola" initials="G">
<organization/>
</author>
<author surname="Cociglio" initials="M">
<organization/>
</author>
<author surname="Chen" initials="M">
<organization/>
</author>
<author surname="Mirsky" initials="G">
<organization/>
</author>
<date month="July" year="2019"/>
</front>
<refcontent>IEEE Network, Vol. 33, Issue 4</refcontent>
<seriesInfo name="DOI" value="10.1109/MNET.2019.1800152"/>
</reference>
<reference anchor='IEEE-ACM-ToN-MPNPM'> <reference anchor="IEEE-ACM-TON-MPNPM">
<front> <front>
<title>Multipoint Passive Monitoring in Packet Networks</title> <title>Multipoint Passive Monitoring in Packet Networks</title>
<author> <author surname="Cociglio" initials="M">
<organization>IEEE/ACM TRANSACTION ON NETWORKING</organization> <organization showOnFrontPage="true"/>
</author> </author>
<date year='2019' /> <author surname="Fioccola" initials="G">
</front> <organization showOnFrontPage="true"/>
<seriesInfo name='DOI' value='10.1109/TNET.2019.2950157'/> </author>
</reference> <author surname="Marchetto" initials="G">
<organization showOnFrontPage="true"/>
</author>
<author surname="Sapio" initials="A">
<organization showOnFrontPage="true"/>
</author>
<author surname="Sisto" initials="R">
<organization showOnFrontPage="true"/>
</author>
<date month="December" year="2019"/>
</front>
<refcontent>IEEE/ACM Transactions on Networking, Vol. 27, Issue 6</refc
ontent>
<seriesInfo name="DOI" value="10.1109/TNET.2019.2950157"/>
</reference>
<?rfc include='reference.I-D.song-opsawg-ifit-framework'?> <!-- [I-D.song-opsawg-ifit-framework] IESG state I-D Exists as of 12/13/22 -->
<xi:include href="https://datatracker.ietf.org/doc/bibxml3/reference.I-D.song-op
sawg-ifit-framework.xml"/>
<?rfc include='reference.I-D.ietf-ippm-route'?> <!-- [I-D.ietf-ippm-route] Published as RFC 9198 -->
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.9198.
xml"/>
<?rfc include='reference.RFC.7011'?> <xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7011. xml"/>
</references>
</references> </references>
<section anchor="Appendix" numbered="true" toc="default">
<section title="Example of Monitoring Network and Clusters Partition" anc <name>Example of Monitoring Network and Clusters Partition</name>
hor="Appendix"> <t><xref target="monitored-graph" format="default"/> shows a simple exampl
e of a
<t><xref target="monitored-graph"/> shows a simple example of a
monitoring network graph:</t> monitoring network graph:</t>
<figure anchor="monitored-graph">
<figure anchor="monitored-graph" title="Monitoring Network Graph"> <name>Monitoring Network Graph</name>
<artwork name="" type="" align="left" alt=""><![CDATA[ <artwork name="" type="" align="left" alt=""><![CDATA[
+------+ +------+
<> R6 <>--- <> R6 <>---
/ +------+ / +------+
+------+ +------+ / +------+ +------+ /
<> R2 <>---<> R4 <> <> R2 <>---<> R4 <>
/ +------+ \ +------+ \ / +------+ \ +------+ \
/ \ \ +------+ / \ \ +------+
+------+ / +------+ \ +------+ <> R7 <>--- +------+ / +------+ \ +------+ <> R7 <>---
---<> R1 <>---<> R3 <>---<> R5 <> +------+ ---<> R1 <>---<> R3 <>---<> R5 <> +------+
+------+ \ +------+ \ +------+ \ +------+ \ +------+ \ +------+ \
skipping to change at line 1102 skipping to change at line 990
\ \ <> R8 <>--- \ \ <> R8 <>---
\ \ +------+ \ \ +------+
\ \ \ \
\ \ +------+ \ \ +------+
\ <> R9 <>--- \ <> R9 <>---
\ +------+ \ +------+
\ \
\ +------+ \ +------+
<> R10 <>--- <> R10 <>---
+------+ +------+
]]></artwork> ]]></artwork>
</figure> </figure>
<t>In the monitoring network graph example, it is possible to identify the
<t>In the monitoring network graph example, it is possible to identify clusters partition
the clusters partition by applying this two-step algorithm described in <xref target="ncluster
by applying this two-step algorithm described in <xref target="ncluster ing_algo" format="default"/>.</t>
ing_algo"/>.</t> <t>The first step identifies the following groups:</t>
<t>The first step identifies the following groups:<list style="numbers"
>
<t>Group 1: (R1-R2), (R1-R3), (R1-R10)</t>
<t>Group 2: (R2-R4), (R2-R5)</t>
<t>Group 3: (R3-R5), (R3-R9)</t>
<t>Group 4: (R4-R6), (R4-R7)</t>
<t>Group 5: (R5-R8)</t>
</list></t>
<t>And then, the second step builds the clusters partition (in particular,
we can underline that Groups 2 and 3 connect together, since R5 is in c
ommon):<list style="numbers">
<t>Cluster 1: (R1-R2), (R1-R3), (R1-R10)</t>
<t>Cluster 2: (R2-R4), (R2-R5), (R3-R5), (R3-R9)</t>
<t>Cluster 3: (R4-R6), (R4-R7)</t>
<t>Cluster 4: (R5-R8)</t>
</list></t>
<t>The flow direction here considered is from left to right. For the opp <ul spacing="normal" empty="true"><li>Group 1: (R1-R2), (R1-R3), (R1-R10)<
osite direction, /li>
<li>Group 2: (R2-R4), (R2-R5)</li>
<li>Group 3: (R3-R5), (R3-R9)</li>
<li>Group 4: (R4-R6), (R4-R7)</li>
<li>Group 5: (R5-R8)</li>
</ul>
<t>Then, the second step builds the clusters partition (in particular,
we can underline that Groups 2 and 3 connect together, since R5 is in c
ommon):</t>
<ul spacing="normal" empty="true"><li>Cluster 1: (R1-R2), (R1-R3), (R1-R10
)</li>
<li>Cluster 2: (R2-R4), (R2-R5), (R3-R5), (R3-R9)</li>
<li>Cluster 3: (R4-R6), (R4-R7)</li>
<li>Cluster 4: (R5-R8)</li>
</ul>
<t>The flow direction considered here is from left to right. For the oppos
ite direction,
the same reasoning can be applied, and in this example, you get the the same reasoning can be applied, and in this example, you get the
same clusters partition.</t> same clusters partition.</t>
<t>In the end, the following 4 clusters are obtained:</t>
<t>In the end, the following 4 clusters are obtained:</t> <figure anchor="clusters">
<name>Clusters Example</name>
<figure anchor="clusters" title="Clusters Example"> <artwork name="" type="" align="left" alt=""><![CDATA[
<artwork name="" type="" align="left" alt=""><![CDATA[
Cluster 1 Cluster 1
+------+ +------+
<> R2 <>--- <> R2 <>---
/ +------+ / +------+
/ /
+------+ / +------+ +------+ / +------+
---<> R1 <>---<> R3 <>--- ---<> R1 <>---<> R3 <>---
+------+ \ +------+ +------+ \ +------+
\ \
\ \
skipping to change at line 1188 skipping to change at line 1070
<> R7 <>--- <> R7 <>---
+------+ +------+
Cluster 4 Cluster 4
+------+ +------+
---<> R5 <> ---<> R5 <>
+------+ \ +------+ \
\ +------+ \ +------+
<> R8 <>--- <> R8 <>---
+------+ +------+
]]></artwork> ]]></artwork>
</figure> </figure>
<t>There are clusters with more than two nodes as well as two-node cluster
<t>There are clusters with more than two nodes as well as two-node clust s.
ers.
In the two-node clusters, the loss is on the link (Cluster 4). In the two-node clusters, the loss is on the link (Cluster 4).
In more-than-two-node clusters, the loss is on the cluster, but In more-than-two-node clusters, the loss is on the cluster, but
we cannot know in which link (Cluster 1, 2, or 3).</t> we cannot know in which link (Cluster 1, 2, or 3).</t>
</section>
</section> <section anchor="Acknowledgements" numbered="false" toc="default">
<name>Acknowledgements</name>
<section title="Changes Log">
<t>Changes from RFC 8889 in draft-fioccola-rfc8889bis-00 include:<list style
="symbols">
<t>Minor editorial changes</t>
<t>Removed section on "Examples of application"</t>
</list></t>
<t>Changes in draft-fioccola-rfc8889bis-01 include:<list style="symbols">
<t>Considerations on BUM traffic</t>
<t>Reference to RFC8321bis for the fragmentation part</t>
<t>Revised section on "Delay Measurements on a Single-Packet Basis"</t>
<t>Revised section on "Timing Aspects"</t>
</list></t>
<t>Changes in draft-fioccola-rfc8889bis-02 include:<list style="symbols">
<t>Clarified the formula in the section on "Timing Aspects" to be align
ed with RFC 8321</t>
<t>Considerations on two-way delay measurements in both sections 8.1 an
d 8.2 on delay measurements</t>
<t>Clarified in section 4.1 on "Monitoring Network" that the descriptio
n is done for one direction
but it can easily be extended to all direction</t>
<t>New section on "Results of the Multipoint Alternate Marking Experime
nt"</t>
</list></t>
<t>Changes in draft-fioccola-rfc8889bis-03 include:<list style="symbols">
<t>Moved and renamed section on "Timing Aspects" as "Synchronization an
d Timing"</t>
<t>Renamed old section on "Multipoint Packet Loss" as "Network Packet L
oss"</t>
<t>New section on "Multipoint Packet Loss Measurement"</t>
<t>Renamed section on "Multipoint Performance Measurement" as "Extensio
n of the Method to Multipoint Flows"</t>
</list></t>
<t>Changes in draft-fioccola-rfc8889bis-04/draft-ietf-ippm-rfc8889bis-00
include:<list style="symbols">
<t>Revised section 5.1 on "Algorithm for Clusters Partition"</t>
</list></t>
<t>Changes in draft-ietf-ippm-rfc8889bis-01 include:<list style="symbols"
>
<t>New section on "Summary of Changes from RFC 8889"</t>
</list></t>
<t>Changes in draft-ietf-ippm-rfc8889bis-02 include:<list style="symbols"
>
<t>Revised sections on "Single- and Double-Marking Measurement", "Hashi
ng Selection Method" and
"Synchronization and Timing"</t>
<t>Revised references</t>
</list></t>
<t>Changes in draft-ietf-ippm-rfc8889bis-03 include:<list style="symbols"
>
<t>Comments addressed from Last Call review</t>
<t>Renamed section 9 as "Recommendations for Deployment"</t>
</list></t>
<t>Changes in draft-ietf-ippm-rfc8889bis-04 include:<list style="symbols" <t>The authors would like to thank <contact fullname="Martin Duke"/> and <
> contact fullname="Tommy Pauly"/>
for their assistance and their detailed and valuable reviews.</t>
</section>
<t>Comments addressed from Last Call review</t> <section anchor="Contributors" numbered="false" toc="default">
<name>Contributors</name>
</list></t> <author fullname="Greg Mirsky" initials="G" surname="Mirsky">
<organization>Ericsson</organization>
<address>
<postal>
</postal>
<email>gregimirsky@gmail.com</email>
</address>
</author>
</section> <author fullname="Tal Mizrahi" initials="T" surname="Mizrahi">
<organization>Huawei Technologies</organization>
<address>
<postal>
</postal>
<email>tal.mizrahi.phd@gmail.com</email>
</address>
</author>
<author fullname="Xiao Min" initials="X" surname="Min">
<organization>ZTE Corp.</organization>
<address>
<postal>
</postal>
<email>xiao.min2@zte.com.cn</email>
</address>
</author>
</section>
</back> </back>
</rfc> </rfc>
 End of changes. 203 change blocks. 
1040 lines changed or deleted 793 lines changed or added

This html diff was produced by rfcdiff 1.48.