<?xml version="1.0" encoding="UTF-8"?> <!-- [CS] updated by Chris 10/03/22 --> <!DOCTYPE rfcSYSTEM "rfc2629.dtd"[<!-- A set of on-line citation libraries are maintained on the xml2rfc web site. The next line defines an entity named RFC2629, which contains the necessary XML for the reference element, and is used much later in the file. This XML contains an anchor (also RFC2629) which can be used to cross-reference this item in the text. You can also use local file names instead of a URI. The environment variable XML_LIBRARY provides a search path of directories to look at to locate a relative path name for the file. There has to be one entity for each item to be referenced. --><!ENTITYRFC2234 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.2234.xml">nbsp " "> <!ENTITYRFC2629 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.2629.xml">zwsp "​"> <!ENTITYRFC4234 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.4234.xml">nbhy "‑"> <!ENTITYRFC5575 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.5575.xml"> <!-- There is also a library of current Internet Draft citations. It isn't a good idea to actually use one for the template because it might have disappeared when you come to test this template. This is the form of the entity definition <!ENTITY I-D.mrose-writing-rfcs SYSTEM "http://xml.resource.org/public/rfc/bibxml3/reference.I-D.mrose-writing-rfcs.xml"> corresponding to a draft filename draft-mrose-writing-rfcs-nn.txt. The citation will be to the most recent draft in the sequence, and is updated roughly hourly on the web site. For working group drafts, the same principle applies: file name starts draft-ietf-wgname-.. and entity file is reference.I-D.ietf-wgname-... The corresponding entity name is I-D.ietf-wgname-... (I-D.mrose-writing-rfcs for the other example). Of course this doesn't change when the draft version changes. --> <!-- Fudge for XMLmind which doesn't have this built in --> <!ENTITY nbsp " ">wj "⁠"> ]><!-- Extra statement used by XSLT processors to control the output style. --> <?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?> <!-- Processing Instructions can be placed here but if you are editing with XMLmind (and maybe other XML editors) they are better placed after the rfc element start tag as shown below. --> <!-- Information about the document. category values: std, bcp, info, exp, and historic For Internet-Drafts, specify attribute "ipr". (ipr values are: full3667, noModification3667, noDerivatives3667), Also for Internet-Drafts, can specify values for attributes "docName" and, if relevant, "iprExtract". Note that the value for iprExtract is the anchor attribute value of a section (such as a MIB specification) that can be extracted for separate publication, and is only useful whenhe value of "ipr" is not "full3667". --> <!-- TODO: verify which attributes are specified only by the RFC editor. It appears that attributes "number", "obsoletes", "updates", and "seriesNo" are specified by the RFC editor (and not by the document author). --><rfc xmlns:xi="http://www.w3.org/2001/XInclude" category="std" submissionType="IETF" consensus="true" ipr="trust200902" docName="draft-ietf-ippm-rfc8321bis-04" number="9341" obsoletes="8321"> <!-- Processing Instructions- PIs (for a complete list and description, see file http://xml.resource.org/authoring/README.html and below... --> <!-- Some of the more generally applicable PIs that most I-Ds might want to use --> <!-- Try to enforce the ID-nits conventions and DTD validity --> <?rfc strict="yes" ?> <!-- Items used when reviewing the document --> <?rfc comments="no" ?> <!-- Controls display of <cref> elements --> <?rfc inline="no" ?> <!-- When no, put comments at end in comments section, otherwise, put inline --> <?rfc editing="no" ?> <!-- When yes, insert editing marks: editing marks consist of a string such as <29> printed in the blank line at the beginning of each paragraph of text. --> <!-- Create Table of Contents (ToC) and set some options for it. Note the ToC may be omitted for very short documents,but idnits insists on a ToC if the document has more than 15 pages. --> <?rfc toc="yes"?> <?rfc tocompact="yes"?> <!-- If "yes" eliminates blank lines before main section entries. --> <?rfc tocdepth="3"?> <!-- Sets the number of levels of sections/subsections... in ToC --> <!-- Choose the options for the references. Some like symbolic tags in the references (and citations) and others prefer numbers. The RFC Editor always uses symbolic tags. The tags used are the anchor attributes of the references. --> <?rfc symrefs="yes"?> <?rfc sortrefs="yes" ?> <!-- If "yes", causes the references to be sorted in order of tags. This doesn't have any effect unless symrefs is "yes" also. --> <!-- These two save paper: Just setting compact to "yes" makes savings by not starting each main section on a new page but does not omit the blank lines between list items. If subcompact is also "yes" the blank lines between list items are also omitted. --> <?rfc compact="yes" ?> <?rfc subcompact="no" ?> <!-- end of list of popular I-D processing instructions --> <!-- ***** FRONT MATTER ***** -->updates="" xml:lang="en" tocInclude="true" tocDepth="3" symRefs="true" sortRefs="true" version="3"> <front><!-- The abbreviated title is used in the page header - it is only necessary if the full title is longer than 42 characters --><title abbrev="AltMark">Alternate-Marking Method</title><!-- add 'role="editor"' below for the editors if appropriate --><seriesInfo name="RFC" value="9341"/> <author fullname="Giuseppe Fioccola" initials="G." role="editor" surname="Fioccola"> <organization>Huawei Technologies</organization> <address> <postal> <street>Riesstrasse, 25</street> <city>Munich</city> <code>80992</code> <region/> <country>Germany</country> </postal> <email>giuseppe.fioccola@huawei.com</email> </address> </author> <author fullname="Mauro Cociglio" initials="M." surname="Cociglio"> <organization>Telecom Italia</organization> <address> <postal><street></street> <city></city> <code></code> <country></country><street/> <city/> <code/> <country/> </postal> <email>mauro.cociglio@outlook.com</email> </address> </author> <author fullname="Greg Mirsky" initials="G." surname="Mirsky"> <organization>Ericsson</organization> <address> <postal> <street/> <city/> <region/> <code/><country></country><country/> </postal> <email>gregimirsky@gmail.com</email> </address> </author> <author fullname="Tal Mizrahi" initials="T." surname="Mizrahi"> <organization>Huawei Technologies</organization> <address> <postal><street></street> <city></city> <country></country><street/> <city/> <country/> </postal> <email>tal.mizrahi.phd@gmail.com</email> </address> </author> <author fullname="Tianran Zhou" initials="T." surname="Zhou"> <organization>Huawei Technologies</organization> <address> <postal> <street>156 Beiqing Rd.</street> <city>Beijing</city> <code>100095</code> <region/> <country>China</country> </postal> <email>zhoutianran@huawei.com</email> </address> </author> <dateyear="2022"/> <!-- month="March" is no longer necessary note also, day="30" is optional --> <!-- WARNING: If the month and year are the current ones, xml2rfc will fill in the day for you. If only the year is specified, xml2rfc will fill in the current day and month irrespective of the day. This silliness should be fixed in v1.31. --> <!-- Meta-data Declarations --> <!-- Notice the use of & as an escape for & which would otherwise start an entity declaration, whereas we want a literal &. --> <area></area> <!-- WG name at the upperleft corner of the doc, IETF fine for individual submissions. You can also omit this element in which case in defaults to "Network Working Group" - a hangover from the ancient history of the IETF! --> <workgroup></workgroup> <!-- The DTD allows multiple area and workgroup elements but only the first one has any effect on output. --> <!-- You can add <keyword/> elements here. They will be incorporated into HTML output files in a meta tag but they have no effect on text or nroff output. -->year="2022" month="December" /> <area>tsv</area> <workgroup>ippm</workgroup> <keyword>Performance</keyword> <keyword>Measurement</keyword> <keyword>Monitoring</keyword> <keyword>Passive</keyword> <keyword>Hybrid</keyword> <keyword>Loss</keyword> <keyword>Delay</keyword> <keyword>Delay Variation</keyword> <abstract> <t>This document describes the Alternate-Marking technique to perform packet loss, delay, and jitter measurements on live traffic. This technology can be applied in various situations and for different protocols. According to the classification defined in RFC 7799, it could be considered Passive or Hybrid depending on the application. This document obsoletes RFC 8321.</t> </abstract> </front> <middle> <section anchor="intro"title="Introduction">numbered="true" toc="default"> <name>Introduction</name> <t>Most Service Providers' networks carry traffic with contents that are highly sensitive to packet loss <xreftarget="RFC7680"></xref>,target="RFC7680" format="default"/>, delay <xreftarget="RFC7679"></xref>,target="RFC7679" format="default"/>, and jitter <xreftarget="RFC3393"></xref>.</t>target="RFC3393" format="default"/>.</t> <t>Methodologies and tools are therefore needed to monitor and accurately measure network performance, in order to constantly control the quality of experience perceived by the end customers. Performance monitoring also provides useful information for improving network management (e.g., isolation of network problems, troubleshooting, etc.).</t> <t><xreftarget="RFC7799">RFC 7799</xref>target="RFC7799" format="default"/> defines Active,PassivePassive, and Hybrid Methods of Measurement. In particular, Active Methods of Measurement depend on a dedicated measurement packet stream; Passive Methods of Measurement are based solely on observations of an undisturbed and unmodified packet stream of interest; Hybrid Methods are Methods of Measurement that use a combination of Active Methods and Passive Methods.</t> <t>This document proposes a performance monitoring technique, calledAlternate-Marking Method,the "Alternate-Marking Method", which is potentially applicable to any kind of packet-based traffic, both point-to-point unicast and multicast, including Ethernet, IP, andMPLS, both unicast and multicast.MPLS. The methodaddressesprimarilypacket lossaddresses packet-loss measurement, but it can be easily extended to one-way or two-way delay and delay variation measurements as well.</t> <t>The Alternate-Marking methodology, described in this document, allows the synchronization of the measurements at different points by dividing the packet flow into batches. So it is possible to get coherent counters and timestamps in every marking period and therefore measure theperformance metricsPerformance Metrics for the monitored flow.</t> <t>The method has been explicitly designed for Passive or Hybrid measurements as stated in <xreftarget="RFC8321"></xref>.target="RFC8321" format="default"/>. But, according to the definitions of <xreftarget="RFC7799">RFC 7799</xref>,target="RFC7799" format="default"/>, the Alternate-Marking Method can be classified as Hybrid Type I. Indeed,the Alternate-MarkingAlternate Marking can be implemented by using reserved bits in the protocolheaderheader, and the change in value of these marking bits at the domain edges (and not along the path) is formally considered a modification of the stream of interest.</t> <t>It is worth mentioning that this is a methodology document, so the mechanism that can be used to transmit the counters and the timestamps is out of scope here. Additional details about the applicability of the Alternate-Marking methodology are described in <xreftarget="IEEE-Network-PNPM"/>.</t>target="IEEE-NETWORK-PNPM" format="default"/>.</t> <sectiontitle="Summarynumbered="true" toc="default"> <name>Summary of Changes from RFC8321">8321</name> <t>This document defines the Alternate-Marking Method, addressing ambiguities and building on its experimental phase that was based on the original specification <xreftarget="RFC8321"></xref>.</t>target="RFC8321" format="default"/>.</t> <t>The relevant changesare:<list style="symbols"> <t>Addedare:</t> <ul spacing="normal"> <li>Added the recommendations about the methods to employ in case one or two flag bits are available for marking (<xreftarget="finding"></xref>).</t> <t>Changedtarget="finding" format="default"/>).</li> <li>Changed the structure to improve thereadability.</t> <t>Removedreadability.</li> <li>Removed the wording about the initial experiments of the method and considerations that no longerapply.</t> <t>Extendedapply.</li> <li>Extended the description of detailed aspects of the methodology,e.g.e.g., synchronization, timing, packet fragmentation, and marked and unmarked traffichandling.</t> </list></t>handling.</li> </ul> <t>It is important to note that all the changes are totally backward compatible with <xreftarget="RFC8321"></xref>target="RFC8321" format="default"/> and no new additional technique has been introduced in this document compared to <xreftarget="RFC8321"></xref>.</t>target="RFC8321" format="default"/>.</t> </section> <sectiontitle="Requirements Language">numbered="true" toc="default"> <name>Requirements Language</name> <t> The key words"MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY","<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>", "<bcp14>REQUIRED</bcp14>", "<bcp14>SHALL</bcp14>", "<bcp14>SHALL NOT</bcp14>", "<bcp14>SHOULD</bcp14>", "<bcp14>SHOULD NOT</bcp14>", "<bcp14>RECOMMENDED</bcp14>", "<bcp14>NOT RECOMMENDED</bcp14>", "<bcp14>MAY</bcp14>", and"OPTIONAL""<bcp14>OPTIONAL</bcp14>" in this document are to be interpreted as described inBCP 14BCP 14 <xreftarget="RFC2119" />target="RFC2119"/> <xreftarget="RFC8174" />target="RFC8174"/> when, and only when, they appear in all capitals, as shownhere.</t>here. </t> </section> </section> <section anchor="brief"title="Overviewnumbered="true" toc="default"> <name>Overview of theMethod">Method</name> <t>In order to performpacket losspacket-loss measurements on a production traffic flow, different approaches exist. The most intuitive one consists in numbering the packets so that each router that receives the flow can immediately detect a packet that is missing. This approach, though very simple in theory, is not simple to achieve: it requires the insertion of a sequence number into each packet, and the devices must be able to extract the number and check it in real time. Such a task can be difficult to implement on live traffic: if UDP is used as the transport protocol, the sequence number is not available; on the other hand, if a higher-layer sequence number (e.g., in the RTP header) is used, extracting that information from each packet and processing it in real time could overload the device.</t> <t>An alternate approach is to count the number of packets sent on one end, count the number of packets received on the other end, and compare the two values. This operation is much simpler to implement, but it requires the devices performing the measurement to be in sync: in order to compare two counters, it is required that they refer exactly to the same set of packets. Since a flow is continuous and cannot be stopped when a counter has to be read, it can be difficult to determine exactly when to read the counter. A possible solution to overcome this problem is to virtually split the flow in consecutive blocks by periodically inserting a delimiter so that each counter refers exactly to the same block of packets. The delimiter could be, for example, a special packet inserted artificially into the flow. However, delimiting the flow using specific packets has some limitations. First, it requires generating additional packets within the flow and requires the equipment to be able to process those packets. In addition, the method is vulnerable to out-of-order reception of delimiting packets and, to a lesser extent, to their loss.</t> <t>The method proposed in this document follows the second approach, but it doesn't use additional packets to virtually split the flow in blocks. Instead, it "marks" the packets so that the packets belonging to the same block will have the same notional "color", whilst consecutive blocks will have different colors. Each change of color represents a sort of auto-synchronization signal that enhances the consistency of measurements taken by different devices along the path.</t> <t><xreftarget="Measurements"></xref>target="Measurements" format="default"/> represents a very simple network and shows how the method can be used to measure packet loss on different network segments: by enabling the measurement on several interfaces along the path, it is possible to perform link monitoring, node monitoring, or end-to-end monitoring. The method is flexible enough to measure packet loss on any segment of the network and can be used to isolate the faulty element.</t> <figureanchor="Measurements" title="Available Measurements"> <artwork><![CDATA[anchor="Measurements"> <name>Available Measurements</name> <artwork name="" type="" align="left" alt=""><![CDATA[ Traffic Flow ========================================================> +------+ +------+ +------+ +------+ ---<> R1 <>-----<> R2 <>-----<> R3 <>-----<> R4 <>--- +------+ +------+ +------+ +------+ . . . . . . . . . . . . . <------> <-------> . . Node Packet Loss Link Packet Loss . . . <---------------------------------------------------> End-to-End Packet Loss ]]></artwork> </figure> </section> <section anchor="detailed"title="Detailedtoc="include" numbered="true"> <name>Detailed Description of theMethod" toc="include">Method</name> <t>This section describes, in detail, how the method operates. A special emphasis is given to the measurement of packet loss, which represents the core application of the method, but applicability to delay and jitter measurements is also considered.</t> <section anchor="ploss"title="Packet Loss Measurement">numbered="true" toc="default"> <name>Packet-Loss Measurement</name> <t>The basic idea is to virtually split traffic flows into consecutive blocks: each block represents a measurable entity unambiguously recognizable by all network devices along the path. By counting the number of packets in each block and comparing the values measured by different network devices along the path, it is possible to measure if packet loss occurred in any single block between any two points.</t> <t>As discussed in the previous section, a simple way to create the blocks is to "color" the traffic (two colors aresufficient),sufficient) so that packets belonging to alternate consecutive blocks will have different colors. Whenever the color changes, the previous block terminates and the new one begins. Hence, all the packets belonging to the same block will have the samecolorcolor, and packets of different consecutive blocks will have different colors. The number of packets in each block depends on the criterion used to create theblocks:<list style="symbols"> <t>ifblocks:</t> <ul spacing="normal"> <li>if the color is switched after a fixed number of packets, then each block will contain the same number of packets (except for any losses); and</t> <t>if</li> <li>if the color is switched according to a fixed timer, then the number of packets may be different in each block depending on the packetrate.</t> </list></t>rate.</li> </ul> <t>The use of a fixed timer for the creation of blocks isREQUIRED<bcp14>REQUIRED</bcp14> when implementing this specification. The switching after a fixed number of packets is an additional possibility, but its detailed specification is out of scope. An example of application is in <xref target="I-D.ietf-ippm-explicit-flow-measurements" format="default"/>.</t> <t>The following figure shows how a flow appears when it is split into traffic blocks with colored packets.</t> <figureanchor="coloring" title="Traffic Coloring"> <artwork><![CDATA[A:anchor="coloring"> <name>Traffic Coloring</name> <artwork name="" type="" align="left" alt=""><![CDATA[A: packet with A coloring B: packet with B coloring | | | | | | | Traffic Flow | | -------------------------------------------------------------------> BBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA -------------------------------------------------------------------> ... | Block 5 | Block 4 | Block 3 | Block 2 | Block 1 | | | | | ]]></artwork> </figure> <t><xreftarget="FIG_simple_network"></xref>target="FIG_simple_network" format="default"/> shows how the method can be used to measure link packet loss between two adjacent nodes.</t> <t>Referring to the figure, let's assume we want to monitor the packet loss on the link between two routers: router R1 and router R2. According to the method, the traffic is colored alternatively with two different colors: A and B. Whenever the color changes, the transition generates a sort of square-wave signal, as depicted in the following figure.</t> <figureanchor="FIG_simple_network" title="Computationanchor="FIG_simple_network"> <name>Computation of Link PacketLoss"> <artwork><![CDATA[Loss</name> <artwork name="" type="" align="left" alt=""><![CDATA[ Color A ----------+ +-----------+ +---------- | | | | Color B +-----------+ +-----------+ Block n ... Block 3 Block 2 Block 1 <---------> <---------> <---------> <---------> <---------> Traffic Flow ===========================================================> Color ...AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA... ===========================================================> ]]></artwork> </figure><t></t><t/> <t>Traffic coloring can be done by R1 itself if the traffic is not already colored. R1 needs two counters, C(A)R1 and C(B)R1, on its egress interface: C(A)R1 counts the packets with color A and C(B)R1 counts those with color B. As long as traffic is colored as A, only counter C(A)R1 will be incremented, while C(B)R1 is not incremented; conversely, when the traffic is colored as B, only C(B)R1 is incremented. C(A)R1 and C(B)R1 can be used as reference values to determine the packet loss from R1 to any other measurement point down the path. Router R2, similarly, will need two counters on its ingress interface, C(A)R2 and C(B)R2, to count the packets received on that interface and colored with A and B, respectively. When an A block ends, it is possible to compare C(A)R1 and C(A)R2 and calculate the packet loss within the block; similarly, when the successive B block terminates, it is possible to compare C(B)R1 with C(B)R2, and so on, for every successive block.</t> <t>Likewise, by using two counters on the R2 egress interface, it is possible to count the packets sent out of the R2 interface and use them as reference values to calculate the packet loss from R2 to any measurement point downstream from R2.</t> <t>The length of the blocks can be chosen large enough to simplify the collection and the comparison of measures taken by different network devices. It's preferable to read the value of the counters not immediately after the color switch: some packets could arrive out of order and increment the counter associated with the previous block (color), so it is worth waiting for some time. A safe choice is to wait L/2 time units (where L is the duration for each block) after the color switch, to read the counter of the previous color (<xreftarget="sync-timing"/>).target="sync-timing" format="default"/>). The drawback is that the longer the duration of the block, the less frequently the measurement can be taken.</t> <t>Two different strategies that can be used when implementing themethod:<list style="symbols"> <t>flow-based: themethod are:</t> <dl> <dt>flow-based: </dt> <dd>the flow-based strategy is used whenwell definedwell-defined traffic flows need to be monitored. According to this strategy, only the specified flow is colored. Counters forpacket losspacket-loss measurements can be instantiated for each single flow, or for the set as a whole, depending on the desired granularity. With this approach,thereit isthe necessitynecessary to know in advance the path followed by flows that are subject to measurement. Path rerouting and trafficload-balancingload balancing need to be taken intoaccount.</t> <t>link-based: measurementsaccount. </dd> <dt>link-based: </dt> <dd>measurements are performed on all the traffic on a link-by-link basis. The link could be a physical link or a logical link. Counters could be instantiated for the traffic as a whole or for each traffic class (in case it is desired to monitor each class separately), but in the second case, two counters are needed for eachclass.</t> </list></t>class. </dd> </dl> <t>The flow-based strategy isREQUIRED<bcp14>REQUIRED</bcp14> when implementing this specification. It requires the identification of the flow to be monitored and the discovery of the path followed by the selected flow. It is possible to monitor a single flow or multiple flows grouped together, but in this case, measurement is consistent only if all the flows in the group follow the same path. Moreover, if a measurement is performed by grouping many flows, it is not possible to determine exactly which flow was affected by packet loss. In order to have measures per single flow, it is necessary to configure counters for each specific flow. Once the flow(s) to be monitored has been identified, it is necessary to configure the monitoring on the proper nodes. Configuring the monitoring means configuring the rule to intercept the traffic and configuring the counters to count the packets. To have just an end-to-end monitoring, it is sufficient to enable the monitoring on the first- and last-hop routers of the path: the mechanism is completely transparent to intermediate nodes and independent of the path followed by traffic flows. On the contrary, to monitor the flow on a hop-by-hop basis along its whole path, it is necessary to enable the monitoring on every node from the source to the destination. In case the exact path followed by the flow is not known a priori (i.e., the flow has multiple paths to reach the destination), it is necessary to enable the monitoring on every path: counters on interfaces traversed by the flow will report packet count, whereas counters on other interfaces will be null.</t> </section> <section anchor="one-way_delay"title="One-Waynumbered="true" toc="default"> <name>One-Way DelayMeasurement">Measurement</name> <t>The same principle used to measure packet loss can be applied also to one-way delay measurement. There are two methodologies, as described hereinafter.</t> <t>Note that, for all the one-way delay alternatives described in the next sections, by summing the one-way delays of the two directions of a path, it is always possible to measure the two-way delay (round-trip "virtual" delay). The Network Time Protocol (NTP) <xreftarget="RFC5905"></xref>target="RFC5905" format="default"/> or the IEEE 1588 Precision Time Protocol(As(PTP) <xref target="IEEE-1588" format="default"/> (as discussed in the previoussection, ) <xref target="IEEE-1588"/>section) can be used for the timestamp formats depending on the needed precision.</t> <section anchor="single-marking"title="Single-Marking Methodology">numbered="true" toc="default"> <name>Single-Marking Methodology</name> <t>The alternation of colors can be used as a time reference to calculate the delay. Whenever the color changes (which means that a new block has started), a network device can store the timestamp of the first packet of the new block; that timestamp can be compared with the timestamp of the same packet on a second router to compute packet delay. When looking at <xreftarget="coloring"></xref>,target="coloring" format="default"/>, R1 stores the timestamp TS(A1)R1 when it sends the first packet of block 1 (A-colored), the timestamp TS(B2)R1 when it sends the first packet of block 2 (B-colored), and so on for every other block. R2 performs the same operation on the receiving side, recording TS(A1)R2, TS(B2)R2, and so on. Since the timestamps refer to specific packets (the first packet of each block), in the case where no packet loss or misordering exists, we would be sure that timestamps compared to compute delay refer to the same packets. By comparing TS(A1)R1 with TS(A1)R2 (and similarly TS(B2)R1 with TS(B2)R2, and so on), it is possible to measure the delay between R1 and R2. In order to have more measurements, it is possible to take and store more timestamps, referring to other packets within each block. The number of measurements could be increased by considering multiple packets in theblock:block; for instance, a timestamp could be taken every N packets, thus generating multiple delay measurements. Taking this to the limit, in principle, the delay could be measured for each packet by taking and comparing the corresponding timestamps (possible but impractical from an implementation point of view).</t> <t>In order to coherently compare timestamps collected on different routers, the clocks on the network nodesMUST<bcp14>MUST</bcp14> be in sync (<xreftarget="sync-timing"/>).target="sync-timing" format="default"/>). Furthermore, a measurement is valid only if no packet loss occurs and if packet misordering can be avoided; otherwise, the first packet of a block on R1 could be different from the first packet of the same block on R2 (for instance, if that packet is lost between R1 and R2 or it arrives after the next one). Since packet misordering is generallyundetectableundetectable, it is not possible to check whether the first packet on R1 is the same onR2R2, and this is part of the intrinsic error in this measurement.</t> <section anchor="meand"title="Mean Delay">numbered="true" toc="default"> <name>Mean Delay</name> <t>The method previously exposed for measuring the delay is sensitive to out-of-order reception of packets. In order to overcome this problem, an approach based on the concept of mean delay can be considered. The mean delay is calculated by considering the average arrival time of the packets within a single block. The network device locally stores a timestamp for each packet received within a single block: summing all the timestamps and dividing by the total number of packets received, the average arrival time for that block of packets can be calculated. By subtracting the average arrival times of two adjacent devices, it is possible to calculate the mean delay between those nodes. This method greatly reduces the number of timestamps that have to be collected (only one per block for each networkdevice)device), and it is robust toout&nbhy;of&nbhy;orderout-of-order packets with only a small error introduced in case of packet loss. But, when computing the mean delay, the measurement error could be augmented by accumulating the measurement error of a lot of packets. Additionally, it only gives one measure for the duration of the block, and it doesn't give the minimum, maximum, and median delay values <xreftarget="RFC6703"></xref>.target="RFC6703" format="default"/>. This limitation could be overcome by reducing the duration of the block (for instance, from minutes to seconds), which implies a highly optimized implementation of the method. For this reason, the mean delay calculation may not be so viable in some cases.</t> </section> </section> <section anchor="double-marking"title="Double-Marking Methodology">numbered="true" toc="default"> <name>Double-Marking Methodology</name> <t>As mentioned above, the Single-Marking methodology for one-way delay measurement has some limitations, since it is sensitive to out-of-order reception ofpacketspackets, and even the mean delay calculation is limited because it doesn't give information about the delay value's distribution for the duration of the block. Actually, it may be useful to have not only the mean delay but also the minimum, maximum, and median delay values and, in wider terms, to know more about the statistical distribution of delay values. So, in order to have more information about the delay and to overcome out-of-order issues, a different approach can beintroducedintroduced, and it is based on a Double-Marking methodology.</t> <t>Basically, the idea is to use the first marking to create the alternate flow and, within this colored flow, a second marking to select the packets for measuring delay/jitter. The first marking is needed for packet loss and may be used for mean delay measurement. The second marking creates a new set of marked packets that are fully identified over thenetwork,network so that a network device can store the timestamps of these packets. These timestamps can be compared with the timestamps of the same packets on the next node to compute packet delay values for each packet. The number of measurements can be easily increased by changing the frequency of the second marking. But the frequency of the second marking must not be too high in order to avoid out-of-order issues. Between packets with the second marking, there should be an adequate time gap to avoid out-of-order issues and also to have a number of measurement packets that are rate independent. This gap may be, at the minimum, the mean network delay calculated with the previous methodology. Therefore, it is possible to choose a proper time gap to guarantee a fixed number of double-marked packets uniformly spaced in each block. If packets with the second marking are lost, it is easy to recognize the loss since the number of double-marked packets is known for each block. Based on the spacing between these packets, it can also be possible to understand which packet of the second marking sequence has been lost and perform the measurements only for the remaining packets.But,But this may be complicated if more packets are lost. In thiscasecase, an implementation may simply discard the delay measurements for the corrupted block and proceed with the next block.</t> <t>An efficient and robust mode is to select a single packet with the second marking for eachblock,block; in thiswayway, there is no time gap to consider between the double-marked packets to avoid their reorder. In addition, it is also easier to identify the only double-marked packet in each block and skip the delay measurement for the block if it is lost.</t> <t>The Double-Marking methodology can also be used to get more statistics of delay extent data, e.g., percentiles, variance, and median delay values. Indeed, a subset of batch packets is selected for extensive delay calculation by using the secondmarkingmarking, and it is possible to perform a detailed analysis on these double-marked packets. It is worth noting that there are classic algorithms for median and variance calculation, but they are out of the scope of this document. The conventional range (maximum-minimum) should be avoided for several reasons, including stability of the maximum delay due to the influence by outliers. In this regard, <xreftarget="RFC5481">RFC 5481</xref>, Section 6.5target="RFC5481" format="default" sectionFormat="of" section="6.5"/> highlights how the 99.9th percentile of delay and delay variation is more helpful to performance planners.</t> </section> </section> <sectiontitle="Delaynumbered="true" toc="default"> <name>Delay VariationMeasurement">Measurement</name> <t>Similar to one-way delay measurement (both for Single Marking and Double Marking), the method can also be used to measure the inter-arrival jitter. We refer to the definition in <xreftarget="RFC3393">RFC 3393</xref>.target="RFC3393" format="default"/>. The alternation of colors, for a Single-Marking Method, can be used as a time reference to measure delay variations. In case of Double Marking, the time reference is given by the second-marked packets. Considering the example depicted in <xreftarget="coloring"></xref>,target="coloring" format="default"/>, R1 stores the timestamp TS(A)R1 whenever it sends the first packet of a block, and R2 stores the timestamp TS(B)R2 whenever it receives the first packet of a block. The inter-arrival jitter can be easily derived from one-way delay measurement, by evaluating the delay variation of consecutive samples.</t> <t>The concept of mean delay can also be applied to delay variation, by evaluating the average variation of the interval between consecutive packets of the flow from R1 to R2.</t> </section> </section> <sectiontitle="Alternate-Marking Functions">numbered="true" toc="default"> <name>Alternate-Marking Functions</name> <sectiontitle="Markingnumbered="true" toc="default"> <name>Marking thePackets">Packets</name> <t>The coloring operation is fundamental in order to create packet blocks and marked packets. This implies choosing where to activate the coloring and how to color the packets.</t> <t>In case of flow-based measurements, the flow to monitor can be defined by a set of selection rules (e.g., header fields) used to match a subset of the packets; in this way, it is possible to control the number of nodes involved, the path followed by the packets, and the size of the flows. It is possible, in general, to have multiple coloring nodes or a single coloring node that is easier to manage and doesn't raise any risk of conflict. Coloring in multiple nodes can be done, and the requirement is that the coloring must change periodically between the nodes according to the timing considerations in <xreftarget="sync-timing"/>;target="sync-timing" format="default"/>; so every node that is designated as a measurement point along the path should be able to identify unambiguously the colored packets. Furthermore, <xreftarget="I-D.ietf-ippm-rfc8889bis"target="RFC9342" format="default"/> generalizes the coloring for multipoint-to-multipoint flow. In addition, it can be advantageous to color the flow as close as possible to the source because it allows an end-to-end measure if a measurement point is enabled on the last-hop router as well.</t> <t>For link-based measurements, all traffic needs to be colored when transmitted on the link. If the traffic had already been colored, then it has to be re-colored because the color must be consistent on the link. This means that each hop along the path must (re-)color the traffic; the color is not required to be consistent along different links.</t> <t>Traffic coloring can be implemented by setting specific flags in the packet header and changing the value of that bit periodically. How to choose the marking field depends on the application and is out of scope here.</t> </section> <sectiontitle="Countingnumbered="true" toc="default"> <name>Counting and TimestampingPackets">Packets</name> <t>For flow-based measurements, assuming that the coloring of the packets is performed only by the source nodes, the nodes between source and destination (inclusive) have to count and timestamp the colored packets that they receive and forward: this operation can be enabled on every router along the path or only on a subset, depending on which network segment is being monitored (a single link, a particular metro area, the backbone, or the whole path). Since the color switches periodically between two values, two counters (one for each value) are needed for each flow and for every interface being monitored. The number of timestamps to be stored depends on the method for delay measurement that is applied. Furthermore, <xreftarget="I-D.ietf-ippm-rfc8889bis"target="RFC9342" format="default"/> generalizes the counting for multipoint-to-multipoint flow.</t> <t>In case of link-based measurements, the behavior is similar except that coloring,countingcounting, and timestamping operations are performed on a link-by-link basis at each endpoint of the link.</t> <t>Another important consideration is when to read the counters or when to select the packets to be double-marked for delay measurement. It involves timing aspects to consider that are further described in <xreftarget="sync-timing"/>.</t>target="sync-timing" format="default"/>.</t> </section> <sectiontitle="Datanumbered="true" toc="default"> <name>Data Collection andCorrelation">Correlation</name> <t>The nodes enabled to perform performance monitoring collect the value of the counters and timestamps, but they are not able to directly use this information to measure packet loss and delay, because they only have their own samples.</t> <t>Data collection enables the transmission of the counters and timestamps as soon as it has been read. Data correlation is the mechanism to compare counters and timestamps for packet loss, delay, and delay variation calculation.</t> <t>There are two main possibilities to perform both data collection and correlation depending on the Alternate-Marking application and usecase:<list style="symbols"> <t>Usecase:</t> <ul spacing="normal"> <li>Use of a centralized solution using the Network Management System (NMS) to correlate data. This can be done in Push Mode or Polling Mode. In the first case, each router periodically sends the information to the NMS; in the latter case, it is the NMS that periodically polls routers to collectinformation.</t> <t>Definitioninformation.</li> <li>Definition of a protocol-based distributed solution to exchange values of counters and timestamps between the endpoints. This can be done by introducing a new protocol or by extending the existing protocols (e.g., the Two-Way Active Measurement Protocol (TWAMP) as defined in <xreftarget="RFC5357">RFC 5357</xref>target="RFC5357" format="default"/> or the One-Way Active Measurement Protocol (OWAMP) as defined in <xreftarget="RFC4656">RFC 4656</xref>)target="RFC4656" format="default"/>) in order to communicate the counters and timestamps betweennodes.</t> </list></t>nodes.</li> </ul> <t>In the following paragraphs, an example data correlation mechanism is explained and could be used independently of the adopted solutions.</t> <t>When data is collected on the upstream and downstream nodes, e.g., packet counts forpacket losspacket-loss measurement or timestamps for packet delay measurement, and is periodically reported to or pulled by other nodes or an NMS, a certain data correlation mechanismSHOULD<bcp14>SHOULD</bcp14> be in use to help the nodes or NMS tell whether any two or more packet counts are related to the same block of markers or if any two timestamps are related to the same marked packet.</t> <t>The Alternate-Marking Method described in this document literally splits the packets of the measured flow into different measurement blocks. An implementationMAY<bcp14>MAY</bcp14> use a Block Number (BN) for data correlation. The BNMUST<bcp14>MUST</bcp14> be assigned to each measurement block and associated with each packet count and timestamp reported to or pulled by other nodes or NMSs. When the nodes or NMS see, for example, the same BNs associated with two packet counts from an upstream and a downstream node, respectively, it considers that these two packet counts correspond to the same block. The assumption of this BN mechanism is that the measurement nodes are time synchronized. This requires the measurement nodes to have a certain time synchronization capability (e.g., the NTP <xreftarget="RFC5905"></xref>target="RFC5905" format="default"/> or the IEEE 1588 PTP <xreftarget="IEEE-1588"/>).</t>target="IEEE-1588" format="default"/>).</t> </section> </section> <section anchor="sync-timing"title="Synchronizationnumbered="true" toc="default"> <name>Synchronization andTiming">Timing</name> <t>Color switching is the reference for all the network devices acting as measurement points, and the only requirement to be achieved is that they have to recognize the right batch along the path in order to get the related information of counters and timestamps.</t> <t>In general, clocks in network devices are not accurate and for this reason, there is a clock error between the measurement points R1 and R2. And, to implement the methodology, they must be synchronized to the same clock reference with an adequate accuracy in order to guarantee that all network devices consistently match the marking bit to the correct block. Additionally, in practice, besides clock errors, packet reordering is also common in a packet network due to equal-cost multipath (ECMP). In particular, the delay between measurement points is the main cause ofout of orderout-of-order packets because each packet can be delayed differently. If the block is sufficiently large, packet reordering occurs only at the edge of adjacentblocksblocks, and it can be easy to assign reordered packets to the right interval blocks.</t> <t>In summary, we need to take into account two contributions: clock error between network devices and the interval we need to wait to avoid packets being out of order because of network delay.</t> <t>The following figure explains bothissues.</t>issues:</t> <figureanchor="timing" title="Timing Aspects"> <artwork><![CDATA[anchor="timing"> <name>Timing Aspects</name> <artwork name="" type="" align="left" alt=""><![CDATA[ ...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB... |<======================================>| | L | ...=========>|<==================><==================>|<==========... | L/2 L/2 | |<===>| |<===>| d | | d |<==========================>| available counting interval ]]></artwork> </figure><t>Where<t>where L is the time duration of each block.</t> <t>It is assumed that all network devices are synchronized to a common reference time with an accuracy of +/- A/2. Thus, the difference between the clock values of any two network devices is bounded by A.</t> <t>The network delay between the network devices can be represented as a normal distribution and 99.7% of the samples are within 3 standarddeviationdeviations of the average.</t> <t>The guard band d is given by:</t><t>d<artwork><![CDATA[ d = A + D_avg +3*D_stddev,</t>3*D_stddev, ]]></artwork> <t>where A is the clock accuracy, D_avg is the average value of the network delay between the network devices, and D_stddev is the standard deviation of the delay.</t> <t>The available counting interval is L -2d that2d, which must be > 0.</t> <t>The condition thatMUST<bcp14>MUST</bcp14> be satisfied and is a requirement on the synchronization accuracy is:</t><t>d < L/2.</t><artwork><![CDATA[ d < L/2. ]]></artwork> <t>This is the fundamental rule for deciding when to read the counters and when to select the packets to bedouble-marked, indeeddouble-marked; indeed, packetcountercounters and double-marked packetsMUST<bcp14>MUST</bcp14> respectively be taken and chosen within the available counting interval that is not affected by error factors.</t> <t>If the time duration L of each block is not so small, the synchronization requirement could be satisfied even with a relatively inaccurate synchronization method.</t> </section> <section anchor="fragmentation"title="Packet Fragmentation">numbered="true" toc="default"> <name>Packet Fragmentation</name> <t>Fragmentation can be managed with the Alternate-Marking Method using the followingguidance:<list> <t>Markingguidance:</t> <ul empty="true" spacing="normal"> <li>Marking nodesMUST<bcp14>MUST</bcp14> mark all fragments if there are flag bits to use(i.e.(i.e., it is in the specific encapsulation), as if they were separatepackets.</t> <t>Nodespackets.</li> <li>Nodes that fragment packets within the measurement domainSHOULD,<bcp14>SHOULD</bcp14>, if they have the capability to do so, ensure that only one resulting fragment carries the marking bit(s) of the original packet. Failure to do so can introduce errors into themeasurement.</t> <t>Measurementmeasurement.</li> <li>Measurement pointsSHOULD<bcp14>SHOULD</bcp14> simply ignore unmarked fragments and count marked fragments as full packets. However, if resources allow, measurement pointsMAY<bcp14>MAY</bcp14> make note of both marked and unmarked initial fragments and only increment the corresponding counter if (a) other fragments are alsomarked,marked or (b) it observes all other fragments and they areunmarked.</t> </list></t>unmarked.</li> </ul> <t>The proposed approach allows the marking node to mark all the fragments except in the case of fragmentation within the networkdomain,domain; in thateventevent, it is suggested to mark only the first fragment.</t> </section> <section anchor="finding"title="Recommendationsnumbered="true" toc="default"> <name>Recommendations forDeployment">Deployment</name> <t>The methodology described in the previous sections can be applied to various performance measurement problems. The only requirement is to select and mark the flow to be monitored; in this way, packets are batched by the sender, and each batch is alternately marked such that it can be easily recognized by the receiver. <xreftarget="RFC8321"></xref>target="RFC8321" format="default"/> reports experimentalexamplesexamples, and <xreftarget="IEEE-Network-PNPM"/>target="IEEE-NETWORK-PNPM" format="default"/> also includes some information about the deployment experience.</t> <t>Either one or two flag bits might be available for marking in differentdeployments:<list> <t>Onedeployments:</t> <dl> <dt>One flag:packet loss</dt> <dd>packet-loss measurementMUST<bcp14>MUST</bcp14> be done as described in <xreftarget="ploss"></xref>,target="ploss" format="default"/>, while delay measurementMUST<bcp14>MUST</bcp14> be done according to thesingle-marking methodSingle-Marking Method described in <xreftarget="single-marking"></xref>.target="single-marking" format="default"/>. Mean delay (<xreftarget="meand"></xref>) MAYtarget="meand" format="default"/>) <bcp14>MAY</bcp14> also be used but it could imply more computationalload.</t> <t>Twoload. </dd> <dt>Two flags:packet loss</dt> <dd>packet-loss measurementMUST<bcp14>MUST</bcp14> be done as described in <xreftarget="ploss"></xref>,target="ploss" format="default"/>, while delay measurementMUST<bcp14>MUST</bcp14> be done according todouble-marking methodthe Double-Marking Method as described in <xreftarget="double-marking"></xref>.target="double-marking" format="default"/>. In thiscase single-marking MAYcase, Single Marking <bcp14>MAY</bcp14> also be used in combination withdouble-markingDouble Marking, and the two approaches provide slightly different pieces of information that can be combined to have a more robust dataset.</t> </list></t>set. </dd> </dl> <t>There are some operational guidelines to consider for the purpose of deciding to follow the recommendations above and to use one or twoflags.<list> <t>Theflags.</t> <ul empty="false" spacing="normal"> <li>The Alternate-MarkingmethodMethod utilizes specific flags in the packet header, so an important factor is the number of flags available for the implementation. Indeed, if there is only one flagavailableavailable, then there is no otherway, whileway; if two flags areavailableavailable, then the option with two flags is certainly morecomplete.</t> <t>Thecomplete.</li> <li>The duration of the Alternate-Marking period affects the frequency of themeasurementmeasurement, and this is a parameter that can be decided on the basis of the required temporal sampling. But it cannot be freely chosen, as explained in <xreftarget="sync-timing"/>.</t> <t>Thetarget="sync-timing" format="default"/>.</li> <li>The Alternate-Marking methodologies enable packet loss,delaydelay, and delay variation calculation, but in accordance with the method used(e.g. single-marking(e.g., Single Marking ordouble-marking),Double Marking), there is a different kind of information that can be derived. For example, to get more statistics of extent data, the option with two flags is desirable. For this reason, the type of data needed in the specific scenario is an additional element to take intoaccount.</t> <t>Theaccount.</li> <li>The Alternate-MarkingmethodsMethods imply different computational load depending on the method employed. Therefore, the available computational resources on the measurement points can also influence the choice. As an example, mean delay calculation may require moreprocessingprocessing, and it may not be the best option to minimize the computationalload.</t> </list></t>load.</li> </ul> <t>The experiment with Alternate-Marking methodologies confirmed the benefits already described in <xreftarget="RFC8321"></xref>.</t>target="RFC8321" format="default"/>.</t> <t>A deployment of the Alternate-Marking Method should also take into account how to handle and recognize marked and unmarked traffic. SinceAlternate-MarkingAlternate Marking normally employs a marking fieldwhichthat is dedicated, reserved, and included in a protocol extension, the measurement points can learn whether the measurement is activated or not by checking if the specific extension is included or not within the packets.</t> <t>It is worth mentioning some relatedwork:work; inparticularparticular, <xreftarget="IEEE-Network-PNPM"/>target="IEEE-NETWORK-PNPM" format="default"/> explains the Alternate-MarkingmethodMethod together with new mechanisms based on hashing techniques.</t> <sectiontitle="Controllednumbered="true" toc="default"> <name>Controlled Domainrequirement">Requirement</name> <t>The Alternate-Marking Method is an example of a solution limited to a controlled domain <xreftarget="RFC8799"></xref>.</t>target="RFC8799" format="default"/>.</t> <t>A controlled domain is a managed network that selects, monitors, and controls access by enforcing policies at the domainboundaries,boundaries in order to discard undesired external packets entering the domain and to check internal packets leaving the domain. It does not necessarily mean that a controlled domain is a single administrative domain or a single organization. A controlled domain can correspond to a single administrative domain or multiple administrative domains under a defined network management. It must be possible to control the domainboundaries,boundaries and use specific precautions to ensure authentication,encryptionencryption, and integrity protection if traffic traverses the Internet.</t> <t>For security reasons, the Alternate-Marking MethodMUST<bcp14>MUST</bcp14> only be applied to controlled domains.</t> </section> </section> <sectiontitle="Compliancenumbered="true" toc="default"> <name>Compliance with Guidelines from RFC6390"> <t>RFC 6390 <xref target="RFC6390"></xref>6390</name> <t><xref target="RFC6390" format="default"/> defines a framework and a process for developing Performance Metrics for protocols above and below the IP layer (such as IP-based applications that operate over reliable or datagram transport protocols).</t> <t>This document doesn't aim to propose a new Performance Metric but rather a new Method of Measurement for a few Performance Metrics that have already been standardized. Nevertheless, it's worth applying guidelines from <xreftarget="RFC6390"></xref>target="RFC6390" format="default"/> to the present document, in order to provide a more complete and coherent description of the proposed method. The mechanisms described in this document use a combination of the Performance Metric Definition template defined inSection 5.4 of<xreftarget="RFC6390"></xref>target="RFC6390" sectionFormat="of" section="5.4" format="default"/> and the Dependencies laid out in Section5.5<xref target="RFC6390" section="5.5" sectionFormat="bare"/> of that document.<list style="symbols"> <t>Metric</t> <ul spacing="normal"> <li>Metric Name / Metric Description: as already stated, this document doesn't propose any new Performance Metrics. On the contrary, it describes a novel method for measuring packet loss <xreftarget="RFC7680"></xref>.target="RFC7680" format="default"/>. The same concept, with small differences, can also be used to measure delay <xreftarget="RFC7679"></xref>target="RFC7679" format="default"/> and jitter <xreftarget="RFC3393"></xref>.target="RFC3393" format="default"/>. The document mainly describes the applicability topacket loss measurement.</t> <t>Methodpacket-loss measurement.</li> <li>Method of Measurement or Calculation: according to the method described in the previous sections, the number of packets lost is calculated by subtracting the value of the counter on the source node from the value of the counter on the destination node. Both counters must refer to the same color. The calculation is performed when the value of the counters is in a steady state. The steady state is an intrinsic characteristic of the marking method counters because the alternation of color makes the counter associated with a color inactive for the duration of a markingperiod.</t> <t>Unitsperiod.</li> <li>Units of Measurement: the method calculates and reports the exact number of packets sent by the source node and not received by the destinationnode.</t> <t>Measurementnode.</li> <li>Measurement Point(s) with Potential Measurement Domain: the measurement can be performed between adjacent nodes, on a per-link basis, or along a multi-hop path, provided that the traffic under measurement follows that path. In case of a multi-hop path, the measurements can be performed bothend-to-endend to end andhop-by-hop.</t> <t>Measurementhop by hop.</li> <li>Measurement Timing: the method has a constraint on the frequency of measurements. This is detailed in <xreftarget="sync-timing"/>,target="sync-timing" format="default"/>, where it is specified that the marking period and the guard band interval are strictly related to each other to avoid out-of-order issues. That is because, in order to perform a measurement, the counter must be in a steady state, and this happens when the traffic is being colored with the alternatecolor.</t> <t>Implementation:color.</li> <li>Implementation: the method uses one or two marking bits to color the packets; this enables the use of policy configurations on the router to color the packets and accordingly configure the counter for each color. The path followed by traffic being measured should be known in advance in order to configure the counters along the path and be able to compare the correctvalues.</t> <t>Verification:values.</li> <li>Verification: the methodology has been tested and deployed experimentally in both lab and operational network scenarios performing packet loss and delay measurements on traffic patterns created by traffic generators together with precision test instruments and networkemulators.</t> <t>Useemulators.</li> <li>Use and Applications: the method can be used to measure packet loss with high precision on live traffic; moreover, by combining end-to-end and per-link measurements, the method is useful to pinpoint the single link that is experiencing lossevents.</t> <t>Reportingevents.</li> <li>Reporting Model: the value of the counters has to be sent to a centralized management system that performs the calculations; such samples must contain a reference to the time interval they referto,to so that the management system can perform the correctcorrelation; thecorrelation. The samples have to be sent while the corresponding counter is in a steady state (within a time interval); otherwise, the value of the sample should be storedlocally.</t> <t>Dependencies:locally.</li> <li>Dependencies: the values of the counters have to be correlated to the time interval they referto.</t> <t>Organizationto.</li> <li>Organization of Results: the Method of Measurement produces singletons, according to the definition of <xreftarget="RFC2330"></xref>.</t> <t>Parameters:target="RFC2330" format="default"/>.</li> <li>Parameters: the main parameters of the method are the information about the flow or the link to be measured, the time interval chosen to alternate the colors and to read thecounterscounters, and the type of method selected forpacket losspacket-loss and delaymeasurements.</t> </list></t>measurements.</li> </ul> </section> <sectiontitle="IANA Considerations">numbered="true" toc="default"> <name>IANA Considerations</name> <t>This document has no IANA actions.</t> </section> <section anchor="seccons"title="Security Considerations">numbered="true" toc="default"> <name>Security Considerations</name> <t>This document specifies a method to perform measurements that does not directly affect Internet security nor applications that run on the Internet. However, implementation of this method must be mindful of security and privacy concerns.</t> <t>There are two types of security concerns: potential harm caused by the measurements and potential harm to the measurements.<list style="symbols"> <t>Harm</t> <ul spacing="normal"> <li>Harm caused by the measurement: the measurements described in this document are Passive, so there are no new packets injected into the network causing potential harm to the network itself and to data traffic. Nevertheless, the method implies modifications on the fly to a header or encapsulation of the data packets: this must be performed in a way that doesn't alter the quality of service experienced by packets subject to measurements and that preserves stability and performance of routers doing the measurements. One of the main security threats inOAMOperations, Administration, and Maintenance (OAM) protocols is network reconnaissance; an attacker can gather information about the network performance by passively eavesdropping on OAM messages. The advantage of the methods described in this document is that the marking bits are the only information that is exchanged between the network devices. Therefore, Passive eavesdropping ondata-planedata plane traffic does not allow attackers to gain information about the networkperformance.</t> <t>Harmperformance.</li> <li>Harm to the Measurement: the measurements could be harmed by routers altering the marking of the packets or by an attacker injecting artificial traffic. Authentication techniques, such as digital signatures, may be used where appropriate to guard against injected traffic attacks. Since the measurement itself may be affected by routers (or other network devices) along the path of IP packets intentionally altering the value of marking bits of packets, as mentioned above, the mechanism specified in this document can be applied just in the context of a controlled domain; thus, the routers (or other network devices) are locallyadministeredadministered, and this type of attack can beavoided.</t> </list></t>avoided.</li> </ul> <t>An attacker that does not belong to the controlled domain can maliciously send marked packets.ButHowever, no problems occur ifAlternate-MarkingAlternate Marking is not supported in the controlleddomain, no problem happens. While if Alternate-Markingdomain. If Alternate Marking is supported in the controlled domain, it isalsonecessary toavoid thatkeep the measurementsare affected and externalfrom being affected; therefore, externally marked packets must bechecked.</t>checked to see if they are marked and eventually filtered or cleared. </t> <t>The precondition for the application of the Alternate-Marking Method is that itMUST<bcp14>MUST</bcp14> be applied in specific controlled domains, thus confining the potential attack vectors within the network domain. A limited administrative domain provides the network administrator with the means to select,monitormonitor, and control the access to the network, making it a trusted domain. In thisregardregard, it is expected to enforce policies at the domain boundaries to filter both external marked packets entering the domain and internal marked packets leaving the domain. Therefore, the trusted domain is unlikely subject to the hijacking of packets since marked packets are processed and used only within the controlled domain. But despite that, leakages may happen for different reasons, such as a failure or a fault. In this case, nodes outside the domain are expected to ignore marked packets since they are not configured to handle it and should not process it.</t> <t>It might be theoretically possible to modulate the marking to serve as a covert channel to be used by an on-path observer. This may affect both the data and management plane, but, here too, the application to a controlled domain helps to reduce the effects.</t> <t>It is worth highlighting that an attacker can't gain information about network performance from a single monitoring point;itthey must use synchronized monitoring points at multiple points on thepath,path because they have to do the same kind of measurement and aggregation that Service Providers usingAlternate-MarkingAlternate Marking must do.</t> <t>Attacks on the data collection and reporting of the statistics between the monitoring points and thenetwork management systemNMS can interfere with the proper functioning of the system. Hence, the channels used to report back flow statisticsMUST<bcp14>MUST</bcp14> be secured.</t> <t>The privacy concerns of network measurement are limited because the method only relies on information contained in the header or encapsulation without any release of user data. Although information in the header or encapsulation is metadata that can be used to compromise the privacy of users, the limited marking technique in this document seems unlikely to substantially increase the existing privacy risks from header or encapsulation metadata. It might be theoretically possible to modulate the marking to serve as a covert channel, but it would have a very low data rate if it is to avoid adversely affecting the measurement systems that monitor the marking.</t> <t>Delay attacks are another potential threat in the context of this document. Delay measurement is performed using a specific packet in each block, marked by a dedicated color bit. Therefore, an on-path attacker can selectively induce synthetic delay only to delay-colored packets, causing systematic error in the delay measurements. As discussed in previous sections, the methods described in this document rely on an underlying time synchronization protocol. Thus, by attacking the time protocol, an attacker can potentially compromise the integrity of the measurement. A detailed discussion about the threats against time protocols and how to mitigate them is presented in <xreftarget="RFC7384">RFC 7384</xref>.</t> </section> <section anchor="Contributors" title="Contributors"> <t>Xiao Min<vspace blankLines="0" /> ZTE Corp.<vspace blankLines="0" /> Email: xiao.min2@zte.com.cn</t> <t>Mach(Guoyi) Chen<vspace blankLines="0" /> Huawei Technologies<vspace blankLines="0" /> Email: mach.chen@huawei.com</t> <t>Alessandro Capello<vspace blankLines="0" /> Telecom Italia<vspace blankLines="0" /> Email: alessandro.capello@telecomitalia.it</t> </section> <section anchor="Acknowledgements" title="Acknowledgements"> <t>The authors would like to thank Alberto Tempia Bonda, Luca Castaldelli and Lianshu Zheng for their contribution to the experimentation of the method.</t> <t>The authors would also thank Martin Duke and Tommy Pauly for their assistance and their detailed and precious reviews.</t>target="RFC7384" format="default"/>.</t> </section><!-- Possibly a 'Contributors' section ... --></middle><!-- *****BACK MATTER ***** --><back><!-- References split to informative and normative --> <references title="Normative References"> <?rfc include='reference.RFC.2119'?> <?rfc include='reference.RFC.8174'?> <?rfc include='reference.RFC.3393'?> <?rfc include='reference.RFC.7679'?> <?rfc include='reference.RFC.7680'?><displayreference target="I-D.ietf-ippm-explicit-flow-measurements" to="EXPLICIT-FLOW-MEASUREMENTS"/> <references> <name>References</name> <references> <name>Normative References</name> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.3393.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7679.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7680.xml"/> </references><references title="Informative References"> <!-- A reference written by by an organization not a persoN. --> <?rfc include='reference.RFC.5905'?><references> <name>Informative References</name> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.5905.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8321.xml"/> <reference anchor="IEEE-1588"> <front> <title>IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems</title> <author> <organization>IEEE</organization> </author> <date/>month="July" year="2008"/> </front> <seriesInfo name="IEEE" value="Std 1588-2008"/> <seriesInfo name="DOI" value="10.1109/IEEESTD.2008.4579760"/> </reference><?rfc include='reference.RFC.8321'?> <?rfc include='reference.I-D.ietf-ippm-rfc8889bis'?> <?rfc include='reference.I-D.ietf-ippm-explicit-flow-measurements'?> <?rfc include='reference.RFC.7384'?> <?rfc include='reference.RFC.8799'?> <?rfc include='reference.RFC.5357'?> <?rfc include='reference.RFC.4656'?> <?rfc include='reference.RFC.5481'?> <?rfc include='reference.RFC.7799'?> <?rfc include='reference.RFC.6703'?> <?rfc include='reference.RFC.6390'?> <?rfc include='reference.RFC.2330'?><!-- [I-D.ietf-ippm-rfc8889bis] in EDIT state as of 11/23/22; companion document RFC YYY1 --> <referenceanchor='IEEE-Network-PNPM'>anchor='RFC9342' target='https://www.rfc-editor.org/info/rfc9342'> <front> <title>Clustered Alternate-Marking Method</title> <author initials="G." surname="Fioccola" fullname="Giuseppe Fioccola" role="editor"> <organization>Huawei Technologies</organization> </author> <author initials="M." surname="Cociglio" fullname="Mauro Cociglio"> <organization>Telecom Italia</organization> </author> <author initials="A." surname="Sapio" fullname="Amedeo Sapio"> <organization>Intel Corporation</organization> </author> <author initials="R." surname="Sisto" fullname="Riccardo Sisto"> <organization>Politecnico di Torino</organization> </author> <author initials="T." surname="Zhou" fullname="Tianran Zhou"> <organization>Huawei Technologies</organization> </author> <date month="December" year="2022"/> </front> <seriesInfo name="RFC" value="9342"/> <seriesInfo name="DOI" value="10.17487/RFC9342"/> </reference> <!-- [I-D.ietf-ippm-explicit-flow-measurements] IESG state I-D Exists as of 11/23/22--> <xi:include href="https://datatracker.ietf.org/doc/bibxml3/reference.I-D.ietf-ippm-explicit-flow-measurements.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7384.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8799.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.5357.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4656.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.5481.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7799.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.6703.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.6390.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2330.xml"/> <reference anchor="IEEE-NETWORK-PNPM" quoteTitle="true" target="https://doi.org/10.1109/MNET.2019.1800152"> <front> <title>AM-PM: Efficient Network Telemetry using Alternate Marking</title><author> <organization>IEEE Network</organization><author surname="Mizrahi" initials="T"> <organization showOnFrontPage="true"/> </author> <author surname="Navon" initials="G"> <organization showOnFrontPage="true"/> </author> <author surname="Fioccola" initials="G"> <organization showOnFrontPage="true"/> </author> <author surname="Cociglio" initials="M"> <organization showOnFrontPage="true"/> </author> <author surname="Chen" initials="M"> <organization showOnFrontPage="true"/> </author> <author surname="Mirsky" initials="G"> <organization showOnFrontPage="true"/> </author> <dateyear='2019' />year="2019" month="July"/> </front> <seriesInfoname='DOI' value='10.1109/MNET.2019.1800152'/>name="IEEE Network" value="Vol. 33, Issue 4"/> <seriesInfo name="DOI" value="10.1109/MNET.2019.1800152"/> </reference> </references> </references> <sectiontitle="Changes Log"> <t>Changes from RFC 8321 in draft-fioccola-rfc8321bis-00 include:<list style="symbols"> <t>Minor editorial changes</t> <t>Replacement ofanchor="Acknowledgements" numbered="false" toc="default"> <name>Acknowledgements</name> <t>The authors would like to thank <contact fullname="Alberto Tempia Bonda"/>, <contact fullname="Luca Castaldelli"/>, and <contact fullname="Lianshu Zheng"/> for their contribution to thesection on "Applications, Implementation, and Deployment" with "Findingexperimentation of theAlternate Marking Implementationsmethod.</t> <t>The authors would also like to thank <contact fullname="Martin Duke"/> and <contact fullname="Tommy Pauly"/> for their assistance andDeployments"</t> <t>Moved advantagestheir detailed andbenefits of the method from "Introduction" toprecious reviews.</t> </section> <section anchor="Contributors" numbered="false" toc="default"> <name>Contributors</name> <author fullname="Xiao Min"> <organization>ZTE Corp.</organization> <address> <postal> <street/> <city/> <region/> <code/> <country/> </postal> <email>xiao.min2@zte.com.cn</email> </address> </author> <author fullname="Mach(Guoyi) Chen"> <organization>Huawei Technologies</organization> <address> <postal> <street/> <city/> <region/> <code/> <country/> </postal> <email>mach.chen@huawei.com</email> </address> </author> <author fullname="Alessandro Capello"> <organization>Telecom Italia</organization> <address> <postal> <street/> <city/> <region/> <code/> <country/> </postal> <email>alessandro.capello@telecomitalia.it</email> </address> </author> </section> <!-- [rfced] Throughout thenew section on "Finding oftext, theAlternate Marking Implementationsfollowing terminology appears to be used inconsistently. Please review these occurrences andDeployments"</t> <t>Removed section on "Hybrid Measurement"</t> </list></t> <t>Changes in draft-fioccola-rfc8321bis-01 include:<list style="symbols"> <t>Considerations on the reference: [IEEE-Network-PNPM]</t> <t>Clarified that thelet us know if/how they may be made consistent. - Single-Marking Method vs. Single-Marking methodbased on a fixed timer is specified inNote: We made thisdocument while the method based on a fixed number of packets is only mentioned but not detailed.</t> <t>Explanation of the intrinsic error in section 3.3.1 on "Single-Marking Methodology"</t> <t>Deleted some partsterm consistent by capitalizing "method" per RFC 8321 (also updated 1 instance insection 4 "Considerations" that no longer apply</t> <t>New section on "Packet Fragmentation"</t> </list></t> <t>Changes in draft-fioccola-rfc8321bis-02 include:<list style="symbols"> <t>Considerations on howRFC-to-be 9342). Please let us know if any further changes are needed. FYI, these terms appear as follows. If any further updates are needed tohandle unmarked traffic in section 5 on "Results ofthe case, please let us know. - Alternate-Marking Method (per 8321) - Alternate-Marking methodology - Alternate MarkingExperiment"</t> <t>Minor rewording in section 4.4 on "Packet Fragmentation"</t> </list></t> <t>Changes in draft-fioccola-rfc8321bis-03 include:<list style="symbols"> <t>Deleted numeric examples in sections on "Packet Loss Measurement" and on "Single-Marking Methodology"</t> <t>New section on "Alternate(no hyphen when not followed by a noun) - Single-Marking Method - Single-Marking methodology - Single MarkingFunctions"</t> <t>Moved sections 3.1.1 on "Coloring the Packets", 3.1.2 on "Counting the Packets" and 3.1.3 on "Collecting Data and Calculating Packet Loss" into the new section on "Alternate(per 8321) - Double MarkingFunctions"</t> <t>Renamed sections 4.1 as "Marking the Packets", 4.2 as "Counting and Timestamping Packets" and 4.3 as "Data Collection and Correlation"</t> <t>Merged old section on "Data Correlation" with section 4.3 on "Data Collection and Correlation"</t> <t>Moved and renamed section on "Timing Aspects" as "Synchronization and Timing"</t> <t>Merged old section on "Synchronization" with section on "Synchronization and Timing"</t> <t>Merged old section on "Packet Reordering" with section on "Synchronization and Timing"</t> </list></t> <t>Changes in draft-fioccola-rfc8321bis-04/draft-ietf-ippm-rfc8321bis-00 include:<list style="symbols"> <t>Revised "Introduction" section</t> <t>Revised sections 4.2 "Counting and Timestamping Packets" and 4.3 on "Data Collection and Correlation"</t> <t>Revised section 5 on "Synchronization and Timing"</t> </list></t> <t>Changes in draft-ietf-ippm-rfc8321bis-01 include:<list style="symbols"> <t>New section on "Summary of Changes from RFC 8321"</t> <t>Revised sections on "Single-Marking Methodology"--> <!-- [rfced] Please review the "Inclusive Language" portion of the online Style Guide <https://www.rfc-editor.org/styleguide/part2/#inclusive_language> and"Double-Marking Methodology"</t> </list></t> <t>Changes in draft-ietf-ippm-rfc8321bis-02 include:<list style="symbols"> <t>Revised section on "Double-Marking Methodology"</t> <t>Revised references</t> </list></t> <t>Changes in draft-ietf-ippm-rfc8321bis-03 include:<list style="symbols"> <t>Comments addressed from Last Call review</t> <t>Renamed section 7let us know if any changes are needed. Note that our script did not flag any words in particular, but this should still be reviewed as"Recommendations for Deployment"</t> </list></t> </section>a best practice. --> </back> </rfc>