Rate Measurement Problem
Statement
AT&T Labs
200 Laurel Avenue South
Middletown,
NJ
07748
USA
+1 732 420 1571
+1 732 368 1192
acmorton@att.com
http://home.comcast.net/~acmacm/
There is a rate measurement scenario which has wide-spread attention
of users and seemingly all industry participants, including regulators.
This memo presents an access rate-measurement problem statement for IP
Performance Metrics. Key aspects require the ability to control packet
size on the tested path and enable asymmetrical packet size testing in a
controller-responder architecture.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119.
There are many possible rate measurement scenarios. This memo
describes one rate measurement problem and presents a rate-measurement
problem statement for IP Performance Metrics (IPPM).
The access-rate scenario or use case has wide-spread attention of
users and seemingly all industry participants, including regulators. It
is being approached with many different measurement methods.
The scope and purpose of this memo is to define the measurement
problem statement for access rate measurement on production networks. We
characterize this scenario as follows:
The Access portion of the network is the focus of this effort.
The user typically subscribes to a service with bi-directional
access partly described by rates in bits per second.
Rates at the edge of the network are several orders of magnitude
less than aggregation and core portions.
Asymmetrical ingress and egress rates are prevalent.
Extremely large scale of access services requires low complexity
devices participating at the user end of the path.
Today, the majority of widely deployed access services achieve rates
less than 100 Mbit/s, and this is the rate-regime for which a solution
is sought now.
This problem statement assumes that the most-likely bottleneck device
or link is adjacent to the remote (user-end) measurement device, or is
within one or two router/switch hops of the remote measurement
device.
Other use cases for rate measurement involve situations where the
packet switching and transport facilities are leased by one operator
from another and the actual capacity available cannot be directly
determined (e.g., from device interface utilization). These scenarios
could include mobile backhaul, Ethernet Service access networks, and/or
extensions of a layer 2 or layer 3 networks. The results of rate
measurements in such cases could be employed to select alternate
routing, investigate whether capacity meets some previous agreement,
and/or adapting the rate of certain traffic sources if a capacity
bottleneck is found via the rate measurement. In the case of aggregated
leased networks, available capacity may also be asymmetric. In these
cases, the tester is assumed to have a sender and receiver location
under their control. We refer to this scenario below as the aggregated
leased network case.
Only active measurement methods will be addressed here, consistent
with the IPPM working group's current charter. Active measurements
require synthetic traffic dedicated to testing, and do not use user
traffic.
The actual path used may influence the rate measurement results for
some forms of access, as it may differ between user and test
traffic.
This issue requires further study to list the likely causes for
this behavior. The possibilities include IP address assignment,
transport protocol used (where TCP packets may be routed differently
from UDP).
Although the user may have multiple instances of network access
available to them, the primary intent is to measure one form of access
at a time. It is plausible that a solution for the single access problem
will be applicable to simultaneous measurement of multiple access
instances, but this is beyond the current scope.
A key consideration is whether active measurements will be conducted
with user traffic present (In-Service Testing), or not present
(Out-of-Service Testing), such as during pre-service testing or
maintenance that interrupts service temporarily. Out-of-Service testing
includes activities described as "service commissioning", "service
activation", and "planned maintenance". Both In-Service and
Out-of-Service Testing are within the scope of this problem.
It is a non-goal to solve the measurement protocol specification
problem in this memo.
It is a non-goal to standardize methods of measurement in this memo.
However, the problem statement will mandate that support for one or more
categories of rate measurement methods and adequate control features for
the methods in the test protocol.
This section lists features of active measurement methods needed to
measure access rates in production networks.
Test coordination between Source and Destination devices through
control messages and other basic capabilities described in the methods
of IPPM RFCs are taken as given (these could be listed
later, if desired).
Most forms of active testing intrude on user performance to some
degree. One key tenet of IPPM methods is to minimize test traffic
effects on user traffic in the production network. Section 5 of lists the problems with high measurement
traffic rates, and the most relevant for rate measurement is the
tendency for measurement traffic to skew the results, followed by the
possibility to introduce congestion on the access link. Obviously,
categories of rate measurement methods that use less active test traffic
than others with similar accuracy SHALL be preferred for In-Service
Testing.
On the other hand, Out-of-Service Tests where the test path shares no
links with In-Service user traffic have none of the congestion or skew
concerns, but must address other practical concerns such as conducting
measurements within a reasonable time from the tester's point of view.
Out-of-Service Tests where some part of the test path is shared with
In-Service traffic MUST respect the In-Service constraints.
The **intended metrics to be measured** have strong influence over
the categories of measurement methods required. For example, using the
terminology of , a it may be possible to
measure a Path Capacity Metric while In-Service if the level of
background (user) traffic can be assessed and included in the reported
result.
The measurement *architecture* MAY be either of one-way (e.g., ) or two-way (e.g., ), but the scale and complexity aspects of
end-user or aggregated access measurement clearly favor two-way (with
low-complexity user-end device and round-trip results collection, as
found in ). However, the asymmetric rates
of many access services mean that the measurement system MUST be able to
assess each direction of transmission. In the two-way architecture, it
is expected that both end devices MUST include the ability to launch
test streams and collect the results of measurements in both (one-way)
directions of transmission (this requirement is consistent with previous
protocol specifications, it is not a unique problem for rate
measurements).
The following paragraphs describe features for the roles of test
packet SENDER, RECEIVER, and results REPORTER.
SENDER:
Ability to generate streams of test packets with various
characteristics as desired (see Section 4). The SENDER may be located at
the user end of the access path, or may be located elsewhere in the
production network, such as at one end of an aggregated leased network
segment.
RECEIVER:
Ability to collect streams of test packets with various
characteristics (as described above), and make the measurements
necessary to support rate measurement at the other end of an end-user
access or aggregated leased network segment.
REPORTER:
Ability to use information from test packets and local processes to
measure delivered packet rates.
The design of rate measurement methods can be divided into two
phases: test stream design and measurement (SENDER and RECEIVER), and a
follow-up phase for analysis of the measurement to produce results
(REPORTER). The measurement protocol that addresses this problem MUST
only serve the test stream generation and measurement functions.
For the purposes of this problem statement, we categorize the many
possibilities for rate measurement stream generation as follows;
Packet pairs, with fixed intra-pair packet spacing and fixed or
random time intervals between pairs in a test stream.
Multiple streams of packet pairs, with a range intra-pair spacing
and inter-pair intervals.
One or more packet ensembles in a test stream, using a fixed
ensemble size in packets and one or more fixed intra-ensemble packet
spacings (including zero).
One or more packet chirps, where intra-packet spacing typically
decreases between adjacent packets in the same chirp and each pair
of packets represents a rate for testing purposes.
For all categories, the test protocol MUST support:
Variable payload lengths among packet streams
Variable length (in packets) among packet streams or
ensembles
Variable header markings among packet streams
Variable number of packets-pairs, ensembles, or streams used in a
test session
are additional variables that the test protocol MUST be able to
communicate.
The test protocol SHALL support test packet ensemble generation
(category 3), as this appears to minimize the demands on measurement
accuracy. Other stream generation categories are OPTIONAL.
Measurements for each test packet transferred between SENDER and
RECEIVER MUST be compliant with the singleton measurement methods
described in IPPM RFCs (these could be listed later, if desired). The
time-stamp information or loss/arrival status for each packet MUST be
available for communication to the protocol entity that collects
results.
Essentially, the test protocol MUST support the measurement features
described in the sections above. This requires:
Communicating all test variables to the Sender and Receiver
Results collection in a one-way architecture
Remote device control for both one-way and two-way
architectures
Asymmetric and/or pseudo-one-way test capability in a two-way
measurement architecture
The ability to control packet size on the tested path and enable
asymmetrical packet size testing in a two-way architecture are
REQUIRED.
The test protocol SHOULD enable measurement of the Capacity metric, either Out-of-Service,
In-Service, or both. Other metrics are
OPTIONAL.
The security considerations that apply to any active measurement of
live networks are relevant here as well. See and .
There may be a serious issue if a proprietary Service Level Agreement
involved with the access network segment provider were somehow leaked in
the process of rate measurement. To address this, test protocols SHOULD
NOT convey this information in a way that could be discovered by
unauthorized parties.
This memo makes no requests of IANA.
Dave McDysan provided comments and text for the aggregated leased use
case. Yaakov Stein suggested many considerations to address, including
the in-service vs. out-of-service distinction and its implication on
test traffic limits.
This Appendix is intended to briefly summarize previous rate
measurement experience. (There is a large body of research on rate
measurement, so there is a question of what to include and what to
omit.)